Every researcher who has submitted papers to reputable journals will be familiar with the peer-review system.
When a paper is received by a peer-reviewed journal, it is first assessed by the editor. If deemed relevant to the journal, it is then sent to specialists (usually at least two) in the field, who assess its validity, originality and innovation and whether the underlying science is sound. Most journals (including Minerals Engineering) adopt the single-blind review system, where the author’s identity is known to the reviewers, but the reviewers’ identities are hidden from the author.
Peer-review is an essential dividing line for judging what is scientific and what is speculation and opinion. Innovative research usually has its own unique features, which are difficult to predict with a check-list and which require expert judgement about their validity, significance and originality.
A major criticism is that in our world of instant electronic communication, the peer-review process is often frustratingly slow, usually taking several weeks, and sometimes months.
As a journal editor of 23 years standing I do find the review process slow and cumbersome at times, but feel that it is the best system that we have, and does help to root out bad science and enhance the quality of papers that are finally accepted for publication. My main frustrations tend to be with reviewers who do not respond to invitations to review, thus losing a precious week, or, more worryingly, those who do agree to review but then fail to provide a report despite several reminders, adding weeks to the review process, as alternative reviewers have to be sought at a late stage.
An independent report found that peer review is widely supported by academics, who overwhelmingly (93%) disagreed in a survey that peer review is unnecessary. The large majority (85%) agreed that peer review greatly helps scientific communication and believed (83%) that without peer review there would be no control. 90% of researchers said that the main area of effectiveness of peer review was in improving the quality of the published paper, and a similar percentage said it had improved their own last published paper, including identifying scientific errors and missing and inaccurate references.
So what are your opinions on the peer-review process? What are your experiences, both good and bad, and how do you think it might be improved? I am particularly interested in the views of authors, but would also like to hear what reviewers and editors think.
Monday 21 March 2011
Subscribe to:
Post Comments (Atom)
.jpg)
I am not an academic but do publish regularly in journals. I think peer review is essential not only in maintaining high standards of publication but in identifying weaknesses and errors in a paper and greatly has helped me as an author improve my work. I do not think I would consider publishing in a journal that did not offer peer review.
ReplyDeleteRob Bowell
Corporate Consultant
SRK Consulting, Cardiff, Wales
I agree.
ReplyDeleteAs an author I have had significant inputs to my work derived from valid and key criticisms which have definitively improved the final result.
As a reviewer I always try to improve the submissions by a positive criticism and suggestions on additional comments or discussions, when it is possible.
I think peer review is absolutely necessary to keep and improve the quality and relevance of papers in a prestigious journal.
Best regards,
Juan Yianatos, Santa Maria University, Chile
By email:
ReplyDeletePlease consider that:
1. no system is perfect
2. a review process in necessary. It is carried out for papers, projects etc
3. most reviewers have also other activities. Research, teaching, project meetings, travel etc. Personally I do most reviews in airports and planes
4. may reviewers believe that they do not want to do it for nothing (this is not my view). They say that when they are asked to evaluate proposals they get something. Even countries of the former eastern block pay 50 € ffor the evaluation of a 10 page proposal within a specific time period, e.g 20 days
5. reviewers who do not have other activities usually delay.
So, if no review process exists, which is the alternative option?
Regarding Minerals Engineering I believe that you can consider that one rejection should stop paper from further review (this is what is done by journal of Hazardous Materials, which indicates also in the invitation letter that rejection rate is something like 70%). You can select some reviewers that you trust and respect their judgement. Several papers then will go quickly to the bin. There are papers, which initially were bad, that are coming back for another review. Probably you can abolish the option of technical paper (or consider it only in selected cases and if they are technically good)
Kostas Komnitsas, Technical University of Crete, Greece
Hi Kostas
ReplyDeleteI think it would be very unwise for any journal editor to reject papers on the basis of only one referee recommending rejection. By the same token, I would not accept a paper if only one referee recommended ‘acceptance in its present form’.
Reviews are to some extent subjective, which is why more than one reviewer is always used and in the event of conflicting reviews, the decision on the paper is in the hands of the editor. It is not unusual for one reviewer to recommend rejection and the other acceptance, and in such cases I would normally bring in a 3rd referee, usually a member of the Editorial Board, who would have access to the two reviewers’ reports, and would act as a mediator.
Regarding payment for reviews, I don’t think any self-respecting scientist would expect that. Being part of the peer-review process should be a responsibility of any serious scientist, and part of the job description of a researcher.
Regarding being too busy, all reviewers have the option of making themselves unavailable at any time. However I have had examples of academics who say they are unable to get involved in the review process, as they are too busy writing papers!!
I am in full support of this system. It is essential to weed out rubbish and to improve the rest. The system is not perfect but is the best that there is.
ReplyDeleteI don't think speed is at all an issue for Minerals Engineering (ME). I have papers in some journals that wait for 1-2 years. Worst I've had is a paper on lava flow that was completed in 2003. It went to 3 journals each waiting a year and never got a review back. For one journal we couldn't even get replies from the editors towards the end. These are too long, but anything measured in handfuls of weeks is fine.
I have a bit more of an issue with the uniformity of the reviewing in general. I see papers in supposedly good journals and shake my head about how some of them get through. Then other really good papers struggle.
For some more "elite" journals, meaning ones with high impact factors and whose volumes of submitted papers are rising sharply I am also finding that the editors narrow the interpretation of the journal scope to be much narrower than the publicised scope. I have had editors apologise that some submitted papers would have been in scope a few years ago but is now not in the scope (even though it has not ostensibly changed according to what is on the web). This is not a problem with ME (which is very good.
There are also far too many papers claiming to be innovative extensions of the numerical methods. Mostly they are near trivial modifications, tested on very narrow ranges of test problems (often only one or two) and which are of such a simple nature that they cannot really demonstrate anything. More than half such papers are just repeats of standard method descriptions that disguise the lack of real content.
These are all questions though about how editors do their job and respond to questions like increasing submission volumes and the ability to get really expert reviewers. The review process itself is OK and the best system I can imagine.
Hope this helps a bit
Paul Cleary, CSIRO, Australia
Peer review is the right way to go, as it improves papers in several ways.
ReplyDeleteHowever, reviewers should be a lot quicker in their response.
I don't see any serious alternative to peer review. Let's make it faster !
Thank you for bringing this to our attention. I'm somewhat surprised that the necessity of the peer review method would be questioned for validity.
ReplyDeletePeer review is the guiding process for training in the scientific method. As practitioners of philosophy at the doctoral level the fundamental purpose of our work is the search for truth. The truths we seek are to provide progress in our fundamental understanding of the physical world in which we live. Our
approach to these endeavors is guided by the scientific method, and our training in the use of this method is the peer review system. The earliest stages of training come from teachers and then professors, however as we become teachers and professors ourselves we never grow outside the realm of requiring training and correction ourselves. This life-long guidance in the method of pursuing understanding of the physical world comes through peers outside ourselves and their feedback is critical to both our development and the progress of science itself.
Since no human being is fully objective in their views, the peer review process itself, as an entity made of human beings, will contain a
fundamental flaw if objective truth is sought. The Sokal affair is a grand example. However, peer review is overwhelmingly positive in the continual refinement of scientific knowledge and progress of science and understanding
of the physical world. When a scientist publishes his/her accomplishments into the public realm any errors in method or conclusion are scrutinized for correction or rejection. The public is trusting that peers trained and
versed in the art of that body of work are acting as gatekeepers to refine or reject that work, such that its truth can be added to our body of knowledge. The peer review examination has proved critically important in recent years as shown by the exposure of flaws in famous work by Hwang Woo-suk in stem cell research and by Fleischmann and Pons in cold fusion research.
I would hope that the editors of Elsevier are just asking for input on this debate and are not seriously considering elimination of the peer review process in their journals. It is always important to question our processes
in terms of their ability to provide better understanding. However, if the primary purpose of questioning peer review is solely based on expediency, then the goal of "genuine contributions to the science and health
communities" has been discarded in favor of financial gain.
Regards,
Hank Rawlins, PhD, P.E.
Montana Process Research, USA
Hi Hank
ReplyDeleteI don't think there is any intention on the part of Elsevier to eliminate peer-review, but merely to seek views, and maybe refine the process.
I cannot think of an alternative that would preserve a journal's intergrity as well as the peer review system developed and tested over so many years (centuries?). Most papers in disciplines like mineral processing probably aim to be archival, rather than answer some more immediate issue as may the case in other fields such as computer science (this from my experience reviewing CVs across engineering). In that case a 'delay' of a few weeks does not seem overly long. The problem of reviewers not indicating availability in timely fashion and others taking time to respond after accepting are issues for the editor and board; frequent offenders should not be asked to review again. Most of us, however, see the need as we like to contribute papers, to exchange ideas and contribute to our profession, and thus 'play the game' for our own benefit as much as the benefit of the journal. This is the incentive I would foster.
ReplyDeleteI am in full agreement with peer review and the stats of the independant report.
ReplyDeleteDear Reviewers
ReplyDeleteIn short, in my opinion, the peer review system principle is as sound as democracy - flawed, but fundamentally about as good as it gets. In practice, the outcome is somewhat variable, as with most theory.
I can commend the remark by another reviewer under this topic on the proliferation of papers published with microscopic increments in novelty. On this point I believe the peer review system is failing the scientific community (which includes the Engineering community), and by implication, the world.
It is a matter of concern to what extent, and at what cost to the significant advancement of science, the academic community is chasing academic credits in terms of the low hanging fruit called publication count. Here the peer review system should pull up its socks and apply a deft hand at rejecting insignificant manuscripts, regardless of the author list, affiliations and loyalties.
One should publish only if one has something significant to say. The peer review system should be the gatekeeper. All things are corruptible, but this one is of cardinal importance and fails us at the world's detriment.
On others points, I support peer review as a free service to the engineering and broader scientific community.
As for delaying everyone by being late - set your calender and install a big hammer to hit you over the fingers, if that helps.
Single review is not such a good idea, just as voting with a single voter is not too bright either. At least two reviewers should be assigned to a paper submission.
The Elsevier web system works reasonably well - once my occasional login issues have been sorted out.
I believe the system as presently set up works very well. Perhaps some encouragement could be made to hasten the review process - normally it only takes a couple of hours for a primary read, a think about and then provide, where required, detailed feedback to the author(s) to suggest improvements and perhaps where the work could be expanded in future related activity. If speed is really considered a major point the review form might be modified to indicate a review in a week, two weeks or at worst say a month, depending on the available time of the reviewer.
ReplyDeleteResearch - and science - by nature contains an element of uncertainty and the work of any researcher would be credible only if it undergoes the peer-review process.
ReplyDeleteHi Barry
ReplyDeleteAs an author and reviewer I still have faith in the peer review system as the best option available. I agree that the turnaround time can be a concern and admit that I often have to be reminded. My biggest concern relates to the time and effort reviewers put into the process. I have on several occasions reviewed papers where there have been serious flaws in experimental design and particularly data interpretation, leading to the suggestion of major review or rejection, only to find the second reviewer has accepted as is with only a line or two of comment. Completing a detailed review is a time consuming process, but I still think it is the best way of preventing poor science from being published.
I think peer review is absolutely necessary.
ReplyDeleteWhat I would like to suggest to MEI is to hide the author’s identity like the reviewers’ identities. This may eliminate a possible biasedness that may occur during a review.
My view is every reviewer should be committed to deliver the reviews quickly as possible if the process needs to get any quicker, given that every reviewer has a busy time schedule.
There is no alternative. Without peer review, scientific publications will degenerate into rapid-fire blogs with endless counterarguments and corrections. Peer review is essential to maintain quality of what appears in the literature.
ReplyDeletePeer review is a frustrating exercise for the author, reviewer and editor, but there is simply no alternative. How effective the peer review process is depends largely on the judgement and diligence of the editor. In my experience “Minerals Engineering” is managed better than most journals from an author’s and reviewer’s perspective.
ReplyDeleteAs a regular author in high impact and pure science journals, I find that a more difficult hurdle than satisfying the reviewers has become writing a sharp abstract to fit within the “fashion” (more narrow than the scope) of the journal. In practice the assistant to the editor will glance through the paper and if it does not have the right key words or looks too specialised, it does not go to review. This is a frustration in emerging areas like synchrotron analysis of geopolymers where there is only a small community publishing in the top end journals. Our group will shortly publish a rebuttal supported by further data to correct critical statements by another group on our work in a chemistry journal. That group had a fundamental flaw in their understanding, which also happens with reviewers, as we are all human. Therefore, a manuscript should not be rejected on the basis of one rejection only. My experience is that once a manuscript has been reviewed, most editors will consider a logical and robust rebuttal from the authors, and equally strong views from the reviewers. This is a healthy process, despite the frustration to all parties involved, especially for the editor, as it obliges the editor to read the paper, reviews and rebuttals (which could be as long as the paper) in sufficient detail to form a judgement. The practice in “Minerals Engineering” to use a third or fourth “trusted” reviewer as an adjudicator is exemplary.
I have been a reviewer for 70 journals and receive on average five review invitations every week. Therefore, I can accept only selected invitations but still feel that it is a substantial time commitment. What I find frustrating is the increasing number of marginal manuscripts which claim to have leached this particular ore, or have made that geopolymer using a new starting material. There is often little conceptualisation or novelty, but it cannot be evident to the editor without a proper review; what a waste of resources nevertheless! As a reviewer I often feel like writing just one word “garbage!” but in terms of self-respect and in an attempt to educate the rapidly growing community of authors one feels compelled to produce a thorough review. Many of us reject invitations to join the editorial boards of a growing number of free on-line rapid publication “journals.” There is little quality control in these “journals” despite their claim of peer review. Without the frustrating process of peer review there is no way of controlling quality or maintaining the reputation of a top journal. The editor is the ultimate gatekeeper of quality and must be able to distinguish between quality and controversy.
As an associate editor of a journal I am frustrated by the difficulty of finding suitable reviewers, especially in a more specialised area. The acceptance of my review invitations is less than 50%, even from “friends” (on-line we all proclaim to be these days), with the excuse being too busy. It used to be an honour to be invited to review a decent paper for a good journal. Today it has become a burden and a mere moral obligation to our colleagues. The number of submitted manuscripts to all journals has risen substantially, but the number of quality reviewers has grown marginally at best. An aggravating factor is that the English of some technically competent reviewers is no better than the broken English of the authors. There is no easy solution in sight, except for editors and reviewers getting tougher. Unfortunately, in getting tougher there will be top quality but controversial papers that get rejected. A persevering author will succeed though. I admire an editor who can do this job well over 23 years!
I agree with the process of peer review, otherwise, how else is a journal supposed to maintain its standard and ensure the publication of high quality, good research papers.
ReplyDeleteThe peer-review process is needed. It is a verification of sound science and/or engineering. It also helps to minimize duplication. Furthermore, researchers in countries like Australia get supplementary/merit pay based on it. It is time-consuming though and greatly dependent on the volunteer/service work of the reviewers/editors. Times have changed. Perhaps, it is not the peer-review process that should change but rather it be considered more like a paid job. At Montana Tech, and I know I am not alone, I teach 9 credits of courses per semester or 18 credits per year. I also do research as well as perform service in numerous capacities particularly for the professional societies, in my case SME and TMS. Working in volunteer time is no easy task but I have to pay attention to my salaried position first (and family as well). Research often pays extra comp. Perhaps the journals and societies need to consider the same?
ReplyDeleteCourtney Young, Montana Tech, USA
I strongly support the need for Peer review. The quality of papers received for review is falling and without the system the journal would be flooded with poor-quality papers. As an academic a journal needs to be of good standing before I would use it, however there are many authors just wanting large numbers of publications. In the simulation area authors are running a standard analysis poorly and trying to get it published, which would not be in the interest of genuine code users who want to know what best-practices are in a particular area.
ReplyDeleteWaiting some weeks for a review is reasonable, it is those who take many months that slow down the process unreasonably. However, this is something that the editor can manage.
Stay with the current system as to remove pear review would be to destroy the quality of the journal.
David F. Fletcher, University of Sydney, Australia
I read the text with interest but was hoping to see an alternative to the peer-reviewed process. Unless there is more exciting, fast responsive and quality enhancing alternative, the peer-reviewed process will remain the only choice for editors, but there is an option to significantly reduce the peer-reviewed process.
ReplyDeleteThe recommendation I have to Elsevier editors is to rethink the way in which an incentive will be generated for the reviewers to read and comment on the articles very quickly, within 3 days to 1 week at most after submission. This may be possible if the authors submitting articles for publication are required to pay a fee (lets say $300 per article submission) and this fee is used as an incentive to pay reviewers (lets say $100 per reviewer) for their fast response. I am aware that some journals are charging fees of over $1,000 per paper for publication, but paying fee on submission is really charging for the fast peer-reviewed process. Elsevier might want to think and create two options for submission of articles, one is fast responsive with fixed fee that guarantees review within 1 week, or slow and free which has 1 to 3 months review turnaround.
If I would have this alternative I wouldn't mind paying reasonable fee to have my most important papers reviewed quickly.
A/Prof Vladimir Strezov, Macquarie University, Australia
We will never get a perfect system but Peer review is a critical part of publishing and just as we expect people to take the time to evaluate our work , we do the same with others - and the work always improves - even when the reviewer gets the wrong end of the stick and misjudges the work - the author has to sharpen the argument... and its all worth it in the end!
ReplyDeleteDee Bradshaw, JKMRC, Australia
I agree with the opinions expressed by most of you. We seem all to accept the current peer-review system as necessary evil, but do not have any alternative.
ReplyDeleteJannie van Deventer pointed out that the problem to a large extent is the volume of review invitations, and to some extent the English of the contributions. For the latter I can tell you, it is the same if the contributions are written in German or Swedish. This makes me think that maybe it might be possible to implement a pre-review system, where proper use of language and grammar is used to pre-screen the contributions. I guess there are some semi-automatic language analysis tools that can be used. This would mean that the reviewer could concentrate on the actual message of the contribution, and wouldn't need to act as an English teacher.
Bertil Pålsson, Luleå University of Technology, Sweden
Bertil, regarding English. Elsevier do have a pre-screening system which weeds out the papers with atrocious English, but many sent to me are still well below standard in this respect. However, if the paper has potential I send it to referees, and their questionnaire asks them to comment on whether the English is up to standard. This is all I ask of the referees- you and others do not need to act as 'English teachers', just evaluate the technical merits of the paper. Once accepted as such, I, and an experienced English editor, look after the final language polishing.
ReplyDeleteThanks to all of you who have contributed so far (the posting is not a day old yet!). Rather than try to answer all your questions, I will leave it for a week or so, and then summarise in a new blog post.
A agree that peer review is a MUST and should be done in the quickest possible time. In many of the papers I have recently reviewed for your journal, the major problem was with the English language.. Such papers can be be rejected at the initial stages itself at the receiving office itself.
ReplyDeleteProf.K.A.Natarajan
Indian Institute of Science, India
Perhaps a moot point, but with no intent of mischief, is the list of authors that appears on a paper. While the choice rests with the authoring party, without an effective mechanism for independent verification, it has come to my attention that some departmental chairmen have their names included in author lists.
ReplyDeleteThere is reasonable doubt in my mind whether the chair of any scientific or engineering department has the time substantially to contribute to the academic content of any paper originating from his/her department.
In my view, signing checks for cost points do not qualify someone as co-author of a technical paper, even if being co-author raises the academic accreditation, which indirectly might count towards drawing increased research funds.
I agree with most of the comments and discussions above. I do think, however the double-blind approach is better then everything remains anonymous and there is less opportunity for additional bias. I believe there would be a tendency to review a submission from a well known scholar versus an unknown one.
ReplyDeleteUntil we have a better system, if one exists, the double blind is the best we have. I think all submitting authors understand that it takes time to review papers on the part of the volunteer reviewer and we try to be as patient as we can.
Dr Michael Hitch PhD P.Geo, P.Eng
University of British Columbia
Department of Mining Engineering
Thanks Michael. In an ideal world, the double-blind system would be preferred. But in reality most experienced specialist reviewers would be able to make a fairly astute guess as to the identity of an author, based on the nature of the work and the references.
ReplyDeleteI agree that the peer review process is absolutely necessary for quality science and scientific progress.
ReplyDeleteThe researchers interested in seeing rapid, non-reviewed publication of their work can do so via the plethora of international conferences which are seldom refereed beyond the abstract. There are a few journals available that would publish non-reviewed papers for general and topical interest. For Minerals Engineering in particular, I do not think any recourse to publishing non-reviewed papers would be sensible.
I seldom find the time to review papers. The reviewing process is particularly tedious when the English is sub-standard. I agree that some form of pre-filtering of the papers is required. What is worse is the number of papers that are published with the same authors that are essentially the same in content, using the same data presented earlier with marginal differences in presentation and discussion.
While R&D institutions and universities often have a publish-or-perish attitude, sometimes with a number of reward systems for publication (often government incentivised), there is no reward mechanism for reviewers. This noble calling requires either:
1) A sacrifice of family time
2) A sacrifice of personal time
3) A sacrifice of professional time
We can make money in many ways in life, but we cannot create more time – and doing thankless reviews takes time away from family / personal life / performing our primary duties at the workplace.
In the end, it is a question of economics:
The supply side is overly stimulated (incentive schemes, publish-or-perish, prestige), whereas the demand side (reviewers and even the readers of archival journals) cannot keep up. Unless something is done to moderate the submission of poor quality papers and incentivise the reviewing process, the free market principles will assert itself with fewer and fewer scientists making themselves available for review and more and more poor quality submissions swamping the publishing houses.
If authors are really convinced that their papers are truly novel and of high standard, they (or their institutions) would be willing to pay. Paying would not guarantee any publication, neither does it even guarantee a review (the editor may still reject it prior to review), but it will force authors to reconsider if they seriously want their work reviewed. Conversely, I do believe that reviewers should be rewarded.
Money seems to be the only (blunt) instrument to re-establish some form of equilibrium. Time is very precious, in particular for senior researchers, senior academics and professionals in industry. Quality and quality assurance comes at a cost. Free market economics would lead to either:
1) Pricing in the price of quality & quality control to achieve an improved system of sustainable quality.
2) Deterioration of quality, due the need for rapid publication of large number of papers of questionable quality (similar to most goods & services).
The monetary reward for reviews can be used to ensure timely feedback from reviewers. It therefore becomes a management tool to the Editor.
Let those who want rapid non-reviewed publication publish it on their organisation / institution web pages where it can be found using Google...
Jacques Eksteen
As an author and reviewer, I do agree with peer review system, as many people commented above. However, I think a double blind system is preferred to single blind to prevent or reduce bias by the reviewers. Also, it is very important for both authors and reviewers to educate themselves about the peer review process and the ethics involved. Particularly, reviewers should be aware that they are evaluating and commenting about the research not the researchers or authors. I have seen insulting, useless or careless comments from inexperienced reviewers who were not fully aware about their position as reviewers. I think there are some basic unwritten rules to peer review process that when necessary journal editors must somehow bring to the attention of novice reviewers, so that the authors can improve their manuscript during the process.
ReplyDeleteThe peer review process, while imperfect, is the best we have and should be vigorously defended. It is a method to insure the quality of publications is maintained. Yes, reviewing papers can be burdensome at times; one always seems to get a request to review a submittal at “just the wrong time”. Perhaps those of us who do a reasonable amount of peer-reviewing should look at this as an opportunity to get a preview of what may be an exciting or innovative piece of research. It may also be an opportunity to guide a young and promising researcher in the right direction or to suggest some future, highly promising research that may significantly benefit our industry.
ReplyDeleteEnglish language issues often leave me frustrated with a paper that may otherwise have merit. On several occasions I’ve suggested that large research universities, where English is seldom used, would benefit by employing technically proficient individuals with strong English skills whose primary job would be to improve the language quality of papers submitted for publication.
Corale L. Brierley, Ph.D.
I have no problem with the current system.
ReplyDeleteBrian Loveday, Emeritus Professor, University of Kwazulu-Natal, South Africa
Peer reviews are a way for everyone to learn even outside the academic environment. Current experiences would suggest that it's vital for ALL. No one is above feedback or learning from others even if more junior.
ReplyDeleteHi Barry
ReplyDeleteSorry to be late with these comments, I have been busy writng a lecture on crytals for lay persons who may or may not understand basic chemistry. Luckily it went well with good feedback from a member of the audience who had A-level Chemistry. No peer review however which is probably the case with most lectures and oral presentations.
Published technical papers on the other hand definitely reqire PEER REVIEW. It is essential and I agree with most of the comments posted above and your own views. It can be frustrating especially for poorly written papers in English (or any other publishable langauge)and the only innovation I would likre to suggest is that editors should refer the paper back to a specialist bureau for redrafting papers in decent language, preferably before peer review. It might be hard to judge the scientific merit first to see if this process is justified by the content but it would help reviewers. I know such trancription organisations do exist in places like China but I am not sure if they exist elsewhere. If not, fluent colleagues should be asked to help as it adds to the burden of reviewing if a paper needs to be virtually rewritten to be comprhensible. As one commentator above says "Time is precious".
Another point that I have made before is that whilst I fully agree that reviewing should be voluntary and unpaid, it would help to provide limited expenses for reviewers like me who are retired and do not have access to organisational facilities like paper and printer ink. I know it could all be done on-line but I prefer to be able to scribble comments on the manuscript and refer back to them to write reports.
I am impressed by the quality of most of the journals for which I act as a reviewer particularly the Elsevier publications "Minerals Engineering" and Hydrometallurgy though in my field I sometimes find it hard to decide which of those is the most appropriate.
I also appreciate the problems of the editor having been one myself many years ago though I think the problems are rather different now from when I acted in the early days of "Hydrometallurgy"
Why am I an author?
ReplyDeleteIt is a common requirement in postgraduate studies that papers are published out of the thesis work. I think this may be where some of the practice of publishing starts in a career. Supervising professors (and external examiners) want to see that some of the candidate’s thesis work has been submitted to, and has passed, a peer review process and has been published in a reputable journal. This adds credibility to the thesis work, and to the postgraduate degree awarded. Thereafter, the professional develops a position on a niche topic whilst working in industry or at university (some choose academic careers), and wants to see if their latest addition to the knowledge and best practice of his/her topic stands the test of his/her peers.
So it seems that a peer review system is essential – as Dee Bradshaw has pointed out.
Why am I a reviewer?
Firstly, because you approached me and asked if I would be interested in doing so, and I agreed. That formed a commitment. Secondly, because we all want the journal to continue into the future as a solid, reliable archival reference base in minerals processing.
Some Comments on the Current Peer Review Process
Ideally, we would want a situation where:
1. The author(s) have really done their homework, and have put together a strong scientific case describing their piece of new science or engineering. They have furthermore had their manuscript checked by a colleague whose English is top-notch, and the flaws in the article have been sorted out either by performing more work, or by more advanced interpretation. This is typically the case for experienced, well-published authors.
2. The editor-in-chief looks at the abstract, sees the potential in the article, and easily finds two perfectly-qualified and experienced reviewers who have expert knowledge of the niche topic being written about. They furthermore have ample spare time in which to perform the review in a week or so.
3. Both reviewers spend between eight and ten hours reviewing the manuscript, find a few related publications that should be cited, but are quite comfortable with the scientific component, and submit their reviewers’ reports with a recommendation for “minor edits, then publication”.
Continued in next comment.......
.......continued
ReplyDeleteIn reality, we have a situation where:
1. The author(s) have only sometimes done their homework, and English is not their first language, or even if it is, their writing style is not practical, efficient and concise. Furthermore, this type of author often has not read widely and deeply enough from the published literature. This is more prevalent in the new authors; however the more experienced ones (who have developed good author reputations) more closely resemble the ideal description abovementioned.
In other cases, sometimes the work submitted by the author is a paradigm shift from the conventional, but is sound new work and deserves to be published. It does happen in this type of case that sometimes one of the reviewers has difficulty engaging the new thinking pattern of the author, and puts obstacles in the way of the publication. A long exchange between author(s) and reviewer(s) then develops. This can at times be unpleasant, but as Jannie van Deventer correctly points out, the authors should stick to their view and write rebuttals to explain to the reviewer why their interpretation is correct (this commonly happens when the author(s) have “imported” knowledge from another discipline, in which the reviewer may not necessarily have read).
2. The editor-in-chief is always looking for more reviewers, and occasionally has difficulty in matching the subject material of the submitted article with two ideally qualified, experienced and available reviewers.
3. The reviewers are usually very busy people, and commonly find that the more basic problems of article structure, good English, and especially the lack of a deep literature review, confound the assessment of the scientific work in the submitted manuscript. They thus spend more time revising the writing than assessing the technical work. This can be seen as an annoyance, but in the spirit that we take on the role of a reviewer, are we not also becoming advisers to the new authors? Should we not be assisting new promising authors who have the potential to grow and develop, rather than just constructively criticising the work?
So…. onward with a necessary but imperfect peer review system….problems on both sides (authors and reviewers). I am aware that you and Nag Nagaraj have at times presented short courses on the subject of how to prepare a good paper. Maybe we need to offer that training more frequently – but are the right people attending these courses? Maybe ahead of the bigger conferences, such as at the IMPC – as you did in Beijing?
Yours sincerely
Norman O. Lotter, Xstrata Process Support
Sudbury, Ontario
Hi Barry,
ReplyDeleteHow did the number of papers submitted for review(to Elsevier in general and Minerals Engineering in particular) increase in the past two decades and to what degree did the number of available reviewers increase over the same period (1991-2011)? I am concerned that, despite all our noble intentions and commitments we might have a diverging situation on our hands. This does not imply that we move away from Peer Review. Above are ample reasons why Peer Review is critical. However, Elsevier (and other publishers) should revisit how the process is managed to better align submissions requiring reviews and available reviewers.
I do think that your specialised conferences work well to attract potential reviewers and for reviewers to get a "feel" of the contents of the papers through the presentations.
Jacques Eksteen
Hi Jacques
ReplyDeleteSorry I don't have that information to hand, but there has been an enormous increase in paper flow over the past decade.
Don't be too concerned about the number of reviewer, however. To be honest, finding enthusiastic reviewers is not a great problem (nor should it be). My main concern in the posting was about the slowness of peer-review in this electronic world- but no one seems too worried about this.
Hi Barry
ReplyDeleteI concur with most of the comments that the current single-blind review
system for Minerals Engineering works well. I see no reason why the
system should be changed.
Best regards
Adrian Hinde, Mintek, South Africa
Peer review starts with the author.
ReplyDeleteIs the quality of papers being submitted following the same lazy trend as the lack of readability found in both print and online news articles nowadays? And is our instant communication habit influencing some authors to be less inclined to rethink and refine papers to a higher quality before submitting?
Mike Adams
Mutis Liber, Australia
I have small experience about this subject but have engineering and industrial inspection efforts several years. No scape from inspection and evaluation, therefore the review (or peer-review) on papers is a must. At least 2 reviwers as representative of 2 point of views also make enough confidence about the quality. However for expediting the procedure maybe we can use a statistical survay for rough analysis. I mean for well known authors, you assign at least two qualified reviewrs, but by receiving the evaluation from only one well known referee (reviewer), maybe is enough.
ReplyDeleteWith regards
MH Kouklan
Japan
This fascinating dialogue tells the tale of intense interest in the quality of our publications. Willingness hampered by time impost are common themes. I am actively training my mid-level staff to take over reviewing - by reviewing their reviews.
ReplyDeleteI absolutely agree with reviewing being a form of mentoring especially for younger isolated researchers. It also forces me to keep up to date with the literature.
I think we should be harsher on poor structure and weak content - sending it back for improvement without trawling through editing it ourselves - which takes hours.
Downgrading to a technical note is appropriate for simple extensions of earlier work or interesting test-work. We wish to feel that the papers coming out are worthwhile reading and that the authors deserve the kudos associated with publishing a world top journal.
Malcolm Powell
My comments as a reviewer are as follows;
ReplyDeleteI sincerely believe that all reviewers have the sound intention to complete a review in a timely manner. Unfortunately there are a number of issue around this
1. Reviewer's time commitments
One of the major drawbacks of the review system is that in most instances it is a 'part time' occupation for the reviewer. They have a full time job that in this day and age consumes a lot of their spare time as well. I have found over my career that unless one has at least an 80% total work time commitment to a project/task then deadlines will suffer. Unfortunately having dedicated reviewers does nor seem to be an option consequently late reviews will be an outcome. Paying reviewers may help but I doubt it will.
2. The quality of the manuscript.
Excellent papers do no take long to review. As the 'quality' diminishes so more time is required to complete the task. One can go down the path of rejecting more submissions however my view is that part of a reviewers function is to help mentor and encourage new researchers with their submission. Where else will they learn and improve?
Best Regards
Rob Dunne, Newmont Mining, Australia
The peer review process is good, but can present problems depending on the significance of the research.
ReplyDelete1. If an article is accepted for review then one can assume the editor felt the article was suitable for the journal. Yet sometimes the ruling on the article is that it is not suitable. So why take 3 months evaluating the paper in the first place.
2. People in the field are general the ones reviewing the paper. If your paper is successful in explain a problem they have been trying to solve under their research program then it is likely your paper would be rejected. How can you get an honest peer-review from a competitor who has a financial interest in saying you are wrong?
3. Example: My paper was published in a peer-reviewed journal and presented at a physics conference. The work was rejected by some journals who knew the work was correct. The work successfully showed the link between subatomic structures and the motion of binary stars. It is the biggest discovery in physics and those who are charged with bringing this to the public avoided the paper.
Conclusion. The peer-review process is good when the journals are in support of science. The peer-review process is not good when the journal is actually the voice the its advisory boards universities. String Theorist hated my research even through they knew it was correct and the journals they control reject the paper falsely. The paper still got published but has not received the press it deserves.
The peer review process is a total shambles.
ReplyDeleteIt is OK if you are using conventional approaches, but if your methods are new - particularly using advanced mathematics - it is almost impossible to get a fair review.
Further, the editors tend to directly send your papers to your competitors. When it is obvious that the reviewer is incompetent the Editor tends to do nothing. That is the process is more important than the outcome.
At this stage I generally don't bother sending papers to journals, as the 'leading academics' are so far intellectually behind it is impossible for them to understand simple concepts.
I can outline the 'simple' concepts reviewers have not been able to understand - but it is quite appalling.
Sorry Stephen but I have to disagree with you on this. The peer-review system certainly does have its faults but it is not a shambles.
ReplyDeleteYou have obviously had some bad experiences with journals, but I think many academics might find it a little insulting to be regarded intellectually inferior and incapable of reviewing your work. The same applies to your generalisations on Editors.
I understand that there are limitations in the peer process. Yet in the opening statement people were asked to give an opinion. I regret giving my opinion.
ReplyDeleteI do think the opinions I gave have been exagerated; yet that was my error for lacking sensitivity in my comments.
The inference is that my experiences and those of my colleagues are are not consistent with others, so if others find the review process fair and reasonable all the best...
It should also be stated that the opening comment posed by Mineral Engineering was general, and did not specifically refer to Minerals Engineering.
I am abit perplexed that the issue of peer review was opened up in 'general' terms, and then my comments were criticised because I wrote 'generally'.
I do agree with the comment that the Editors of specific journals are actually the main factor in determining whether the peer review process is functional (in the sense of being fair).
I'm not quite clear what you are saying here Stephen, but I considered your original comment worthy of further debate, so I opened it up early this morning, before your comment above. Maybe you would like to comment on the 21st May posting?
DeleteLast comment
ReplyDeleteClearly I have written negative comments. Yet I do think there is an alternative, and it is one that I implemented for many years (but not in mineral processing).
One pre-submits the paper to true 'peer reviewers' and then after responding to their comments submits to the journal with a list and signature of the reviewers who have already commented. The journal should then decide whether to accept the paper straight off or send for additional review. Such a process would significantly decrease the work-load of all those involed: reviewers, editors, and authors; and would lead to what jounrals should be: a method for exchange of original research methods and concepts.
I think your comments are very useful Stephen, and I appreciate them, as they have highlighted something which I was not particularly aware of. I hope that this does lead to some discussion, and I will bring it to the attention of the Minerals Engineers LinkedIn group. There I would invite you to air your suggestion on the preliminary peer-review process, which I think is interesting and worthy. I know that some highly reputable research institutes submit all their papers to an internal review procedure before sending to journals, and this is of great help to editors.
DeleteIn my opinion usually the mineral processing area is very conservative and it is difficult to introduce new ideas and unconventional methods. In this regard I have had bad experiences and reviews sometimes do not delve into the issues. It is easy to reject new ideas, especially when they are far from being applicable in plants.
ReplyDeleteSee Elsevier's response to the peer-review debate at
ReplyDeletehttp://www.min-eng.com/journal/65.html
This is also of interest: http://www.guardian.co.uk/science/2012/may/25/attacking-publishers-open-access-sustainable
DeleteI think that the present process of peer review is an out dated one. In the days of internet, Post Publication Open Review (PPOR) must be followed. Let an author publish/submit his article any where on the net (for example MEI Blog) and then submit it to a journal. The editorial board of the journal may take a look at the work and if found fit they may ask the reviewers to act upon. If selected the journal may publish in the printed version. The author himself may get his article on the net, reviewed. This is a quick process. Is MEI ready to test?
ReplyDelete[1] CURRENT SCIENCE, VOL. 98, NO.9, 10 MAY 2010
[2] Open Review of Science Publications, Accountability in Research, 17: 1-7, 2010
Dear DMR. As editor of Minerals Engineering, I have been involved with peer-review for 25 years. Despite its limitations I think it is a very effective system. The method that you suggest above is used by MEI to some extent. All MEI Conferences require draft papers, which are used as discussion documents at the meetings. After a conference, authors are invited to submit their final papers to Minerals Engineering for peer-review, and for possible publication in the many special issues that we have.
DeleteThe MEI blog is not an option as you suggest. Could I clarify that, although I am editor, Minerals Engineering is not an MEI publication? It is published by Elsevier, so any change in reviewing policy would be in their hands.
Thanks for your suggestion.
Dear Sir,
ReplyDeleteThanks for your response. True publishing conference papers after peer review is also Post Publication Open Review. As such I am not against peer review but I plead for transparency.
It is very difficult to change the mind sets of peers. One example is the rejection by peers, when I said (http://www.iisc.ernet.in/currsci/may102001/1113.pdf ) that Phosphate Rock in fine size along with Organic Manure (PROM) works as efficiently as Di Ammonium Phosphate even in alkaline soils. Now (2012) PROM is an approved (http://www.promsociety.net/ ) fertilizer in India.
What is proved experimentally will stand if peers agree or not.
Thanks,
DMR Sekhar