The peer review process helps ensure that published research meets generally accepted standards for rigor within its field. This is important because published research is part of a field's permanent record, and is used to inform future research. Here are five things you might not know about the peer review process.
1. Informal peer review improves research in progress
Your research should be under informal peer review well before your last experiments are conducted, and well before you start preparing a manuscript for publication. Scholars have long recognized the importance of discussing their work with knowledgeable peers, whose different experiences and perspectives can lead to new ideas, new research strategies, and new interpretations of the data. To promote these interactions, the British Royal Society was founded in 1660, and the American Academy of Arts and Science was founded in 1780.
For many researchers, their first experience with this sort of review occurred in the classroom. Terrified students presenting science fair projects need to explain the question they're trying to answer, why that question is important, and how they will answer that question. Eventually, they will present their results and their interpretation of the results. While the researchers become more knowledgeable and the research more complicated, the same basic process is repeated by college students, graduate students, post-docs, professors, scientists at pharmaceutical companies, and Nobel laureates. The presenters and the audience may range from students, to full-time scientists, to advisors, but they can all contribute by asking questions from different perspectives.
This process occurs in faculty offices, at lab meetings, at research conferences during poster sessions and formal talks, during thesis presentations, and at job talks. Less formally, the process occurs in the lab, in the hallway, and over coffee or beer. If things go well, your audience will be interested in and excited about your research, and will ask questions or offer suggestions that will improve your research. If things go poorly, people may question the purpose of your study, point out flaws in your research strategy, or disagree with your interpretation of the results. Whether or not they are correct, you will need to go back and address these concerns—either by modifying your research or modifying how you present your research.
2. Peer review is required for journal publication
Once you have gone through multiple iterations of informal peer review, and completed your research and analysis, you can polish your manuscript and submit it to an appropriate journal to be considered for publication. After you've selected a target journal and crafted a manuscript to fulfill their specific requirements (for length, formatting, etc), you will submit it to an appropriate editor. Many journals have an extensive list of editors, and you will need to pick one that is familiar with your field of study, and preferably enthusiastic about your research methods.
You will send that editor your finely polished manuscript, with all text, figures, tables, references, etc. You will also include a thoughtfully constructed cover letter that explains why your research paper is a good fit for the journal. You will typically mention related studies that were published in the journal, and briefly explain how your research expands upon that work. In the cover letter, you may also include a list of researchers who work within your field of study (aka peers) who would be qualified to review your work. Importantly, these potential peer reviewers cannot be in a position to benefit from the publication of your work. People with a conflict of interest include co-authors on the paper, your current or former advisors, and people who work at the same institution.
The editor then decides whether your paper should undergo the formal peer review process, or whether it should be rejected without further review. Papers might be rejected at this stage for poor quality research or writing, research that is insufficiently novel, research that is insufficiently comprehensive, or research that falls outside the topics covered by the journal. At that point, you would need to decide whether and how to address these criticisms before submitting to another journal.
3. Peer review involves surprisingly few people
If the editor decides that your manuscript merits further consideration, s/he will ask experts within your field to serve as peer reviewers. For most journals, the goal is to find three qualified reviewers for each manuscript, but they often move forward with just two. The reason for these small numbers is that qualified reviewers are busy people—they have research to conduct and oversee, students to teach and advise, conferences to attend and prepare for, and their own grants and papers to write. Furthermore, reviewers are not paid; they are asked to volunteer their time as a service to their scientific community. So it's not surprising that potential reviewers often decline to review articles. Others will accept the invitation, but may assign the task to a graduate student or postdoc so s/he becomes more familiar with the process of publication.
Reviewers have weeks or months to return their comments to the editor. Typically, they will comment on specific strengths and weaknesses in the manuscript, and may recommend additional work that should be included. Each reviewer may recommend that the paper be rejected, revised and resubmitted, or accepted. If the reviewers disagree, the editor will accept the opinion of the majority (two out of three). If there are only two reviewers—and they disagree—the editor may try to find a third reviewer, or may cast the tie-breaking vote.
If your paper is returned with "revise and resubmit", it has not been accepted. You will be provided with the reviewers' detailed comments, and you must respond to every negative comment. Some comments ask you to include information you can easily access. Other comments ask for additional experiments, which may or may not be possible with the resources available to you. It's often best to conduct the experiments you can, and explain why the other requests are unrealistic. Typically, journals give authors only a few months to resubmit a revised manuscript. Depending on the extent of the revisions, the editor may make the decision to accept or reject, or the manuscript may be sent back to the original reviewers, who vote to accept or reject.
4. Peer review is deeply flawed, but is the least worst option we have
The goal of peer review is to provide an objective, reliable, and consistent way to evaluate the quality of papers submitted for publication. However, given the small number of people involved in the process for each paper (3-4, including the editor), it often falls far short of this goal. Ironically, researchers have only recently started studying the process of peer review, and the results are troubling. In one study, the British Medical Journal inserted multiple major errors into manuscripts that were then submitted to multiple reviewers. On average, each reviewer found a quarter of the major errors; some found none. In the end, the votes to accept or reject the manuscript were only slightly better than what you'd expect from a coin toss.
There's also evidence that editors and reviewers are influenced by the author's gender and affiliation with a prestigious institution. More obviously, journals actively seek to publish papers with exciting results. Therefore, high-quality research that gives negative results (i.e. a particular drug is found to have no effect on a particular illness), may not even be submitted for publication. This biases future studies, which rely on the published literature to report results as accurately as possible. Some journals are now attempting to address these problems, for example by evaluating the protocols for drug studies in progress, before the final results are available. Of course, all humans carry their own biases. As a result, the peer-reviewed scientific literature is littered with old studies that are astonishingly sexist, racist, or embarrassing in other ways.
Despite all of these flaws, the process of peer review—or perhaps the threat of peer review—has contributed to the undeniable advancement of science in recent centuries, especially in recent decades. Peer review is deeply flawed, but is the least worst option we have.
5. You can influence your reviewers
With all of the inherent flaws in the peer review process, it might seem like researchers should just throw up their hands and hope for the best. But don't forget that what you write—before you present a single result—deeply influences your reviewers. By clearly explaining how your work builds upon a foundation of important previous work, you can help convince readers that your work is necessary. By elaborating on how your results pave the way for future research in your field and related fields, you can help convince readers that your work needs to be published now. Of course, you may end up with reviewers who find your research unconvincing. If nothing else, this provides evidence that you are indeed working to increase our collective understanding of the world. No other research is worthwhile.
