How to Write Peer Reviews That Actually Improve Science

Most researchers learn to review by being reviewed. They internalize the form from the other side—what comments felt useful, what felt dismissive, what sent them back to the manuscript with a clear sense of what to do—and they reproduce some version of that when it's their turn. There is no formal curriculum for peer review in most PhD programs. There is no widely agreed standard. There is mostly imitation, and the quality of what gets imitated varies enormously.

The results are consistent with that origin. Research published in peer-reviewed journals has examined the quality of peer review directly, and the findings are not flattering: roughly one third of reviews are considered genuinely helpful, one third are of reasonable but limited use, and one third are either unhelpful or grossly at fault. Editors who depend on external reviewers for their decision-making are working with this distribution every time they send a manuscript out. The system functions despite its quality variance, but it doesn't have to.

What the Research Says About Who Produces Good Reviews

A study published in the Journal of General Internal Medicine examined the characteristics of peer reviewers who produce high-quality reviews and found that certain factors predicted quality with striking specificity. When a reviewer was under 40 years of age, affiliated with a top academic institution, and blinded to the author's identity, the probability of producing a good-quality review was 87 percent. For reviewers without these characteristics, the probability was 7 percent—a twelve-fold difference.

Additional factors that correlated with review quality included previous training in clinical research methods, additional postgraduate degrees, and the amount of time actually spent on the review. Experienced editors who know a reviewer's prior work also produce better calibrated reviews than those relying on reviewer databases alone.

What this tells us is that review quality is not random—it is systematically related to the reviewer's own engagement with the research process as a practitioner. The reviewers producing the best work are the ones who understand what it takes to generate the evidence they are evaluating. They know what methodological choices were available at each decision point. They can distinguish between a design limitation that is fundamental and one that is characteristic of the research context. That knowledge shapes both what they see and how they communicate it.

The Reviewer's Role—and Where Most Reviews Misunderstand It

A peer reviewer is not the editor. The decision about whether to publish belongs to the editor, informed by the reviewer's expert opinion. Most reviewers intellectually understand this distinction; many don't write as though they do.

The reviewer's job is to provide an expert opinion on three specific questions: Is the methodology sound enough to support the conclusions? Are the conclusions appropriate given the data? Does this work contribute something meaningful to the field? Those answers inform the editor's decision. They are not the decision.

When reviewers write as though they are making the accept/reject call—structuring their comments around rejection rationale rather than author guidance, declining to offer constructive paths forward on papers they think should be rejected—they are conflating their role with the editor's and producing reviews that are less useful to both the editor and the author.

This is not an abstract distinction. A review that says "this paper does not meet the standards for publication in this journal" without identifying which specific standards it falls short of, why, and what would be required to meet them provides the editor with a vote, not an analysis. It provides the author with nothing.

Structure of a Review That Improves the Work

The reviews that generate the most positive responses from authors and editors tend to follow a structure that is neither universal nor widely taught, but that most experienced reviewers arrive at through trial and error. The elements are not complicated.

Summary of the work

Begin with a brief, honest summary of what the paper is doing: the research question, the approach, and the main finding. This serves two purposes. First, it demonstrates to the author and editor that you read and understood the paper—not always guaranteed. Second, it surfaces any fundamental misreading early, before it infects the subsequent comments. If your summary is wrong, the author can correct it; if your summary is right, it establishes a shared frame for the critique that follows.

Major concerns

Major concerns are problems that would materially change the interpretation of the findings if not addressed. A confound that undermines the causal claim. A measurement approach that introduces systematic bias. A missing comparison condition that makes the effect size uninterpretable. These are the issues that the paper cannot be published without resolving. State them clearly, explain specifically why each is a problem, and where possible indicate what would address it. A major concern that only identifies the problem without indicating what a solution would look like is half a review.

Minor concerns

Minor concerns are issues of clarity, completeness, presentation, or framing that don't change the fundamental interpretation but that a reader would benefit from having addressed. The distinction between major and minor matters: authors need to understand which items are required and which are suggested. Reviews that list thirty items without indicating which are critical and which are optional create confusion, produce resentment, and result in revisions that address the wrong things first.

Specific strengths

Name what the paper does well, specifically. Not "this is an interesting paper" but "the use of a pre-registered analysis plan and the inclusion of the sensitivity analysis in Table S3 substantially strengthen the causal argument." Reviewers who identify genuine strengths demonstrate that their critique is calibrated rather than reflexively negative. It also gives authors a clear signal of what to protect in revision.

Five Failure Modes in Peer Review Writing

The dismissal without diagnosis. "This paper does not meet the standards of this journal" is a verdict, not a review. If the paper has fundamental problems, name them specifically. The author needs to know what failed and why. The editor needs to know whether the problems are correctable. Neither gets that from a verdict without evidence.

The methodological fixation. Some reviewers spend two pages on statistical modeling choices and one sentence on whether the interpretation is supported by the results. Methods matter, but a methodologically rigorous paper that draws unwarranted conclusions is not a contribution to the field. Balance the technical critique with engagement with the intellectual claims.

The bibliography audit. Flagging the literature you wish had been cited—particularly your own—as a condition of publication is a misuse of the reviewer role. Recommend additional citations when they are genuinely necessary for a complete argument, not when they're papers you prefer be acknowledged.

The scope expansion. "This paper would be much stronger if it also tested the intervention in a clinical population, collected 24-month follow-up data, and included a cost-effectiveness analysis" is not a review of the paper that was submitted. It is a description of a different paper. Review the study that exists; recommend future directions as suggestions, not requirements.

The contradictory revision request. Nothing produces more author frustration than a review that requires, in adjacent bullet points, changes that are in direct tension with each other. Before submitting a review, read through the complete list of requests and check for internal consistency. Authors must respond to the review as written; a review that contains contradictions forces them to choose which request to honor and which to explain away.

What Makes a Review Improve Science Rather Than Just Gate It

There is a meaningful distinction between peer review that functions as a filter—accepting or rejecting based on an overall judgment of quality—and peer review that functions as a development process, improving the work that passes through it. Both happen, but the second is considerably rarer and considerably more valuable.

The reviews that improve science tend to engage with the intellectual claims of the paper, not just the methodology. They ask whether the interpretation is the most parsimonious one available, whether alternative explanations have been considered and addressed, whether the theoretical contribution is made explicit, and whether the limitations acknowledged in the paper are the right ones—not just the obvious ones. This kind of engagement requires that the reviewer actually think about the paper's argument, not just audit its procedure.

Reviewers who produce this kind of review become known for it. Editors request them again. Authors cite the review process as having strengthened their work. The reputation that follows—within a field where most reviewing is anonymous—belongs to the reviewer's institutional affiliation and the quality signal that reputation carries. But it also compounds in the reviewer's own practice: thinking carefully about what makes other people's arguments work makes you better at constructing your own.

The editors who have written most clearly about what distinguishes useful reviews from unhelpful ones tend to emphasize the same principle: a review should leave the author better equipped to improve the manuscript or understand its limitations, regardless of the outcome. A rejection with a thorough, specific, constructive review serves science. A rejection with a vague dismissal does not.