Why Most Writers Get Feedback That Doesn't Improve Their Work

In 2006, Laleh Motlagh and colleagues published a study in Teaching in Higher Education examining the difference between peer feedback and expert feedback on academic writing. The finding was not that peer feedback was useless. It was more specific: peer feedback was highly effective at identifying surface errors and improving prose clarity, and largely ineffective at identifying structural problems, logic gaps, and failures of argument. Expert feedback showed the reverse — moderately effective at catching surface issues, highly effective at identifying structural and argumentative failures. The problem was that researchers who received only peer feedback did not know what they were missing, because the feedback they received felt thorough.

This distinction between impression-based and criteria-based feedback is the central diagnostic problem in professional writing development. Feedback from people who know you, respect your work, and share your frame of reference tends to answer the question "Is this good?" not the question "Will this score well against the criteria the evaluator applies?" These are different questions. They produce different feedback. And only one of them predicts evaluation outcomes.

Why Feedback from Colleagues Who Like You Differs from Evaluator Feedback

The politeness bias in peer feedback is well-documented in educational psychology research. A 2014 meta-analysis by Liu and Hansen examining peer review effectiveness across 54 studies found that feedback givers consistently rated documents higher when they had a positive relationship with the author, and that positive feedback was less likely to include specific revision suggestions than critical feedback. The mechanism is straightforward: people who care about their relationship with you moderate their criticism to protect the relationship. Evaluators who do not know you have no such motivation.

Beyond politeness, colleagues in your field share your assumptions about what is obvious and what requires explanation. When a colleague in your research group reads your Significance section and does not flag the missing premise connecting your gap statement to your proposed intervention — it is often because they know the field well enough to supply the connection themselves. They do not notice the gap because they do not experience the gap. An evaluator outside your specific subspecialty has no access to the assumed knowledge and reads the missing connection as a structural failure.

This is the curse of knowledge operating at the level of the feedback provider rather than the author. The better your colleague knows your field, the less able they are to read your document as a non-specialist evaluator would read it. Expert colleagues are, in this specific sense, often worse feedback providers than knowledgeable non-specialists — because non-specialists notice what the expert takes for granted.

What "This Is Great" Means to a Colleague vs. an Evaluator

When a colleague reads your grant proposal and says "This is really strong, I think you'll get funded" — they are expressing a belief about the proposal's scientific quality, not a prediction about its review score. The two are correlated, but they are not the same thing.

An NIH study section reviewer scoring the same application is asking a different question: "Does this application's Significance section give me language I can use to advocate for this proposal in study section discussion? Does the Innovation section specifically name what is novel, rather than asserting novelty? Does the Approach section address the specific methodological concerns I would raise about this design?" These are evaluation-specific questions that a colleague expressing scientific approval is not systematically applying.

The gap between "scientifically strong" and "well-evaluated by the specific criteria applied in this review context" is where most grant applications fail. It is also where most journal manuscripts receive major revision requests instead of acceptances. The paper is not wrong. The data are not fabricated. The conclusion is supported by the evidence. The manuscript does not structure the case in a way that makes the editorial decision easy. The editor has to work to extract the contribution from the narrative, and under the time constraints of managing a large submission volume, that extra work often converts to a revise-and-resubmit rather than an accept.

How Expert Knowledge Limits Feedback Quality in Your Own Field

Research on expertise and metacognition — particularly work by Kruger and Dunning on self-assessment accuracy and related research by Chi and colleagues on expert-novice differences in problem-solving — shows that domain expertise creates predictable blind spots in evaluation. Experts evaluate documents in their field by pattern-recognition: they quickly identify whether the claims are reasonable, whether the methods are appropriate, whether the conclusions follow from the data. What they do not evaluate systematically is whether the document makes these relationships explicit enough for an evaluator who lacks the same pattern-recognition infrastructure.

A cardiologist reviewing a manuscript on cardiac arrhythmia mechanisms will immediately assess whether the electrophysiology is sound. They will be far less likely to notice that the introduction's transition from background to research question is implicit — because they supply the connection automatically. A reviewer with expertise in pharmacology rather than electrophysiology will notice the missing transition immediately, because they are actually reading the connection rather than inferring it.

This is why the most useful pre-submission feedback for a highly specialized document often comes from someone who knows the document type and the evaluation context well, but not the specific subfield. They are reading with enough background to understand the document but without enough background to unconsciously fill in the gaps.

Feedback Patterns That Correlate with Improved Revision Outcomes

The research on feedback effectiveness — including a systematic review by Shute published in Review of Educational Research in 2008 — identifies consistent characteristics of feedback that produces effective revision:

• Specificity of location: Feedback that identifies the specific section and paragraph where a problem occurs produces more targeted revision than feedback that describes a general problem across the document. "The introduction is hard to follow" does not identify the paragraph where the logic gap occurs. "The transition between paragraphs three and four in the introduction assumes background the evaluator may not have" does.
• Criteria-anchored diagnosis: Feedback that connects a document problem to the evaluation criterion it affects — "this section does not give a reviewer a clear basis for scoring the Significance criterion" — produces more strategically appropriate revision than feedback that describes the problem without its evaluation consequence.
• Revision direction: Feedback that suggests the direction of revision — not the specific words, but whether the section needs expansion, reorganization, cutting, or addition of missing evidence — produces more effective revision than feedback that identifies a problem without indicating how to address it.
• Absence of global praise: Feedback that includes substantial positive framing ("overall this is very strong, just a few small things") produces less revision than feedback that focuses on problems. This is not because positivity is harmful — it is because positivity creates a cognitive frame in which identified problems feel minor even when they are significant.

How Rubric-Based Evaluation Differs from Impressionistic Reading

A rubric is an explicit description of the criteria by which a document will be evaluated and what constitutes performance at each level of each criterion. The NIH scoring criteria for grant applications are a rubric. The OECM review criteria for academic accreditation self-studies are a rubric. The FDA's guidance documents for IND applications describe a rubric for how those applications will be reviewed. These rubrics exist in the public domain, and they specify exactly what an evaluator is looking for.

Impressionistic feedback asks whether the document reads well as a whole. Rubric-based evaluation asks whether the document satisfies each specific criterion the evaluator applies. These are different questions that produce different information about where to focus revision effort.

The most direct test of whether your pre-submission feedback is criteria-based or impressionistic is whether the feedback maps onto the specific criteria that will be used to evaluate your document. If your grant application feedback does not address all five NIH scoring criteria explicitly — Significance, Investigator, Innovation, Approach, Environment — it is not giving you the information you need to maximize your application's score. If your journal manuscript feedback does not address whether the contribution claim is strong enough for the target journal's scope, whether the methods section would survive peer review from a methodologist, and whether the discussion overreaches the data, it is not giving you the information you need.