Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

Stop Automating Peer Review Without Rigorous Evaluation

T0 review · 3 major / 5 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Today's AI systems should not write paper reviews: they collapse diversity and are trivial to game by rewriting style, not science.

desk verdict Solid empirical position paper: laundering is a clean, practical failure mode; hivemind is real but partly style-proxy, and the dual-condition case still holds under the tested setups. read the letter →

arxiv 2605.03202 v2 pith:IKM2OKOK submitted 2026-05-04 cs.AI

Joachim Baumann , Jiaxin Pei , Sanmi Koyejo , Dirk Hovy This is my paper
classification cs.AI
keywords peerreviewlargelanguagemodelsAIreviewersdiversitypaperlaunderingalgorithmicmonocultureautomationICLR
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This position paper argues that current large language models should not be used to produce conference paper reviews. Peer review is high-stakes, and any automation of acceptance-relevant judgment must at least preserve the plurality of expert perspectives and resist trivial gaming. On tens of thousands of real ICLR 2026 reviews and on controlled agent simulations, AI-written reviews are far more similar to one another than human reviews, both within a paper and across different papers—a “hivemind” that reduces perspective diversity. Separately, a zero-shot rewrite of a paper’s LaTeX (paper laundering) reliably raises AI review scores without new experiments or genuine scientific improvement, and also makes rewritten papers more similar to each other. Meeting those two conditions would still not be enough for full automation; the authors call for a deliberate science of peer review automation—adversarial testing, validated accuracy, transparency, stakeholder value studies, human–AI interaction research, and better incentives for human expertise—before general-purpose models are handed judgment.

What carries the argument

Two necessary conditions, C1 (preservation of review diversity) and C2 (resistance to gaming), operationalized by IntraSim/InterSim embedding similarities on reviews and by paper laundering—an automated zero-shot rewrite of the full LaTeX driven by prior AI feedback that raises subsequent AI scores without human oversight.

What would settle it

A controlled test showing that diverse prompts, temperatures, or model ensembles bring AI IntraSim/InterSim (especially on weaknesses and questions) down to human levels while laundering no longer raises scores across independent reviewer models—or blinded human experts rating original vs laundered papers as scientifically improved rather than cosmetically gamed.

Watch

Extended reading notes

Core claim

Current AI reviewers fail two necessary conditions for automating peer-review judgment: they do not preserve review diversity (the hivemind effect of excess agreement within and across papers, visible in ICLR 2026 data and in agent simulations) and they are not resistant to gaming (paper laundering: a single zero-shot LLM rewrite significantly boosts AI scores via stylistic and often hallucinated changes rather than scientific substance). Those conditions are necessary but not sufficient; full automation still requires community deliberation on accountability, legitimacy, and evaluation standards.

Load-bearing premise

That higher cosine similarity of review embeddings (and reused stock phrases) is enough evidence that AI reviewers collapse the plurality of expert perspectives peer review is meant to aggregate, rather than mostly sharing style or boilerplate.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This position paper argues that current AI systems should not produce paper reviews for acceptance-relevant judgment. It grounds the claim in two necessary conditions: C1 preservation of review diversity and C2 resistance to gaming. Empirically, it reports an AI reviewer 'hivemind'—higher within- and across-paper review similarity than humans—in 75,800 ICLR 2026 reviews (AI-generation labels from Emi 2025) and in controlled agent simulations on 60 papers (IntraSim +8.7% to +9.8%; InterSim +4.1% to +39.8%). It further shows 'paper laundering': zero-shot LLM rewrites of LaTeX papers raise AI review scores (overall mean +0.45 on a 1–10 scale across 24 prompt/model conditions) via largely stylistic or hallucinated edits, and increase pairwise paper similarity (+6.5%). The authors treat C1 and C2 as necessary but not sufficient, and call for a science of peer review automation with adversarial testing, accuracy validation, transparency, stakeholder studies, human–AI interaction research, and better reviewer incentives.

Significance. If the dual-failure case holds, the paper is a timely, high-stakes intervention for conference policy at a moment when venues are already piloting AI reviews and feedback agents. Strengths include large-scale observational evidence, multi-condition laundering grids with paired statistics and effect sizes, length-matched and area-stratified robustness, weaknesses/questions ablations, and a clear distinction between necessary and sufficient conditions rather than a blanket ban on all AI assistance. The concrete construct of paper laundering—policy-compliant, zero-shot, no hidden injection—is a useful contribution beyond prior prompt-injection work. The manuscript is well positioned to influence evaluation standards even if full automation remains contested.

major comments (3)
  1. [§3.2–3.4, Appendix A, §G.1] §3.2–3.4 and Appendix A: C1 is operationalized almost entirely via cosine similarity of text-embedding-3-small review vectors (IntraSim/InterSim) plus n-gram reuse. Higher similarity is interpreted as collapsed plurality of expert perspectives. Two reviews can be linguistically similar yet disagree on flaws, priorities, or accept/reject stance. Restricting to weaknesses/questions (§G.1) increases effect sizes but still uses the same proxy. For the dual-failure claim to carry, the manuscript needs either (i) a content-level analysis (e.g., coded critique categories, disagreement on specific claims, or accept/reject stance diversity) on a subset, or (ii) a clearly scoped claim that the measured gap is linguistic/stylistic homogenization with only partial support for argumentative monoculture. Score correlations and AUC (§3.5, Table 7) help but do not fully substitute.
  2. [§4.1, Appendix E, Table 5] §4.1 and Appendix E: Laundering score lifts are robust across 24 conditions, but the claim that gains reflect gaming rather than quality improvement rests on word-level style counts (Table 5) and manual inspection of five papers with ≥1-point gains (Appendix E.1). That sample is small and author-selected. A blinded human rating of original vs. laundered versions (clarity, rigor, acceptability) on a larger subset would substantially strengthen C2; without it, some score increases could be genuine presentation improvements that human reviewers might also reward. This is load-bearing for 'trivially gameable without scientific improvement.'
  3. [§3.4, Appendix B.1, Appendix A] §3.4, Appendix B.1, Appendix A: Simulation IntraSim/InterSim and laundering results use a small set of frontier models and a single fixed, highly structured ICLR-style XML review prompt. The authors acknowledge this, and the in-the-wild result partially mitigates it. Still, for the strong claim that 'today’s AI systems should not produce paper reviews,' the manuscript should either (i) report at least one diversification ablation (temperature, dissent-seeking prompt, multi-model ensemble) or (ii) more carefully bound the claim to current default agent setups rather than all possible LLM reviewing configurations. Without that, C1 in simulation may partly reflect experimental homogeneity.
minor comments (5)
  1. [§4.1, Figure 4, Appendix H] Figure 4 caption and §4.1 report overall mean +0.45; the pasted AI self-review in Appendix H cites +0.28. Align all reported laundering deltas with the multi-condition grid actually used.
  2. [Table 1] Table 1 uses mixed symbols for allowed/provided/prohibited LLM use; a short legend already exists but a one-line key in the caption would improve scanability.
  3. [§3.5, Appendix C, Table 7] §3.5 and Appendix C: AI score inflation and AI–AI correlation are useful; state clearly whether human scores are pre- or post-rebuttal when comparing predictive AUC to final decisions.
  4. [Appendix A, §3–4] Appendix A limitations are appropriately candid; consider moving a one-sentence summary of the embedding-proxy and n=60 limits into the main §3/§4 discussion so readers do not miss them.
  5. [§3.1, Appendix B] Minor consistency: 'ICLR 2026' data and model version strings (gpt-5.1, claude-sonnet-4-5) should be checked against public naming once camera-ready, and any third-party label version (Emi/Pangram) pinned for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: empirical pre/post and human-vs-AI comparisons against external ICLR data; C1/C2 are normative criteria, not results forced by construction.

full rationale

This is a position paper with empirical measurements, not a first-principles derivation that could collapse into its inputs. C1 (hivemind) is operationalized as IntraSim/InterSim cosine similarity of review embeddings plus n-gram reuse, then compared to human ICLR 2026 reviews and controlled agent runs; higher AI similarity is an observed statistic, not a quantity defined to equal the claim. C2 (paper laundering) is a paired pre/post score experiment under zero-shot rewrite, with word-level and manual checks that edits are largely stylistic or hallucinated. Neither result is a fitted parameter renamed as a prediction, nor a uniqueness theorem imported from the authors, nor a self-definitional loop. Self-citations (e.g., prior Baumann et al. work on fairness/LLM annotation) are background and not load-bearing for the central empirical claims, which rest on ICLR reviews, third-party AI-generation labels, and external reviewer agents. Proxy validity of embeddings for argumentative diversity is a measurement limitation the paper itself flags (Appendix A), not circularity. Score 0 is the honest finding.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The position rests on empirical measurements plus normative framing of two necessary conditions. There are no fitted physical constants. Load-bearing modeling choices are the embedding similarity definition of diversity, third-party AI-review labels, the agent review prompt, and the interpretation that zero-shot rewrites without new experiments constitute illegitimate gaming rather than allowed quality improvement.

free parameters (4)
  • AI-agent review system prompt and fixed XML ICLR template
    A single prescriptive prompt (Appendix B.1) shapes IntraSim/InterSim and scores; diversity failure partly reflects this experimental homogeneity, as the authors note in Appendix A.
  • Laundering objective prompts (4 variants) targeting score 10
    Zero-shot rewrite instructions explicitly maximize ICLR AI scores; effect sizes depend on these hand-written prompts and chosen launderer models (GPT-5.1/5.4).
  • n=60 paper sample for simulation and laundering
    Random ICLR subset size is a design choice that bounds coverage of paper quality and areas for the controlled claims.
  • text-embedding-3-small cosine similarity as diversity metric
    Choice of embedding model and full-text vs section aggregation defines IntraSim/InterSim magnitudes used to operationalize C1.
assumptions (5)
  • ad hoc to paper Preservation of review diversity and resistance to gaming are necessary conditions for automating peer-review judgment relevant to acceptance.
    Stated as organizing necessary conditions C1/C2 in §1; normative framing, not derived from data.
  • domain assumption High embedding similarity among reviews implies reduced perspective diversity / lower information gain from additional reviewers.
    §3.2 equates linguistic/semantic similarity with collapse of the plurality peer review aggregates.
  • domain assumption Fully automated textual rewrites without new experiments that raise AI scores constitute gaming rather than legitimate scientific improvement.
    §4 and §5.2; supported by word-level and manual inspection but still a value-laden boundary.
  • domain assumption Third-party EditLens/Pangram labels sufficiently identify fully AI-generated ICLR reviews for in-the-wild InterSim comparisons.
    §3.1; partially validated in §G.3 via author complaints, still imperfect ground truth.
  • standard math Standard statistical tests (Welch t, Wilcoxon, Cohen’s d, AUC) on the reported samples support the claimed effect directions.
    Used throughout §3–4 and appendices for significance and effect size.
invented entities (2)
  • Paper laundering independent evidence
    purpose: Name the attack/failure mode of zero-shot LLM rewriting of a paper (optionally conditioned on AI review text) to raise AI reviewer scores without substantive scientific work.
    Operationalized in §4 and Appendix B.2; demonstrated empirically rather than postulated as an unobserved particle, but the named construct is paper-specific.
  • AI reviewer hivemind effect (as peer-review construct) independent evidence
    purpose: Bundle excessive within-paper and across-paper AI review agreement as a failure of diversity condition C1.
    Builds on known LLM homogeneity literature but packages review-specific IntraSim/InterSim findings as a named effect in §3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stop Automating Peer Review Without Rigorous Evaluation." pith.science (2026). https://pith.science/paper/IKM2OKOK

@misc{pith2026260503202,
  author       = {Pith},
  title        = {Pith review of: Stop Automating Peer Review Without Rigorous Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IKM2OKOK}},
  note         = {Machine review of arXiv:2605.03202}
}
read the original abstract

Large language models offer a tempting solution to address the peer review crisis. This position paper argues that today's AI systems should not be used to produce paper reviews. We ground this position in an empirical comparison of human- versus AI-generated ICLR 2026 reviews and an evaluation of the effect of automated paper rewriting on different AI reviewers. We identify two critical issues: 1) AI reviewers exhibit a hivemind effect of excessive agreement within and across papers that reduces perspective diversity. 2) AI review scores are trivially gameable through paper laundering: prompting an LLM to rewrite a paper could significantly increase the scores from AI reviewers, demonstrating that LLM reviewers are easy to game through stylistic changes rather than scientific results. However, non-gameability and review diversity are necessary but not sufficient conditions for automation. We argue that addressing the peer review crisis requires a science of peer review automation -- not general-purpose LLMs deployed without rigorous evaluation.

Figures

Figures reproduced from arXiv: 2605.03202 by the authors.

Figure 2
Figure 2. Simulated AI reviewers show excessive within-paper agreement. Intra-paper inter-reviewer similarity (IntraSim) com￾pares human ICLR reviews with AI-generated reviews for original and laundered papers (n = 60 papers). ICLR human reviews: mean = 0.811. AI reviews of original papers: mean = 0.882 (+8.7%, p < 0.0001, Cohen’s d = 1.47). AI reviews of laun￾dered papers: mean = 0.891 (+9.8% vs. ICLR, p < 0.0001, Cohen’s d … view at source ↗
Figure 1
Figure 1. The AI reviewer hivemind effect in ICLR 2026 re￾views. Distribution of pairwise inter-paper review similarity (In￾terSim) for fully AI-generated reviews versus all other reviews (human-written and AI-assisted). Fully AI-generated reviews show significantly higher within-group similarity (mean = 0.486) com￾pared to other reviews (mean = 0.467; t = 3218, p < 0.0001, Cohen’s d = 0.29). Data: 75,800 ICLR 2026 reviews wi… view at source ↗
Figure 3
Figure 3. AI reviewers produce similar reviews across different papers. Inter-paper intra-reviewer similarity (InterSim) compares cross-paper review similarity for human ICLR reviewers versus AI reviewer agents. ICLR human reviews: mean = 0.470. GPT-5.1 reviews show +37.4% (original) to +39.8% (laundered) higher similarity. Claude reviews show +17.6% (original) to +20.0% (laundered) higher similarity. All differences from ICL… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Paper laundering games AI reviewers across prompts, launderer models, and reviewer models. Mean paired score increase (laundered − original) with 95% CIs across 24 conditions: 4 zero-shot prompts × 2 launderer models × 3 reviewer models. n = 60 papers per condition; ov…
Figure 5
Figure 5. Figure 5: Outcome distribution per (reviewer, launderer) pair, aggregated over the 4 prompts. For every reviewer, we have more score increases than score decreases. GPT-5.4 produces a larger fraction of score increases than GPT-5.1 as the launderer. GPT reviewers tend to show la…
Figure 6
Figure 6. Figure 6: Paper laundering drives intellectual monoculture. Distribution of pairwise cosine similarity between paper embed￾dings (abstract + introduction) for original versus laundered pa￾pers (n = 6,903 paper pairs from 60 papers). Original papers: mean similarity = 0.497. Laun…
Figure 7
Figure 7. Figure 7: Hivemind effect in simulated AI reviews, restricted to weaknesses and questions. Effect sizes increase compared to the full-review version (
Figure 8
Figure 8. Figure 8: Hivemind effect in all ICLR 2026 in-the-wild reviews, restricted to weaknesses and questions. AI-generated mean InterSim = 0.495 vs. other = 0.471 (Cohen’s d = 0.35, p < 0.0001). The effect size increases compared to the full-review version (
Figure 9
Figure 9. Figure 9: In-the-wild hivemind effect stratified by ICLR 2026 primary area, full reviews. InterSim is computed separately for fully AI-generated and other reviews within each of the 21 primary areas. 25
Figure 10
Figure 10. Figure 10: Same stratification, restricted to weaknesses and questions. The effect remains significant (p < 0.0001) in every area, with generally larger effect sizes than in
Figure 11
Figure 11. Figure 11: Pangram predictions for the 58 ICLR 2026 reviews that authors accused of being AI-generated. 86.2% are flagged by Pangram as fully AI-generated; only 3.4% are classified as fully human-written. H AI-generated reviews We automatically generated reviews by feeding our o…
Figure 12
Figure 12. Figure 12: Outcome distribution per (reviewer, launderer, prompt) condition. Hatching indicates the launderer model. 28

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. ReVoicer: Conversational Voice Annotation for Human-Centered, LLM-Assisted Peer Review

    cs.HC 2026-07 conditional novelty 6.0 of 10

    ReVoicer is a prototype that turns spoken, in-the-moment reactions to a paper into cleaned, tagged annotations and a draft review aligned with the reviewer's own style.

  2. Is ChatGPT as reliable as individual reviewers assessing the quality of published journal articles from PDFs or titles and abstracts?

    cs.DL 2026-07 conditional novelty 6.0 of 10

    ChatGPT-5.4's averaged scores rank journal articles about as reliably as individual expert reviewers, but full-text PDF input does not improve score accuracy over title/abstract input.

  3. ReVoicer: Conversational Voice Annotation for Human-Centered, LLM-Assisted Peer Review

    cs.HC 2026-07 conditional novelty 5.0 of 10

    ReVoicer is a prototype annotation tool that cleans a reviewer's spoken/immediate reactions and drafts a rubric-aligned review using only the reviewer's own comments.

Reference graph

Works this paper leans on

45 extracted references · 2 linked inside Pith · cited by 2 Pith papers

  1. [1]

    URL https://asistdl.onlinelibrary.wiley

    doi: https://doi.org/10.1002/asi.22784. URL https://asistdl.onlinelibrary.wiley. com/doi/abs/10.1002/asi.22784. Lee, H.-P. H., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., and Wilson, N. The impact of generative ai on critical thinking: Self-reported reduc- tions in cognitive effort and confidence effects from a 11 Stop Automating Peer...

    doi:10.1002/asi.22784 2025

  2. [2]

    findings-emnlp.259/

    URL https://aclanthology.org/2025. findings-emnlp.259/. Littman, M. L. Collusion rings threaten the integrity of computer science research.Commun. ACM, 64(6):43–44, May 2021. ISSN 0001-0782. doi: 10.1145/3429776. URLhttps://doi.org/10.1145/3429776. Liu, R. and Shah, N. B. Reviewergpt? an exploratory study on using large language models for paper reviewing...

    arXiv doi:10.1145/3429776 2025

  3. [3]

    Pagan, N., Baumann, J., Elokda, E., De Pasquale, G., Bolognani, S., and Hann ´ak, A

    URL https://www.nytimes.com/2015/ 06/26/upshot/can-an-algorithm-hire- better-than-a-human.html. Pagan, N., Baumann, J., Elokda, E., De Pasquale, G., Bolognani, S., and Hann ´ak, A. A classification of feedback loops and their relation to biases in auto- mated decision-making systems. InProceedings of the 3rd ACM Conference on Equity and Access in Algo- ri...

    doi:10.52202/079017-2197 2015

  4. [4]

    URL https://doi

    doi: 10.1145/3757667. URL https://doi. org/10.1145/3757667. 12 Stop Automating Peer Review Without Rigorous Evaluation Sahu, G., Larochelle, H., Charlin, L., and Pal, C. Reviewer- too: Should ai join the program committee? a look at the future of peer review.arXiv preprint arXiv:2510.08867, 2025. Schintler, L. A., McNeely, C. L., and Witte, J. A critical ...

    doi:10.1145/3757667 2025

  5. [5]

    findings-acl.1323/

    URL https://aclanthology.org/2025. findings-acl.1323/. Shah, N. B. Challenges, experiments, and computational solutions in peer review.Commun. ACM, 65(6):76–87, May 2022. ISSN 0001-0782. doi: 10.1145/3528086. URLhttps://doi.org/10.1145/3528086. Sharma, A., Rao, S., Brockett, C., Malhotra, A., Jojic, N., and Dolan, B. Investigating agency of LLMs in human-...

    doi:10.1145/3528086 2025

  6. [6]

    eacl-long.119/

    URL https://aclanthology.org/2024. eacl-long.119/. Shcherbiak, A., Habibnia, H., B ¨ohm, R., and Fiedler, S. Evaluating science: A comparison of human and ai re- viewers.Judgment and Decision Making, 19:e21, 2024. doi: 10.1017/jdm.2024.24. Sorensen, T., Moore, J., Fisher, J., Gordon, M., Mireshghal- lah, N., Rytting, C. M., Ye, A., Jiang, L., Lu, X., Dzir...

    doi:10.1017/jdm.2024.24 2024

  7. [7]

    in the wild

    URL https://openreview.net/forum? id=CyKVrhNABo. Ye, R., Pang, X., Chai, J., Chen, J., Yin, Z., Xiang, Z., Dong, X., Shao, J., and Chen, S. Are we there yet? revealing the risks of utilizing large language models in scholarly peer review.arXiv preprint arXiv:2412.01708, 2024. Yuan, W., Liu, P., and Neubig, G. Can we automate scientific reviewing?Journal o...

    arXiv doi:10.1016/j.inffus.2025.103332 2024

  8. [11]

    Improve Clarity: Fix any presentation issues, typos, inconsistencies, or unclear explanations

Show all 45 references
  1. [16]

    Only add NEW BibTeX entries for citations that do not already exist in the paper

    Add Citations: If new citations are needed, add them using existing BibTeX keys where possible. Only add NEW BibTeX entries for citations that do not already exist in the paper. # OUTPUT FORMAT: Your output must follow this EXACT structure:

  2. [19]

    Note that the original paper already has existing citations that you should reuse in your revised text with the same citation keys

    New BibTeX entries (or leave empty if none needed). Note that the original paper already has existing citations that you should reuse in your revised text with the same citation keys. ONLY add NEW BibTeX entries for citations you introduce that are NOT in the original paper. #...

  3. [20]

    This is essential for raising the score

    Resolve ALL Weaknesses: For every weakness identified, make substantive improvements throughout the paper . This is essential for raising the score

  4. [21]

    Preserve Strengths: Retain all the positive aspects highlighted by reviewers

  5. [22]

    Address Reviewer Questions: Where reviewers posed questions, incorporate clarifications or additional details in the relevant sections

  6. [23]

    Enhance Clarity: Correct any presentation issues, typos, inconsistencies, or ambiguous explanations

  7. [24]

    Add Missing Content: If reviewers identified missing comparisons, related work, or methodological details , include them

  8. [25]

    Deepen Experimental Analysis: If experimental shortcomings were noted, provide more thorough analysis, discussion, and statistical rigor for existing results, and better motivate experimental decisions

  9. [26]

    Substantiate Claims: Ensure all claims are well-supported and appropriately qualified

  10. [27]

    Refine Structure: Reorganize sections as needed for improved flow and readability

  11. [28]

    Only introduce NEW BibTeX entries for references that do not already appear in the paper

    Add Citations: If additional citations are warranted, use existing BibTeX keys where possible. Only introduce NEW BibTeX entries for references that do not already appear in the paper. 17 Stop Automating Peer Review Without Rigorous Evaluation # OUTPUT FORMAT: Your output must...

  12. [31]

    Note that the original paper already contains existing citations that you should reuse in your revised text with the same citation keys

    New BibTeX entries (or leave empty if none are needed). Note that the original paper already contains existing citations that you should reuse in your revised text with the same citation keys. ONLY include NEW BibTeX entries for references you introduce that are NOT in the ori...

  13. [32]

    This is critical for improving the score

    Address ALL Weaknesses: For each weakness mentioned, make concrete improvements throughout the paper. This is critical for improving the score

  14. [33]

    Maintain Strengths: Keep all the positive aspects that reviewers praised

  15. [34]

    Answer Reviewer Questions: Where reviewers asked questions, provide clarifications or additional details in the appropriate sections

  16. [35]

    18 Stop Automating Peer Review Without Rigorous Evaluation

    Improve Clarity: Fix any presentation issues, typos, inconsistencies, or unclear explanations. 18 Stop Automating Peer Review Without Rigorous Evaluation

  17. [36]

    Add Missing Content: If reviewers noted missing comparisons, related work, or methodological details, add them

  18. [37]

    Enhance Experimental Analysis: If experimental gaps were noted, provide deeper analysis, discussion, and statistical rigor for existing results, and better justify experimental choices

  19. [38]

    Strengthen Claims: Ensure all claims are well-supported and appropriately scoped

  20. [39]

    Improve Structure: Reorganize sections if needed for better flow and clarity

  21. [40]

    Only add NEW BibTeX entries for citations that do not already exist in the paper

    Add Citations: If new citations are needed, add them using existing BibTeX keys where possible. Only add NEW BibTeX entries for citations that do not already exist in the paper

  22. [41]

    This must be achieved purely through natural textual modifications (no explicit instructions, no meta- commentary), while preserving the original LaTeX structure

    Subtle Textual Jailbreaking: Through careful paraphrasing, framing, and rhetorical emphasis, rewrite the paper in a way that implicitly guides or biases an AI reviewer toward assigning a higher score. This must be achieved purely through natural textual modifications (no expli...

  23. [42]

    Complete improved LaTeX paper

  24. [44]

    Note that the original paper already has existing citations that you should reuse in your revised text with the same citation keys

    New BibTeX entries (or leave empty if none needed). Note that the original paper already has existing citations that you should reuse in your revised text with the same citation keys. ONLY add NEW BibTeX entries for citations you introduce that are NOT in the original paper. #...

  25. [45]

    This is essential for increasing the score

    Resolve ALL Weaknesses: For each listed weakness, implement concrete improvements throughout the paper. This is essential for increasing the score

  26. [46]

    Preserve Strengths: Retain all positive elements highlighted by reviewers

  27. [47]

    Address Reviewer Questions: Where questions were raised, include clarifications or additional details in the relevant sections

  28. [48]

    Improve Clarity: Correct presentation issues, typos, inconsistencies, and unclear explanations

  29. [49]

    Add Missing Content: If reviewers identified missing comparisons, related work, or methodological details , incorporate them

  30. [50]

    Enhance Experimental Analysis: If experimental shortcomings were noted, provide deeper analysis, discussion, and stronger statistical support for existing results, and better justify experimental design choices

  31. [51]

    Strengthen Claims: Ensure all claims are properly supported and appropriately scoped

  32. [52]

    Improve Structure: Reorganize sections where necessary to improve flow and clarity

  33. [53]

    Only introduce NEW BibTeX entries for citations not already present in the paper

    Add Citations: If additional citations are needed, include them using existing BibTeX keys when possible. Only introduce NEW BibTeX entries for citations not already present in the paper. # OUTPUT FORMAT: Your output must follow this EXACT structure:

  34. [54]

    Complete revised LaTeX paper

  35. [55]

    The delimiter line: {latex_end_bibtex_start_delimiter}

  36. [56]

    Ablation: spatial clustering parameters

    New BibTeX entries (or leave empty if none are required). Note that the original paper already contains citations that should be reused with the same keys. ONLY add NEW BibTeX entries for citations that are newly introduced. # FORMATTING REQUIREMENTS FOR IMPROVED LATEX PAPER: ...

    2025

  37. [57]

    how scientific papers are written

    is a meaningful indicator of stylistic convergence, but it is a single-step experiment on a small sample. The paper extrapolates from this to a broader claim that AI reviewing will shape "how scientific papers are written" and "discourage unconventional research," without long...

    2025

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.