Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that AI involvement in peer review can be reliably detected by focusing on substantive content composition rather than stylistic cues, with a detector achieving 98.24% macro F1 on a new benchmark covering six human-AI colla

desk verdict CoCoNUTS is a genuinely useful benchmark, but the 98% F1 and the real-world trend analysis overstate what the paper actually shows. read the letter →

arxiv 2509.04460 v1 pith:7VU6V4E5 submitted 2025-08-28 cs.CL cs.AI

Yihan Chen , Jiawei Chen , Guozhao Mo , Xuanang Chen , Ben He , Xianpei Han , Le Sun This is my paper · ORCID
classification cs.CLcs.AI
keywords AI-generatedtextdetectionpeerreviewcontentcompositionstylerobustnessmulti-tasklearninghuman-AIcollaborationcost-sensitivemarginlossbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to show that AI involvement in peer reviews is best detected by assessing the substantive content of the review—who actually produced the ideas and judgments—rather than by looking for telltale stylistic fingerprints of machine writing. To test this, the authors built CoCoNUTS, a benchmark of 315,535 review instances spanning six realistic modes of human-AI collaboration, and trained CoCoDet, a detector that learns to separate content from style. CoCoDet reaches a macro F1-score of 98.24% on the three-way Human/Mix/AI classification, roughly 40 points above the best few-shot LLM and far above general detectors. Applied to real conference reviews after ChatGPT's release, it reports a year-over-year increase in AI involvement, including a growing share of fully machine-generated reviews. If the claim holds, reviewers and program chairs could distinguish permissible language polishing from prohibited substantive generation with much greater fairness and reliability.

What carries the argument

CoCoNUTS is the benchmark: 315,535 peer reviews collected from major AI conference venues, transformed into six collaboration modes (human-written, machine-translated, machine-polished, human-machine mixed, machine-generated, machine-generated-then-paraphrased), then collapsed into three content-based labels (Human, Mix, AI). CoCoDet is a ModernBERT encoder trained end-to-end with four objectives: the main content-composition identification with a cost-sensitive margin loss that pushes apart the Human and AI classes, plus auxiliary content-source, textual-style, and collaboration-mode attributions that force the model to disentangle content from stylistic presentation.

What would settle it

Take a held-out set of real human-written reviews, have the same reviewers confirm that an LLM polish changed only wording, and run CoCoDet on the polished versions; if the false-positive rate on these content-preserved-but-polished reviews exceeds the ~1.3% reported in the paper, the content-invariance premise fails. A complementary check: apply CoCoDet to reviews from venues or years outside its training distribution and compare its accuracy against human-verified provenance labels.

Watch

Extended reading notes

Core claim

The central claim is that review text can be typed by content composition: reviews whose substantive content is purely human remain Human even after translation or polishing, reviews with purely machine-generated substance are AI even after paraphrasing to sound human, and reviews that combine substantive contributions from both are Mix. CoCoDet implements this by fine-tuning ModernBERT on a primary content-composition task and three auxiliary attribution tasks, so the model learns 'what is said' separately from 'how it is said.' On the CoCoNUTS test set it classifies the three classes with a macro F1 of 98.24%, and it shows near-zero confusion between the high-cost Human and AI classes. The

Load-bearing premise

The load-bearing premise is that the six construction operations—back-translation, polishing, expansion, and paraphrasing—change textual style without changing substantive content, so a review's content-composition label is determined by who wrote the ideas, not by how they are expressed.

Editorial extensions

If this is right

  • Conference policies that forbid AI-generated substance but allow AI polish become enforceable: a review flagged as Human can clear a reviewer who used a language tool, while an AI or Mix flag signals substantive machine involvement.
  • General-purpose detectors that rely on perplexity or n-gram fingerprints would be replaced or supplemented by content-composition classifiers in domains where substantive claims matter.
  • The real-world trend analysis implies that fully machine-written reviews are no longer hypothetical; they have been appearing in top venues with increasing frequency since 2023.
  • The style-robustness criterion (predicted AI rate rising from Human to Mix to AI subsets) gives evaluators a simple test for whether a detector is actually content-based rather than style-based.
  • Because the auxiliary tasks force the model to identify content sources and collaboration modes, the same architecture could provide provenance details (e.g., which LLM family generated the content) alongside the ternary label.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 98.24% F1 is measured against labels created by the paper's construction pipeline, so the number should not be read as an estimate of accuracy on real-world reviews whose provenance is unknown; a field test with verified ground truth would be needed.
  • The content-invariance premise could be stress-tested by applying the detector to reviews that were paraphrased in ways that inadvertently alter details—such as a summary that drops a nuance or a question that changes emphasis; if those edits flip the label, the content/style boundary is not as clean as assumed.
  • The same multi-task recipe—content-source plus style-source attribution—might transfer to other high-stakes domains such as medical notes or legal memos, where distinguishing substantive authorship from formatting is operationally important.
  • A natural next experiment is adversarial: take MGMP-style humanized reviews and run them through stronger paraphrasing or back-translation loops; the paper's stated robustness would be tested against increasingly aggressive evasion.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. CoCoNUTS is a 315k-instance benchmark of peer reviews from OpenReview, covering six human-AI collaboration modes mapped to ternary labels (Human/Mix/AI). CoCoDet is a ModernBERT-based multi-task detector with a cost-sensitive margin loss; it reports 98.24% macro F1 on the held-out test set, far above LLM and general detectors. Applied to post-2023 conference reviews, the model shows increasing AI involvement and a growing share of fully machine-generated reviews. The paper frames these results as evidence that content composition, not style, is detectable.

Significance. If the central claim holds, CoCoNUTS would be a valuable, large-scale resource for peer-review AI detection, and CoCoDet's results are strong. The paper is transparent about construction prompts, splits, baselines, and hyperparameters, and the six-mode coverage is a genuine contribution. However, the ground-truth labels are provenance labels produced by the same pipeline used for training and testing, and the only content-preservation support is qualitative. The reported F1 may therefore reflect surface artifacts. The real-world trend is model output, not independently validated, and inherits this risk.

major comments (4)
  1. [Task Definition and Dataset Construction] The central construct 'content composition' is operationalized by the construction pipeline (HW/HWMT/HWMP→Human; HWMG→Mix; MG/MGMP→AI). The only support for this operationalization is manual inspection of 100 instances and three side-by-side examples in Appendix D.1. There is no quantitative semantic-equivalence metric, no independent annotation of content origin, and the test split (Appendix B.2) is stratified by source LLM, not by unseen transformation mode. The 98.24% F1 (Table 3) could therefore measure how well CoCoDet reproduces pipeline provenance labels. Add independent annotation, semantic-similarity checks, and evaluation on held-out transformation modes/generators.
  2. [Figures 4-5 and Dataset Construction] The dataset is truncated to the 5th-95th percentile HW length, yet classes still differ in mean length: 431/506/494 words for Human/Mix/AI (Figures 4-5). A model can separate classes by length or related format cues. The style-robustness criterion in Table 4 (monotone predicted AI rate) does not exclude this, because length also increases monotonically. Report length-matched evaluations or a length/format-only baseline, and show that CoCoDet's decisions are invariant to such surface features.
  3. [AI Usage Trends (Figure 3)] Figure 3 is not independently validated: it is CoCoDet's output on unlabeled reviews. The 'pre-ChatGPT' label for ICLR 2023 is questionable because ICLR 2023 reviews were written around the public release of ChatGPT (Nov 2022). Year-over-year changes could reflect shifts in review length, formatting, or instructions. Validate on a manually annotated sample of real reviews and provide confidence intervals.
  4. [Ablation Study (Table 5)] Table 5's ablations show each auxiliary loss helps in-domain F1, but all auxiliary labels derive from the same pipeline, so the gains may reflect better prediction of mode/style rather than content/style disentanglement. The claim that auxiliary tasks 'separate content features from stylistic ones' (CoCoDet Detector) is not demonstrated. Test-time style perturbations (paraphrasing with preserved content) and analysis of prediction stability would directly support the disentanglement claim.
minor comments (4)
  1. [Introduction] Typo: 'Researches indicates' should be 'Research indicates'.
  2. [Tables 3 and 5] Irregular spacing in Table 3 entries like '3 .29'; full model average is 98.24 in Table 3 but 98.23 in Table 5. Please harmonize.
  3. [Appendix B.2 / Table 7] The heading 'corrected validation set' is unexplained; describe any correction made to the validation split, or rename the table.
  4. [Experimental Setup] All experiments use a single seed (42) with hyperparameters tuned on the validation set. Report multiple seeds and standard deviations, especially for the 98% macro-F1 claim.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; benchmark labels are pipeline-defined by design, but the training/evaluation chain is not circular.

full rationale

The paper constructs CoCoNUTS by defining six human-AI collaboration modes and then maps them to three content-composition classes (Human, Mix, AI) under an explicitly stated semantic-invariance premise. This is a dataset-design decision, not a circular derivation: the labels are not defined in terms of the detector's outputs, and the detector's 98.24% macro F1 is an empirical result on a held-out test set, not a quantity that is forced by construction. The auxiliary tasks (content source, style, collaboration mode) use construction metadata as supervision, which is a legitimate training signal rather than a disguised form of the target. The real-world trend analysis applies the trained detector to unlabeled conference reviews; the resulting year-over-year trend is an extrapolation from the model's predictions, not an input to the training procedure. The only self-citation we identified (Zhou, He, and Sun 2024) supports a background claim about paraphrasing attacks and is not load-bearing for the paper's central contribution. The paper's manual inspection of 100 instances in Appendix D.1 is a validation limitation but not circularity. Overall, the derivation chain is self-contained and transparent; the main risk is external validity (whether pipeline-defined labels reflect true content composition), which is a correctness/robustness concern rather than circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

These are the assumptions and tuned settings that the central evaluation depends on. The most consequential is the pipeline-defined label: the benchmark's ground truth is generated by the authors' own prompts, and CoCoDet is trained and tested on that same ground truth. This does not invalidate the benchmark as a tool, but it limits what the 98% F1 and the real-world trend analysis can mean.

free parameters (6)
  • base margin mbase = 0.25
    Hyperparameter in CSM-Loss that enforces a large decision boundary between classes; tuned on the validation set.
  • cost margin mcost = 0.25
    Hyperparameter that adds an extra penalty for confusing Human and AI; tuned on the validation set and central to the main loss.
  • scaling factor s = 30
    Scaling parameter in CSM-Loss; tuned on the validation set.
  • auxiliary task weights (alpha, beta, gamma) = 0.4, 0.4, 0.2
    Weights in the composite loss controlling the three auxiliary tasks; tuned by sequential grid search on validation.
  • learning rate = 2e-5
    Selected by grid search over {1e-5, 2e-5, ..., 5e-5} on validation.
  • token-length percentile cutoff = [5, 95]
    Samples outside the 5th-95th percentile of HW token length were removed from all categories; this ad hoc truncation shapes the benchmark distribution.
assumptions (5)
  • domain assumption Semantic-invariant operations (machine translation, polishing, paraphrasing) alter only style, not substantive content; hence HWMT/HWMP retain human content and MGMP retains AI content.
    Invoked in Task Definition and Appendix D.1; supported only by three hand-picked case studies, not by semantic measurement or independent annotation.
  • domain assumption Reviews written in or before 2022 are purely human-written, before ChatGPT, and serve as the Human ground truth.
    Used in HW construction; ignores the possibility of other AI tools or pre-2022 automated assistance.
  • domain assumption The generating or editing model named in the pipeline is the true content source; HWMG content indeed contains both human and AI substantive contributions.
    Labels are assigned by pipeline rather than by independent human annotation of content provenance; used throughout dataset construction.
  • domain assumption Stratified split on generating LLM yields an i.i.d. test set, and benchmark performance transfers to real-world reviews.
    Used for evaluation and for the Figure 3 trend analysis; no out-of-domain validation set is provided.
  • domain assumption ModernBERT representations and the multi-task losses achieve content/style disentanglement as claimed.
    Training setup; the disentanglement claim is inferred from ablations rather than directly verified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection." pith.science (2026). https://pith.science/paper/7VU6V4E5

@misc{pith2026250904460,
  author       = {Pith},
  title        = {Pith review of: CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7VU6V4E5}},
  note         = {Machine review of arXiv:2509.04460}
}
read the original abstract

The growing integration of large language models (LLMs) into the peer review process presents potential risks to the fairness and reliability of scholarly evaluation. While LLMs offer valuable assistance for reviewers with language refinement, there is growing concern over their use to generate substantive review content. Existing general AI-generated text detectors are vulnerable to paraphrasing attacks and struggle to distinguish between surface language refinement and substantial content generation, suggesting that they primarily rely on stylistic cues. When applied to peer review, this limitation can result in unfairly suspecting reviews with permissible AI-assisted language enhancement, while failing to catch deceptively humanized AI-generated reviews. To address this, we propose a paradigm shift from style-based to content-based detection. Specifically, we introduce CoCoNUTS, a content-oriented benchmark built upon a fine-grained dataset of AI-generated peer reviews, covering six distinct modes of human-AI collaboration. Furthermore, we develop CoCoDet, an AI review detector via a multi-task learning framework, designed to achieve more accurate and robust detection of AI involvement in review content. Our work offers a practical foundation for evaluating the use of LLMs in peer review, and contributes to the development of more precise, equitable, and reliable detection methods for real-world scholarly applications. Our code and data will be publicly available at https://github.com/Y1hanChen/COCONUTS.

Figures

Figures reproduced from arXiv: 2509.04460 by the authors.

Figure 1
Figure 1. Overview of our CoCoNUTS benchmark. The left side illustrates data acquisition and preprocessing, while the right [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Confusion matrix of CoCoDet on the ternary clas [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The predicted AI involvement of recent conference [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Average text length (in words) for each of the [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 4
Figure 4. Figure 4: Aggregated average length (in words) across [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 6
Figure 6. Figure 6: Word count distributions for each of the six fine-grained collaboration modes in the CoCoNUTS dataset. This detailed [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Team DACTYL at PAN 2026: Bayesian Data Mixing and Empirical X-risk Minimization for AI-text Detection

    cs.CL 2026-07 conditional novelty 5.0 of 10

    A Bayesian-uncertainty text filter, partial-AUROC training, and MCGrad calibration produce the second-ranked AI-text detector (0.974 mean score) on the PAN 2026 leaderboard.

Reference graph

Works this paper leans on

35 extracted references · 31 canonical work pages · cited by 1 Pith paper

  1. [3]

    Keep the academic rigor while making it fluent in Chinese

    work page

  2. [4]

    Please translate the following Chinese peer review into English, paying attention to:

    Output only the translated text, without any other information Review: {en review} # CN2EN: You are a professional AI field translator. Please translate the following Chinese peer review into English, paying attention to:

    work page

  3. [5]

    arXiv:2412.13663

    Smarter, Better, Faster, Longer: A Modern Bidirec- tional Encoder for Fast, Memory Efficient, and Long Con- text Finetuning and Inference. arXiv:2412.13663. Wu, K.; Pang, L.; Shen, H.; Cheng, X.; and Chua, T.-S

    arXiv

  4. [7]

    What is said?

    Large Language Models Penetration in Scholarly Writing and Peer Review. arXiv:2502.11193. Zhou, R.; Chen, L.; and Yu, K. 2024. Is LLM a Reliable Reviewer? A Comprehensive Evaluation of LLM on Auto- matic Paper Reviewing Tasks. In Calzolari, N.; Kan, M.-Y .; Hoste, V .; Lenci, A.; Sakti, S.; and Xue, N., eds., Proceed- ings of the 2024 Joint International ...

    arXiv 2024

  5. [12]

    Technical terms should be accurately translated

    work page

  6. [13]

    Maintain the original review structure

    work page

  7. [14]

    Keep the academic rigor while making it fluent in English

    work page

  8. [15]

    Please polish the following peer review

    Output only the translated text, without any other information Review: {cn review} HW&MP You are a senior AI researcher and experienced reviewer for top-tier AI conferences. Please polish the following peer review. Please maintain the original technical content and core evaluation while improving sentence structure, terminology consistency, and readabilit...

    work page

Show all 35 references
  1. [16]

    Delete some redundant content make it more concise

  2. [17]

    Expanding its content based on the provided paper content

  3. [18]

    Improve the sentence structure, terminology consistency, and 10 readability

  4. [19]

    polished

    Output: Provide only the raw text of the elevated review , do not include any other details. Review: {review} Paper content : {paper content} MG You are a senior AI researcher and experienced reviewer for top-tier AI conferences. Please carefully read the example reviews and t...

  5. [22]

    This includes some sections written by a human and others generated by an AI

    **‘mix‘**: Classify as ‘mix‘ only if the text contains substantive content contributions from both human and AI. This includes some sections written by a human and others generated by an AI. Your response must be *only* one of these three words. Do not provide any explanations...

  6. [23]

    This includes texts that were later machine-translated or polished by language tools

    **‘human‘**: Classify as ‘human‘ if the core content was written by a human. This includes texts that were later machine-translated or polished by language tools

  7. [24]

    This includes texts that were later edited or ’humanized’ by a person to sound more natural

    **‘ai‘**: Classify as ‘ai‘ if the core content was generated by an AI. This includes texts that were later edited or ’humanized’ by a person to sound more natural

  8. [25]

    This includes some sections written by a human and others generated by an AI

    **‘mix‘**: Classify as ‘mix‘ only if the text contains substantive content contributions from both human and AI. This includes some sections written by a human and others generated by an AI. Your response must be *only* one of these three words. Do not provide any explanations...

  9. [26]

    The studied problem is interesting and important because a theoretical analysis on criteria of learning good hash codes remains largely unexplored

  10. [27]

    The proposed method seems to be reasonable and effec- tive

  11. [28]

    Weaknesses:

    Experiment seems to be extensive. Weaknesses:

  12. [29]

    There exist some typos and grammatical errors in the pa- per

  13. [30]

    The training sets on all datasets are relatively small. Is the performance improvement still significant when we increase the size of training set? The authors have adequately addressed the limitations and potential negative societal impact of their work. This paper first demo...

  14. [31]

    The research question is interesting and important, as the- oretical analysis of the criteria for learning good hash codes has not been sufficiently explored

  15. [32]

    The proposed method appears reasonable and effective

  16. [33]

    Weaknesses:

    The experiments seem quite extensive. Weaknesses:

  17. [34]

    There are some spelling and grammatical errors in the pa- per

  18. [35]

    The training sets for all datasets are relatively small. Does the performance improvement remain significant when the size of the training set is increased? The authors adequately address the limitations and potential negative societal impacts of their work. Table 10: Content ...

  19. [36]

    How are the hyperparameters β1 and β2 determined?

  20. [37]

    Is there any convergence analysis for the proposed attack method? How can we guarantee its convergence?

  21. [38]

    Is it possible to extend this approach to the L-infinity norm setting?

    The proposed method is presented as an L2-norm- minimizing attack. Is it possible to extend this approach to the L-infinity norm setting?

  22. [39]

    Strengths

    Have the authors evaluated the attack method against DenseNet as a victim model? Overall, the paper is well-written, clear, and provides excel- lent background and context. The core idea is original and compelling. The authors have conducted experiments that ef- fectively demo...

  23. [343]

    Artemova, E.; Lucas, J

    Miami, Florida, USA: Association for Computational Linguistics. Artemova, E.; Lucas, J. S.; Venkatraman, S.; Lee, J.; Tilga, S.; Uchendu, A.; and Mikhailov, V . 2025. Beemo: Bench- mark of Expert-edited Machine-generated Outputs. In Chiruzzo, L.; Ritter, A.; and Wang, L., eds....

    2025

  24. [2023]

    InThe 2023 Conference on Em- pirical Methods in Natural Language Processing

    LLMDet: A Third Party Large Language Models Gen- erated Text Detection Tool. InThe 2023 Conference on Em- pirical Methods in Natural Language Processing. Ye, R.; Pang, X.; Chai, J.; Chen, J.; Yin, Z.; Xiang, Z.; Dong, X.; Shao, J.; and Chen, S. 2024. Are We There Yet? Reveal- ...

    2023

  25. [2024]

    In Al-Onaizan, Y .; Bansal, M.; and Chen, Y .- N., eds., Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 5081–5099

    LLMs Assist NLP Researchers: Critique Paper (Meta- )Reviewing. In Al-Onaizan, Y .; Bansal, M.; and Chen, Y .- N., eds., Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 5081–5099. Mi- ami, Florida, USA: Association for Computational Lingu...

    2024 arXiv

  26. [2025]

    arXiv:2505.14271

    FAID: Fine-grained AI-generated Text Detection us- ing Multi-task Auxiliary and Multi-level Contrastive Learn- ing. arXiv:2505.14271. Tao, Z.; Chen, Y .; Xi, D.; Li, Z.; and Xu, W. 2024. Towards Reliable Detection of LLM-Generated Texts: A Comprehensive Evaluation Framework wi...

    2024

  27. [7018]

    ISBN 979-8-89176-189-6

    Albuquerque, New Mexico: Association for Compu- tational Linguistics. ISBN 979-8-89176-189-6. Bao, G.; Rong, L.; Zhao, Y .; Zhou, Q.; and Zhang, Y . 2025. Decoupling Content and Expression: Two-Dimensional De- tection of AI-Generated Text. arXiv:2503.00258. Bao, G.; Zhao, Y .;...

    2025 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.