pith. sign in

arxiv: 2604.07964 · v1 · submitted 2026-04-09 · 💻 cs.AI · cs.LG

Are we still able to recognize pearls? Machine-driven peer review and the risk to creativity: An explainable RAG-XAI detection framework with markers extraction

Alin-Gabriel V\u{a}duva , Simona-Vasilica Oprea , Adela B\^ara This is my paper

Pith reviewed 2026-05-10 18:11 UTC · model grok-4.3

classification 💻 cs.AI cs.LG
keywords machine-generated text detectionpeer review automationexplainable AIRAG frameworkscientific creativityepistemic homogenizationLLM markersreview quality assessment
0
0 comments X

The pith

Machine-generated peer reviews risk automated editorial systems that favor standardized research and penalize unconventional ideas, but a RAG-XAI framework detects them at 99.61% accuracy.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that LLMs in peer review could lead to fully automated pipelines that systematically prefer pattern-conforming work while disadvantaging creative or paradigm-shifting contributions needing human context. It introduces an explainable RAG-XAI framework that extracts markers such as absence of personal signals and repetition patterns to identify machine-written reviews. Ensemble models within the framework reach 99.61% accuracy, AUC-ROC above 0.999, and F1-scores of 0.9925 with very low error rates, outperforming a logistic regression baseline. The RAG retrieval component adds 90.5% top-1 accuracy and supports interpretability via feature importance and SHAP analysis. A reader would care because unchecked automation could reshape what counts as valuable science by incentivizing safe, algorithm-friendly outputs over genuine discovery.

Core claim

Machine-driven assessment in peer review may produce epistemic homogenization by favoring standardized, pattern-conforming research and penalizing unconventional ideas that require contextual human judgment. The introduced RAG-XAI framework counters this by using an LLM markers extractor combined with retrieval-augmented generation and explainable AI to detect automated reviews, achieving 99.61% accuracy with XGBoost, Random Forest, and LightGBM on the test set along with AUC-ROC above 0.999, F1-scores of 0.9925, false positive rates below 0.23%, and false negative rates around 0.8%. Absence of personal signals and repetition patterns emerge as the dominant predictors, while the RAG module's

What carries the argument

The RAG-XAI framework, which combines retrieval-augmented generation for context retrieval with an LLM-based markers extractor and explainable AI analysis (feature importance and SHAP) to identify and interpret signals distinguishing machine-generated from human reviews.

If this is right

  • Editors could deploy the framework to flag machine-generated reviews before decisions, preserving space for human judgment on novel ideas.
  • Dominant markers identified through SHAP allow targeted improvements in review guidelines to reduce detectable automation artifacts.
  • High RAG retrieval accuracy enables consistent marker extraction even when review contexts vary within the same domain.
  • The performance gap versus logistic regression shows that ensemble methods are required for reliable low-error detection in this setting.
  • Low false positive and negative rates support practical use without flooding editors with erroneous alerts or missing many automated cases.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If detection becomes routine, researchers may begin optimizing manuscripts and review styles toward features that evade the markers, shifting incentives even before full automation.
  • The same marker-extraction approach could extend to spotting AI assistance in grant proposals or journal submissions, broadening protection for creative work.
  • As newer LLMs reduce repetition and add simulated personal signals, periodic retraining on updated marker sets would be needed to keep detection effective.
  • The framework highlights a larger tension between efficiency gains from automation and the preservation of epistemic diversity in science publishing.

Load-bearing premise

The extracted markers such as absence of personal signals and repetition patterns remain stable indicators that generalize beyond the specific training data, LLMs, and review domains used in the study.

What would settle it

Applying the trained framework to a fresh dataset of peer reviews generated by LLMs released after the training data or drawn from scientific fields absent from the original corpus, and observing accuracy fall below 95% or a sharp rise in false negatives, would show the markers do not generalize.

read the original abstract

The integration of large language models (LLMs) into peer review raises a concern beyond authorship and detection: the potential cascading automation of the entire editorial process. As reviews become partially or fully machine-generated, it becomes plausible that editorial decisions may also be delegated to algorithmic systems, leading to a fully automated evaluation pipeline. They risk reshaping the criteria by which scientific work is assessed. This paper argues that machine-driven assessment may systematically favor standardized, pattern-conforming research while penalizing unconventional and paradigm-shifting ideas that require contextual human judgment. We consider that this shift could lead to epistemic homogenization, where researchers are implicitly incentivized to optimize their work for algorithmic approval rather than genuine discovery. To address this risk, we introduce an explainable framework (RAG-XAI) for assessing review quality and detecting automated patterns using markers LLM extractor, aiming to preserve transparency, accountability and creativity in science. The proposed framework achieves near-perfect detection performance, with XGBoost, Random Forest and LightGBM reaching 99.61% accuracy, AUC-ROC above 0.999 and F1-scores of 0.9925 on the test set, while maintaining extremely low false positive rates (<0.23%) and false negative rates (~0.8%). In contrast, the logistic regression baseline performs substantially worse (89.97% accuracy, F1-score 0.8314). Feature importance and SHAP analyses identify absence of personal signals and repetition patterns as the dominant predictors. Additionally, the RAG component achieves 90.5% top-1 retrieval accuracy, with strong same-class clustering in the embedding space, further supporting the reliability of the framework's outputs.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The manuscript proposes a RAG-XAI framework for detecting machine-generated peer reviews. It extracts markers (e.g., absence of personal signals, repetition patterns) via an LLM-based extractor, retrieves contextual examples with RAG, and classifies reviews using ensemble models (XGBoost, Random Forest, LightGBM) that achieve 99.61% accuracy, AUC-ROC >0.999, and F1-score 0.9925 on a test set, with SHAP analysis highlighting the key features. The work argues that automated peer review risks epistemic homogenization and reduced creativity by favoring standardized outputs over unconventional ideas.

Significance. If the reported detection performance and marker stability hold under proper validation, the framework would represent a meaningful advance in tools for preserving human oversight in scientific publishing. The integration of RAG for grounding outputs and SHAP for interpretability is a positive design choice that goes beyond opaque classifiers, directly supporting the paper's claims about accountability and the identification of automation patterns.

major comments (3)
  1. [Abstract and Results] Abstract and Results sections: The manuscript reports 99.61% accuracy, AUC-ROC above 0.999, F1 0.9925, FPR <0.23%, and FNR ~0.8% for XGBoost/RF/LightGBM without any information on dataset size, construction method, provenance of human reviews, exact LLMs/prompts/temperature used to synthesize machine reviews, or whether the train/test split was leakage-free with respect to the RAG corpus. These omissions are load-bearing for the central claim that the extracted markers are stable and generalizable rather than synthetic artifacts.
  2. [Results] Feature importance and SHAP analysis (Results): The dominant predictors (absence of personal signals, repetition patterns) are identified post-hoc from differences in the training data. This creates a circularity risk where the model may simply rediscover generation-specific artifacts rather than intrinsic, stable signals that would appear in real-world LLM-assisted reviews; no independent validation or ablation on held-out generation protocols is described.
  3. [Methods/Results] RAG component (Methods/Results): The reported 90.5% top-1 retrieval accuracy and same-class clustering are presented without details on the retrieval corpus size, embedding model, or overlap with training/test data. This leaves open the possibility of data leakage that inflates both retrieval and downstream classification performance.
minor comments (2)
  1. [Abstract] The abstract refers to a 'markers LLM extractor' without defining the extraction process or the precise set of markers; this notation should be expanded for clarity.
  2. [Results] The logistic regression baseline (89.97% accuracy, F1 0.8314) is mentioned but no implementation details (features, regularization, or hyperparameter search) are given, making the performance gap harder to interpret.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. The comments correctly identify areas where additional methodological transparency is required to support claims of marker stability and generalizability. We address each point below and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [Abstract and Results] Abstract and Results sections: The manuscript reports 99.61% accuracy, AUC-ROC above 0.999, F1 0.9925, FPR <0.23%, and FNR ~0.8% for XGBoost/RF/LightGBM without any information on dataset size, construction method, provenance of human reviews, exact LLMs/prompts/temperature used to synthesize machine reviews, or whether the train/test split was leakage-free with respect to the RAG corpus. These omissions are load-bearing for the central claim that the extracted markers are stable and generalizable rather than synthetic artifacts.

    Authors: We agree these details are essential for assessing reproducibility and generalizability. In the revised manuscript we will add a dedicated subsection in Methods describing: total dataset size and class balance, sources and selection criteria for human reviews, the exact LLMs, prompts, and temperature settings used to generate the machine reviews, and the procedure ensuring the train/test split has no overlap with the RAG retrieval corpus. This will allow readers to evaluate whether the reported markers reflect stable signals or dataset-specific artifacts. revision: yes

  2. Referee: [Results] Feature importance and SHAP analysis (Results): The dominant predictors (absence of personal signals, repetition patterns) are identified post-hoc from differences in the training data. This creates a circularity risk where the model may simply rediscover generation-specific artifacts rather than intrinsic, stable signals that would appear in real-world LLM-assisted reviews; no independent validation or ablation on held-out generation protocols is described.

    Authors: The concern about post-hoc circularity is valid. While the markers were initially motivated by prior literature on LLM text characteristics, the current analysis does not include independent validation on held-out generation protocols. We will add an ablation study using reviews generated by LLMs and prompts excluded from the original training set, together with cross-protocol performance metrics, to test whether the identified features remain predictive outside the original synthesis conditions. revision: yes

  3. Referee: [Methods/Results] RAG component (Methods/Results): The reported 90.5% top-1 retrieval accuracy and same-class clustering are presented without details on the retrieval corpus size, embedding model, or overlap with training/test data. This leaves open the possibility of data leakage that inflates both retrieval and downstream classification performance.

    Authors: We acknowledge the absence of these implementation details. The revised Methods section will specify the retrieval corpus size, the embedding model employed, the exact construction of the corpus, and explicit confirmation that it was built independently with zero overlap to the classification train/test partitions. We will also report retrieval metrics stratified by class to further demonstrate absence of leakage. revision: yes

Circularity Check

0 steps flagged

No significant circularity in the empirical framework

full rationale

The paper presents an applied ML detection system (RAG-XAI with marker extraction fed to tree-based classifiers) and reports test-set metrics plus post-hoc SHAP analysis. No mathematical derivation, self-definitional loop, or load-bearing self-citation is present in the abstract or described structure. Performance claims rest on standard train/test evaluation rather than any quantity that is definitionally equivalent to the input features or data-generation process. The framework is therefore self-contained against external benchmarks with no reduction of its central results to its own inputs by construction.

Axiom & Free-Parameter Ledger

2 free parameters · 1 axioms · 0 invented entities

The central performance claim rests on the assumption that detectable stylistic markers reliably separate human and machine reviews in real editorial settings, plus standard ML assumptions about i.i.d. data and feature stability.

free parameters (2)
  • XGBoost / LightGBM hyperparameters
    Tree-based models require tuned hyperparameters (learning rate, depth, etc.) that are fitted to the training data.
  • Embedding model and retrieval parameters
    RAG component depends on embedding dimensionality and similarity thresholds chosen or tuned during development.
axioms (1)
  • domain assumption Machine-generated peer reviews exhibit consistent, detectable patterns (repetition, lack of personal signals) that do not appear in human reviews at comparable rates.
    Invoked to justify the marker-extraction approach and feature importance results.

pith-pipeline@v0.9.0 · 5634 in / 1493 out tokens · 78316 ms · 2026-05-10T18:11:15.393042+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

3 extracted references · 3 canonical work pages

  1. [1]

    I think” and “after reading this twice

    Estimate proportion of AI-generated text in peer-review corpora Maximum likelihood estimation; corpus-level statistical modeling 6.5–16.9% of review text AI-generated; detection works better at aggregate level, not individual [15] Measure temporal growth of AI-generated peer reviews Detection model trained on historical data Near-zero pre-2022→~20% (ICLR)...

    work page arXiv 2022

  2. [2]

    AI-assisted peer review,

    Results This section presents the experimental results of the proposed AI-generated peer review detection framework. We report the findings in five subsections: dataset overview, descriptive analysis of the marker distributions, classifier comparison on the full dataset, SHAP-based explainability analysis and RAG retrieval evaluation. 4.1. Dataset overvie...

    work page doi:10.1057/s41599-020-00703-8 2017

  3. [3]

    Using an Artificial intelligence chatbot to critically review the scientific literature on the use of Artificial intelligence in Environmental Impact Assessment,

    A. Bond, D. Cilliers, F. Retief, R. Alberts, C. Roos, and J. Moolman, “Using an Artificial intelligence chatbot to critically review the scientific literature on the use of Artificial intelligence in Environmental Impact Assessment,” Impact Assess. Proj. Apprais., 2024, doi: 10.1080/14615517.2024.2320591. [8] B. Kocak, M. R. Onur, S. H. Park, P. Baltzer, ...

    work page doi:10.1080/14615517.2024.2320591 2024