Pith. sign in

REVIEW 3 major objections 6 minor 24 references

Pushing AI-generated text outside a detector's training distribution defeats even adversarial fine-tuning, while surface mimicry fails entirely.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 04:45 UTC pith:5EZX2PGR

load-bearing objection A solid empirical attack paper with a genuinely new mechanism, but the headline asymmetry is demonstrated for one RoBERTa-large recipe, not for adversarial fine-tuning in general. the 3 major comments →

arxiv 2607.13565 v1 pith:5EZX2PGR submitted 2026-07-15 cs.CR cs.AIcs.CL

UTS at ELOQUENT 2026 Voight-Kampff: structural shifts in AI writing bypass state-of-the-art detectors

Dima Galat , Marian-Andrei Rizoiu This is my paper
classification cs.CR cs.AIcs.CL
keywords AI-generated text detectionadversarial fine-tuningout-of-distribution attacksregister shiftstream-of-consciousnessfool rateevasion attack
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that AI-text detectors, including those adversarially fine-tuned to catch known evasion tricks, remain vulnerable to text that shifts the structural register or narrative form out of the training distribution. The authors show that the previous year's winning attacks—back-translation and imperfection prompts—are closed by a retraining pass, dropping fool rates below 0.025. In contrast, prompting a generator to write in early-twentieth-century English register or in modernist stream-of-consciousness form yields fool rates up to 0.846 and 0.65 respectively against the same strong detectors, even after the defender adds historical prose to the training set. The paper argues that evasion is powered by moving the token distribution out-of-distribution rather than by mimicking human surface text, and reports that this mechanism produced the top five positions in a shared benchmark.

Core claim

The central discovery is an asymmetry: adversarial fine-tuning trivially closes evasion attacks that rely on grammar-level or surface perturbations, but leaves a persistent blind spot for structurally out-of-distribution text. The authors' strongest in-distribution classifier, tuned to separate vanilla generations from human text, inverts under a back-translation attack, returning near-certain human scores. A deployer-class retraining collapses that attack's fool rate to at most 0.025, but two new attack families—historical-register prose (including a two-stage style-template variant) and a forced stream-of-consciousness narrative form—survive, reaching fool rates of 0.846 and 0.65 against t

What carries the argument

The key mechanism is the structural out-of-distribution direction: generating text whose token-level distribution lies outside the region the detector calibrated during adversarial fine-tuning. The detector family is built by fine-tuning a transformer classifier on the authors' generations plus human text, using leave-one-bucket-out and leave-one-strategy-out cross-validation; some variants add historical prose to the human side. The attack families exploit the fact that pre-modern register and radical narrative form are absent from that training pool. The central comparison is the paired change in fool rate under each strategy, and the decisive control is the variant that adds historical pr

Load-bearing premise

The load-bearing premise is that the authors' adversarial retraining recipe (fine-tuning a medium-size transformer on their own generations plus human text) fairly represents how a real deployer would respond; if a deployer used a different architecture, a larger and more diverse human corpus, or an active-learning loop that feeds the attack's own outputs back into training, the reported asymmetry could shrink or disappear.

What would settle it

Retrain an adversarial detector with the same architecture but add both (a) a large corpus spanning historical, legal, and colloquial registers and (b) the attack outputs themselves as hard negatives during training. If the historical-register attack's fool rate drops below about 0.2 while ordinary machine text is still flagged, the OOD asymmetry is an artifact of the specific training recipe rather than a fundamental detector blind spot.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A defender who retrains on known attack outputs alone cannot assume closure; the training corpus must also cover or explicitly model historical registers and narrative forms.
  • Naively expanding the human-side corpus with period prose can widen the detector's 'human' region without adding discriminative signal, so data-patching is not a safe fix.
  • Because no single tested attack beats both neutral and adversarial detector classes, deployers may need to maintain committees of detectors with complementary training distributions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The OOD mechanism likely generalizes to other specialized, information-dense registers such as legal or medical prose, since the paper only tested historical and narrative axes; a natural extension is to benchmark those.
  • The observation that translations age faster than their sources suggests that translation-inherited term distributions could serve as a productive attack axis beyond the tested early-20th-century translation register, and as a way to audit a detector's training coverage.
  • An untested but plausible defense is active learning that adds the attack's own outputs to the training pool; the paper's data do not speak to whether such hard-negative mining closes the gap.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper investigates which generation-side attack strategies survive deployer-class adversarial fine-tuning for AI-text detection. The authors train three RoBERTa-large adversarial detectors (Macko-replica, Macko-LOSO, Macko-pp) on their own generations plus PAN'25 humans, and introduce two OOD attack families: cross-decade register attacks (style_pre1923, synth_anchor) and modernist stream-of-consciousness narrative (stream_narrative). They report that the 2025 winning PJS-style attacks collapse under adversarial fine-tuning (fool rates ≤0.025), while synth_anchor reaches a 0.798 Macko-LOSO fool rate, the pg19-augmented Macko-pp detector fails to close the attack (0.846), and stream_narrative escapes Macko-pp at 65%. Their submissions occupy the top five positions on the ELOQUENT 2026 Voight-Kampff leaderboard. The paper's central claim is that pushing generated text out of the detector's training distribution is a durable attack mechanism, whereas pulling it into the human distribution through surface mimicry fails.

Significance. If the central asymmetry holds beyond the specific detector architecture tested, the paper would provide a practically important demonstration that register-level and form-level OOD shifts are durable attack directions, and that simple data augmentation countermeasures are insufficient. The study is unusually rigorous for a shared-task working note: it ships attack prompts and detector checkpoints, includes falsifiable pre-registered predictions (P1.1, P11.1, P12.1, P13.1), and supplements the main results with controls (real_anchor, PAN-mimicry, rewrite_k5, caveman) and a neutral PAN'25-trained committee. The external leaderboard result is a genuine strength, showing that the strategies are effective in a live evaluation rather than only against the authors' own detectors. The main limitation is that the adversarial detector family is a single architecture, and the paper's own neutral-committee results show that the picture inverts across detector training regimes.

major comments (3)
  1. [§3.3, §4.3, Table 5] The central asymmetry is established only for one adversarial detector architecture. Macko-replica/LOSO/pp are three training-data variants of RoBERTa-large fine-tuned on the authors' own generations; the actual PAN'25 winner's recipe ([7]: Qwen3-14B with QLoRA and homoglyph augmentation) is not tested. The neutral PAN'25 committee in Table 5 (RoBERTa-large, RoBERTa+homo, DeBERTa-v3-large) catches synth_anchor at 0.18/0.11/0.28 and style_pre1923 at 0.03/0.04/0.08. The claim that 'structural OOD shifts defeat adversarial fine-tuning' is therefore, on the reported evidence, a property of one RoBERTa decision boundary. Please add at least one non-RoBERTa adversarial fine-tune with the same folds and training data, or restrict the title/abstract/findings to 'the RoBERTa-large adversarial family tested here.'
  2. [§4.4, Table 4] 'The obvious deployer counter-measure fails' is based on a single augmentation condition: ~1,000 pg19 passages mixed at ~40% pg19 / ~60% PAN'25. The falsification of P11.1 shows only that this one budget does not close the attack. A deployer could plausibly use a larger or more heavily weighted period-prose pool. Please add at least two additional augmentation conditions (e.g., 2k/4k passages and 50%/75% pg19 share) or explicitly limit the conclusion to 'at the tested augmentation budget.' Without this, finding 3 and the §6 conclusion overstate the evidence.
  3. [§4.5, Attack 13] The 'pull into the distribution fails' leg of the asymmetry is not supported by a demonstrated in-distribution manipulation. The PAN-mimicry prompt ('match their voice, lexical patterns, and natural imperfections') is a few-shot instruction; the paper reports fool rates 0.000/0.006 but does not verify that the resulting texts are actually close to the PAN'25 human pool in the detector's feature space. If the mimicry attempt is weak, the result is about weak mimicry, not about in-distribution pulls generally. Please add a distributional-distance check (e.g., perplexity-to-human-pool or embedding-space distance) on the PAN-mimicry generations, and/or strengthen the mimicry baseline until it lands inside the human decision region.
minor comments (6)
  1. [Tables 3 and 4] style_pre1923 M-LOSO is 0.640 in Table 3 and 0.657 in Table 4 on the same 20-topic Claude Opus pool; explain the run difference or use one number consistently.
  2. [§3.2 vs §6] §3.2 says 'the 13 attacks reported' while §6 says '12 novel 2026 attacks'; reconcile the counts.
  3. [Table 7] 'roundtrip_imper0' appears truncated; list full submission names and map them to the strategies described in the paper.
  4. [§4.7, Table 6] 'while keeping all three supervised fool rates ≥0.97' — the table also shows zero-shot LogPerp/LogRank at ≥0.97; clarify which cells are supervised vs zero-shot in the prose.
  5. [§4.1, Underlying Mechanism] The dichotomy n-gram-surface vs contextual-distributional is inferred from only four detectors and no feature-level analysis; temper this claim or add an ablation.
  6. [§4.2] Define 'fool rate' and C@1 at first use; C@1 is currently introduced without a definition.

Circularity Check

0 steps flagged

No significant circularity: the OOD attack results are measurements against the authors' own detectors, not derivations from the attacks' definitions.

full rationale

The paper contains no equations and no fitted parameter is renamed as a prediction. The central claim—that cross-decade register and stream-of-consciousness attacks survive the Macko-replica/Macko-LOSO/Macko-pp adversarial fine-tunes—is an empirical evaluation against detectors trained on 2025 replication strategies and PAN'25 humans, with the novel attacks held out. This is a standard OOD evaluation protocol, not a circular derivation: the reported fool rates (0.798, 0.846, 0.65) are not inputs to the detector training, and the paper reports falsified predictions (P11.1 predicted ≤0.20, observed 0.846) plus a neutral PAN'25 committee (Table 5) whose ordering inverts Macko-pp. If the result were forced by construction, such inversion and falsification would be impossible. The self-citations to [1] and [9] are shared-task overviews and are not load-bearing; the load-bearing citations ([5], [6], [7]) are external 2025 PAN/ELOQUENT works. The skeptical concern that the conclusion overgeneralizes from a single RoBERTa-large family is a validity/generalization limitation, not circularity; the paper itself acknowledges limitations (Section 5) and includes the neutral committee's Pareto frontier. No specific reduction of a prediction to an input is exhibited, so no circular step is identified.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 0 invented entities

No new physical or mathematical entities are introduced. The free parameters are pragmatic choices in the attack/defense recipes (pg19 mix ratio, LOSO fallback fold, stream constraints). The central claim depends on the domain assumption that their adversarial fine-tuning recipe is representative of real deployers.

free parameters (3)
  • Macko-pp human-side mix ratio ~40% pg19 / ~60% PAN'25 = ~40% / 60%
    Chosen by hand for the deployer countermeasure; the failure of this exact mix to close the attack is central to the P11.1 falsification (Section 4.4). A different mixing ratio could in principle close the gap.
  • Macko-LOSO fallback fold for novel 2026 strategies (roundtrip_imperf fold) = roundtrip_imperf fold
    An ad hoc assignment rule for strategies not present during training; the deployer-pessimistic label is a modeling choice that affects the reported fool rates (Section 3.3).
  • stream_narrative structural constraints (≤3 full stops per 500 words; concrete particulars once per ~40 words; named wit = ≤3/500, ~1/40
    Hand-chosen constraints that define the attack; the 65% escape rate depends on these particular thresholds (Section 4.7).
axioms (3)
  • domain assumption The RoBERTa-large adversarial fine-tune on generations + PAN'25 humans is a faithful model of a 'deployer-class' detector.
    The entire adversarial-closure analysis uses this recipe (Section 3.3); the conclusion that the 2025 attacks are 'closable' generalizes only if the recipe is representative.
  • domain assumption Fool rate against the constructed detector panels is the right measure of evasion.
    The paper equates leaderboard performance with 'fool rate' and uses it as the central metric (Section 4), but a real deployment could use different thresholds or ensembles.
  • domain assumption The 'same topics' held-out VK-2026 20-topic split is a representative sample of the test distribution.
    The authors themselves state n=20 topics gives wide CIs (Limitations).

pith-pipeline@v1.3.0-alltime-deepseek · 11746 in / 7250 out tokens · 61915 ms · 2026-08-02T04:45:55.847873+00:00 · methodology

0 comments
read the original abstract

We investigate which language model evasion attacks survive state-of-the-art adversarial fine-tuning, developing strategies that sweep the top 5 positions on the ELOQUENT 2026 Voight-Kampff leaderboard. While adversarial fine-tuning trivially closes the 2025 winning evasion recipes, we uncover a fundamental asymmetry in detector vulnerability: pushing generated text out of the detector's training distribution reliably defeats adversarial detection, whereas pulling it into the distribution (e.g., mimicking human training data) fails completely. Exploiting this, we introduce two novel out-of-distribution attack families - cross-decade register attacks and modernist stream-of-consciousness form. Both strategies easily bypass adversarial closure, achieving up to approximately 50x higher fool rates than previous methods while preserving naturalness. Furthermore, experiments show that the obvious deployer countermeasure (augmenting training data with period prose) fails to close the vulnerability. Our findings show that the tested detector families, including adversarially fine-tuned ones, exhibit persistent vulnerabilities under structural out-of-distribution shifts, a mechanism that directly powers our leading competition performance.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 7 linked inside Pith

  1. [1]

    Gunti, B

    R. Gunti, B. Bayramoğlu, J. M. S. Dilinika, G. Devadasu, D. Galat, V. A. Narayana, R. Pakala, V. R. Reddyvari, M.-A. Rizoiu, J. M. Rodriguez, S. S. Sanagala, A. Tommasel, J. Bevendorff, J. Karlgren, Overview and Joint Report of the Voight-Kampff Task at the ELOQUENT 2026 lab for evaluating generative language model quality, in: E. S. Salido, A. Barrón-Ced...

    2026

  2. [2]

    Voight-Kampff

    J. Bevendorff, R. Gunti, J. Karlgren, M. Potthast, B. Stein, Overview of the third “Voight-Kampff” Generative AI / LLM Detection Task at PAN and ELOQUENT 2026, in: E. S. Salido, A. Barrón- Cedeño, A. G. S. de Herrera, S. MacAvaney, J. M. Struß (Eds.), Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2026), CEUR-WS, 2026

    2026

  3. [3]

    Voight-Kampff

    J. Bevendorff, M. Wiegmann, J. Karlgren, L. Dürlich, E. Gogoulou, A. Talman, E. Stamatatos, M. Potthast, B. Stein, Overview of the “Voight-Kampff” generative AI authorship verification task at PAN and ELOQUENT 2024, in: Working Notes of CLEF 2024 – Conference and Labs of the Evaluation Forum, volume 3740 ofCEUR Workshop Proceedings, CEUR-WS.org, 2024, pp....

    2024

  4. [4]

    Voight- Kampff

    J. Bevendorff, Y. Wang, J. Karlgren, M. Wiegmann, M. Fröbe, A. Tsivgun, J. Su, Z. Xie, M. Abassy, J. Mansurov, R. Xing, M. N. Ta, K. A. Elozeiri, T. Gu, R. V. Tomar, J. Geng, E. Artemova, A. Shelmanov, N. Habash, E. Stamatatos, I. Gurevych, P. Nakov, M. Potthast, B. Stein, Overview of the “Voight- Kampff” generative AI authorship verification task at PAN ...

    2025

  5. [5]

    P. Vachharajani, Literal re-translation as a method for AI text disguise and detection evasion, in: Working Notes of CLEF 2025 – Conference and Labs of the Evaluation Forum, volume 4038 ofCEUR Workshop Proceedings, CEUR-WS.org, 2025, pp. 1443–1448. URL: https://ceur-ws.org/ Vol-4038/paper_116.pdf. 2Adversarial checkpoints as in §3.3; the PAN’25-baseline d...

    2025

  6. [6]

    A. Creo, M. Hormazábal-Lagos, H. Cerezo-Costas, P. Alonso-Doval, Fake it ’til you make it human, in: Working Notes of CLEF 2025 – Conference and Labs of the Evaluation Forum, volume 4038 of CEUR Workshop Proceedings, CEUR-WS.org, 2025. URL: https://ceur-ws.org/Vol-4038/paper_109. pdf

    2025

  7. [7]

    Macko, R

    D. Macko, R. Moro, I. Srba, mdok of KInIT: Robustly fine-tuned LLM for binary and multiclass AI-generated text detection, in: Working Notes of CLEF 2025 – Conference and Labs of the Evaluation Forum, CEUR Workshop Proceedings, CEUR-WS.org, 2025. URL: https://ceur-ws.org/ Vol-4038/paper_307.pdf

    2025

  8. [8]

    Karlgren, E

    J. Karlgren, E. Artemova, O. Bojar, M. I. Engels, V. Mikhailov, P. Šindelář, E. Velldal, L. Øvrelid, Overview of ELOQUENT 2025: Shared tasks for evaluating generative language model quality, in: Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the 16th International Conference of the CLEF Association (CLEF 2025), Lectu...

    2025

  9. [9]

    Karlgren, M

    J. Karlgren, M. Barrett, O. Bojar, M. I. Engels, D. Fabre, S. Ettejjari, L. Goeuriot, R. R. Gunti, J. Mothe, P. Mulhem, M. Piacentini, L. F. V. Madriz, D. Schwab, P. Šindelář, G. Stampoulidis, K. Thomas, M. Vartampetian, Overview of ELOQUENT 2026: shared tasks for evaluating generative language model quality, in: M. Hagen, M. Potthast, B. Stein, P. Schaer...

    2026

  10. [10]

    Bevendorff, M

    J. Bevendorff, M. Fröbe, A. Greiner-Petter, A. Jakoby, M. Mayerl, P. Nakov, H. Plutz, M. Potthast, B. Stein, M. N. Ta, Y. Wang, E. Zangerle, Overview of pan 2026: Voight-kampff generative ai detection, text watermarking, multi-author writing style analysis, generative plagiarism detection, and reasoning trajectory detection, in: M. Hagen, M. Potthast, B. ...

    2026

  11. [11]

    Gehrmann, H

    S. Gehrmann, H. Strobelt, A. M. Rush, GLTR: Statistical detection and visualization of generated text, in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 2019. URL: https://aclanthology.org/P19-3019/

    2019

  12. [12]

    Mitchell, Y

    E. Mitchell, Y. Lee, A. Khazatsky, C. D. Manning, C. Finn, DetectGPT: Zero-shot machine-generated text detection using probability curvature, in: International Conference on Machine Learning (ICML), 2023, pp. 24950–24962

    2023

  13. [13]

    A. Hans, A. Schwarzschild, V. Cherepanova, H. Kazemi, A. Saha, M. Goldblum, J. Geiping, T. Gold- stein, Spotting LLMs with binoculars: Zero-shot detection of machine-generated text, in: Interna- tional Conference on Machine Learning (ICML), 2024. URL: https://arxiv.org/abs/2401.12070

    Pith/arXiv arXiv 2024

  14. [14]

    V. S. Sadasivan, A. Kumar, S. Balasubramanian, W. Wang, S. Feizi, Can AI-generated text be reliably detected?, 2023. URL: https://arxiv.org/abs/2303.11156.arXiv:2303.11156

    Pith/arXiv arXiv 2023

  15. [15]

    Krishna, Y

    K. Krishna, Y. Song, M. Karpinska, J. Wieting, M. Iyyer, Paraphrasing evades detectors of AI- generated text, but retrieval is an effective defense, Advances in Neural Information Processing Systems 36 (2023) 27469–27500

    2023

  16. [16]

    Kirchenbauer, J

    J. Kirchenbauer, J. Geiping, Y. Wen, J. Katz, I. Miers, T. Goldstein, A watermark for large language models, in: Proceedings of the 40th International Conference on Machine Learning (ICML), 2023. URL: https://arxiv.org/abs/2301.10226

    Pith/arXiv arXiv 2023

  17. [17]

    E. Reif, D. Ippolito, A. Yuan, A. Coenen, C. Callison-Burch, J. Wei, A recipe for arbitrary text style transfer with large language models, in: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2022. URL: https://aclanthology.org/2022. acl-short.94/

    2022

  18. [18]

    URL: https://huggingface.co/Qwen/Qwen3.6-35B-A3B, moE 35B total / 3B active

    Qwen Team, Qwen3.6-35B-A3B, 2026. URL: https://huggingface.co/Qwen/Qwen3.6-35B-A3B, moE 35B total / 3B active. 4-bit DWQ MLX build viamlx-community/Qwen3.6-35B-A3B-4bit-DWQ

    2026

  19. [19]

    Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, V. Stoyanov, RoBERTa: A robustly optimized BERT pretraining approach, arXiv preprint arXiv:1907.11692 (2019). URL: https://arxiv.org/abs/1907.11692

    Pith/arXiv arXiv 1907

  20. [20]

    J. W. Rae, A. Potapenko, S. M. Jayakumar, C. Hillier, T. P. Lillicrap, Compressive transformers for long-range sequence modelling, in: International Conference on Learning Representations (ICLR),

  21. [21]

    Zheng, W.-L

    L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, H. Zhang, J. E. Gonzalez, I. Stoica, Judging LLM-as-a-judge with MT-bench and chatbot arena, in: Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track,

  22. [22]

    P. He, J. Gao, W. Chen, DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing, 2021. URL: https://arxiv.org/abs/2111.09543. arXiv:2111.09543

    Pith/arXiv arXiv 2021

  23. [2020]

    URL: https://arxiv.org/abs/1911.05507

    Pith/arXiv arXiv 1911

  24. [2023]

    URL: https://arxiv.org/abs/2306.05685

    Pith/arXiv arXiv