Pith. sign in

REVIEW 4 major objections 3 minor 1 cited by

Making machine text more human-like both defeats current detectors and improves the detectors trained on it.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 11:56 UTC pith:QROBQTAI

load-bearing objection Solid benchmark and a useful training recipe, but the headline generalization claim doesn't isolate the alignment mechanism—MGB vs MAGA differs in prompt diversity, decoding, and revision too, so treat the causal story as not yet proven. the 4 major comments →

arxiv 2601.04633 v2 pith:QROBQTAI submitted 2026-01-08 cs.CL

MAGA-Bench: Machine-Augment-Generated Text via Alignment Detection Benchmark

Anyang Song , Ying Cheng , Yiqian Xu , Rui Feng This is my paper
classification cs.CL
keywords machine-generated text detectionhuman alignmentreinforcement learningadversarial attacksdetector generalizationlarge language modelsbenchmarkRLDF
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish a two-sided claim about machine-generated text (MGT): if you actively align MGT to look more like human-written text (HWT), it becomes harder for existing detectors to catch, and a neural detector fine-tuned on that aligned text becomes better at detecting other MGT in the wild. To test this, the authors build the MAGA-Bench dataset, roughly a million human-machine pairs across twenty domains, twelve language models, and four alignment techniques, including their new RLDF method. The headline numbers are an average 8.13-point drop in detector AUC on aligned text and a 4.60-point rise in AUC for a RoBERTa detector fine-tuned on MAGA versus on an unaligned baseline. A sympathetic reading: the hard, aligned examples teach the detector the actual human/machine boundary rather than shallow artifacts.

Core claim

Central claim: RLDF (Reinforcement Learning from Detector Feedback) — using a detector as a reward model to fine-tune the generator so its output evades detection, then re-fine-tuning the detector on that evasive text — makes text more human-like in a way that transfers. The paper reports that on the MAGA validation set the average AUC of twelve detectors drops from 71.40 to 63.27 (−8.13), while a RoBERTa detector fine-tuned on MAGA reaches an average AUC of 87.51 on four external MGT benchmarks, up from 82.91 for the same detector fine-tuned on unaligned MGB (+4.60). The authors interpret this as evidence for the HC-Var theory in a new setting: aligned MGT helps neural detectors learn bette

What carries the argument

RLDF is a multi-round adversarial loop in which a machine-text detector is used as a reward model to fine-tune a large language model via GRPO (a policy-gradient reinforcement algorithm), and the detector is then retrained on the new, harder-to-detect text. The paper's variants, RLDF-CD and RLDF-CM, solve the 'overfitting RoBERTa cannot be a reward model' problem by cross-applying detectors across domains or across generator models, so the reward signal comes from out-of-distribution judgment rather than from a detector that has memorized its training set. RLDF is combined with three other alignment techniques (role-playing prompts, black-box prompt optimization, and self-refine), and the wh

Load-bearing premise

The load-bearing premise is that text which successfully evades a trained detector is genuinely more human-like, and that the improved performance of a detector trained on such text comes from a better human/machine decision boundary rather than from spurious cues or overlap with the evaluation sets.

What would settle it

Evaluate the same MAGA-trained and MGB-trained detectors on a held-out set drawn from domains and generator families that appear in neither training set — e.g., medical or legal text written by a model released after MAGA was built. If the MAGA-trained detector's AUC advantage over MGB shrinks to zero or reverses, the claim that aligned text improves generalization collapses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Existing detectors, neural and metric-based, lose significant performance on aligned text (average AUC drops from 71.40 to 63.27), so alignment is a realistic attack surface.
  • Fine-tuning on aligned text raises generalization AUC from 82.91 to 87.51 across four external benchmarks, so alignment can be used constructively to train better detectors.
  • Ablations show each of the four alignment methods contributes to both attack difficulty and downstream generalization, with the full pipeline giving the largest combined gain.
  • Because alignment mainly lowers the machine-text true-positive rate while the human-text true-negative rate stays stable, the attacks shift the detector's decision boundary without making human text look machine-like.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: The 4.6-point generalization gain is measured only on four existing benchmarks; the paper's own limitations section notes the absence of low-resource languages and professional domains. Testing on a fully held-out domain/generator pair would determine whether the gain comes from learned human-likeness or from accidental overlap with the benchmark distributions.
  • Inference: The core RLDF loop could be extended by using a metric-based detector (e.g., a zero-shot score) as the reward model instead of RoBERTa; if that also yields the dual attack/generalization benefit, it would suggest the mechanism is general human-likeness rather than RoBERTa-specific shortcuts.
  • Inference: The paper's 1:1 title-matched human-machine pairing likely removes surface-level topic cues; a natural test is to swap titles across pairs and re-measure the generalization gain, to see how much of the improvement depends on the pairing.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper introduces MAGA-Bench, a large-scale multilingual benchmark of machine-generated text (MGT) constructed by augmenting the generation process with four 'alignment' methods: roleplaying prompts, BPO prompt optimization, self-refine, and a proposed RLDF-CMD detector-reward reinforcement learning scheme. The authors report two main results: (i) existing detectors perform substantially worse on MAGA than on an unaligned baseline (MGB), with average AUC dropping from 71.40 to 63.27 (Table 2), and (ii) a RoBERTa detector fine-tuned on MAGA generalizes better to four external MGT detection benchmarks than the same model fine-tuned on MGB, with average AUC improving from 82.91 to 87.51 (Table 3). The paper interprets these results through the HC-Var theory: aligning MGT toward human-written text improves both attack success and the generalization of trained detectors.

Significance. If the central causal claim were established, this would be a meaningful contribution: it would provide a concrete recipe for building training sets that yield more robust MGT detectors, and it would offer a large-scale, multi-domain, multi-generator, bilingual resource (936k entries) with code and data promised publicly. The external evaluation on SemEval-2024-M4, M4GT, HC3, and MAGE is a strength, as is the ablation study in Table 4. However, the paper's headline generalization gain is currently confounded with multiple simultaneous changes to the data-generation pipeline, so the causal interpretation is not yet supported. The benchmark itself is useful regardless of the causal framing, but the manuscript's central claim needs substantially more controlled evidence.

major comments (4)
  1. [§2.3, §4.2, Table 3] The claimed generalization benefit of 'human-like alignment' is underdetermined because MAGA differs from MGB along several axes at once: 80 coarse-grained roleplaying system prompts, BPO prompt suffixes, a self-refine revision pass, and RLDF-CMD GRPO optimization. Table 4 shows that roleplaying, BPO, and self-refine each independently raise generalization AUC (e.g., +2.48, +1.96, +2.93 respectively), even without the detector-reward component. These interventions change prompt diversity, lexical diversity, and text revision behavior, all of which are known regularization mechanisms for fine-tuned classifiers. The 4.60-point average AUC gain in Table 3 is therefore compatible with a diversity/regularization effect rather than specifically with the HC-Var claim that aligned MGT is closer to HWT in the features defining a better decision boundary. To support the causal claim, the authors s
  2. [Conclusion vs. Table 2] The Conclusion states that MAGA causes an average 5.58% drop in ACC and 11.16% drop in TPR, but Table 2's averages show an ACC drop of 4.50 (65.08 → 60.59) and a TPR drop of 8.99 (54.09 → 45.10). The abstract also reports an 8.13% average AUC drop, which does match Table 2. The internal inconsistency in the two headline attack numbers weakens the paper's quantitative credibility. Please correct the Conclusion and ensure all reported aggregate numbers are derived from the same tables.
  3. [Tables 2–4, §4.1–4.2] All results are single-run point estimates with no error bars, confidence intervals, or significance tests. This is especially concerning for small differences, such as the C-MAGE row in Table 3 where R-B MAGA improves over R-B MGB by 3.73 AUC but only 0.72 ACC(@FPR=5%), and for several per-detector differences in Table 2 that are under 1 point. Without multiple seeds or bootstrapped intervals, the reader cannot assess whether the reported 4.60-point average generalization gain and the 8.13-point average attack drop are robust or partly driven by noise. At minimum, report standard deviations over at least 3 seeds for the main fine-tuning experiments.
  4. [§2.4, Eq. (1), §4.1] The attack results in Table 2 are partly a consequence of the training objective: RLDF-CMD explicitly rewards the LLM for producing text that the detector reward model classifies as human (Eq. (1) defines the reward as r_D(y) = 1 − D_phi(y)). Observing that detectors' AUC drops on such data is expected and does not by itself confirm that the text is more 'human-aligned' in any sense beyond evading that specific detector family. The paper should explicitly acknowledge this circularity and provide independent evidence of human-likeness — for example, human evaluation or a validated distributional metric that predicts both attack success and downstream generalization, rather than relying on detector evasion as the operational definition of alignment. The current multidimensional analysis (Figure 8) is suggestive but descriptive; it is not tied to the generalization outcome.
minor comments (3)
  1. [§3 / Table 2] The TNR column for R-L GPT2 in Table 2 shows a '-' for the unaligned condition, but the caption does not explain why. Please clarify whether this is not evaluated or a formatting artifact.
  2. [§4.2 / Table 3] The table caption says 'R-B CGPT is trained on HC3, so it's not boldfaced,' but the table also marks some values with an asterisk in other tables (e.g., DETree in Tables 25–28). Please unify the marking conventions and state clearly which models are excluded from best/second-best comparisons in each table.
  3. [Appendix D] The grouping of models in RLDF-CMD lists DeepSeek-V3 in both MMA and MMB (the sentence 'MMB includes ... and DeepSeek-V3' appears after MMA already contains DeepSeek-V3). This is either a typo or an important detail that needs clarification, as overlapping groups would undermine the 'cross-model' design.

Circularity Check

1 steps flagged

Attack result partly by construction because RLDF defines alignment as detector evasion; generalization claim is externally tested.

specific steps
  1. fitted input called prediction [§2.4 RLDF; Appendix D (reward definition); Abstract]
    "RLDF uses detector D as the Reward Model (RM) to fine-tune LLM via RL. This fine-tuning aims to make the text generated by the LLM harder to detect and better aligned with human-like characteristics. ... rD(y) = 1− Dϕ(y): The reward function (higher values mean y is more human-aligned and harder to detect). ... the aligned MGTs in MAGA also lead to an average decrease of 8.13% in the AUC of the selected detectors."

    RLDF's training objective is to maximize r_D(y) = 1 - D(y), i.e., to make the LLM's text classified as human by detector D. Reporting that the resulting MAGA corpus 'leads to an average decrease of 8.13% in AUC of selected detectors' is therefore, for the reward detector and closely related neural detectors, a restatement of the optimization target rather than an independent test of human-likeness. The abstract packages this drop as evidence that alignment 'facilitates robustness testing.' The generalization claim, by contrast, is evaluated on external benchmarks (Table 3) not used as rewards, so the central generalization result retains independent content. The circularity is partial and confined to the attack half of the dual claim.

full rationale

No load-bearing self-citation was found: HC-Var (Xu et al., 2023) is an external prior work, not a theorem from the present authors. The +4.60 AUC generalization gain is measured on four external datasets (S-M4, C-M4GT, C-MAGE, C-HC3) that were not used as RLDF rewards, so that central claim has independent grounding. However, the attack claim is partly tautological because RLDF operationalizes 'human-aligned' as maximizing 1 - D(y), so an AUC drop on the reward detector is the training objective by construction. The paper mitigates this by evaluating many detectors not directly used as reward models and by including non-detector alignment methods (roleplaying, BPO, self-refine), so the reduction is not total. Remaining concerns about confounded variables (prompt diversity, decoding changes, revision) and underdetermined causal mechanism are validity issues rather than circularity.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

No genuinely new entities are posited. The free parameters are hand-chosen design choices (domain grouping, prompt set, RL hyperparameters). The axioms are domain assumptions from prior theory (HC-Var) and standard RL practice, plus the external-dataset representativeness assumption.

free parameters (3)
  • RLDF-CMD domain grouping = DA={Wikipedia, wikiHow, CC News, NPR News, S2ORC}; DB={Reddit, Trustpilot, Amazon, Yahoo, Natural Questions}
    Hand-chosen based on inter-domain distance visualizations (Figure 6, §5); the grouping determines which detectors reward which LLM and directly affects alignment quality and downstream generalization.
  • 80 coarse-grained roleplaying prompts = 80 prompts
    Manually crafted (§2.3, Appendix A); prompt content shapes alignment and is not derived from data.
  • GRPO KL penalty coefficient beta and learning rate = 5e-6 LR; beta not specified in text
    Chosen by hand as RL hyperparameters (Appendix I); not fitted to target results but influence training stability and alignment strength.
axioms (4)
  • domain assumption HC-Var theory: more human-aligned MGT is closer to HWT in relevant features and improves neural-detector generalization
    Invoked in §1 and used to justify RLDF; if false, the mechanism behind MAGA's generalization gain is unsupported.
  • domain assumption RoBERTa's out-of-distribution detection relies on relevant rather than shortcut features
    Necessary for RLDF-CD/CM cross-reward to work (§2.4).
  • standard math GRPO with a detector reward provides a valid learning signal and remains stable under LoRA fine-tuning
    Standard RL assumption; no stability proof or convergence analysis provided.
  • domain assumption The four external test datasets (S-M4, C-M4GT, C-MAGE, C-HC3) are representative of wild machine text
    Used to measure generalization; the paper only tests on these four and acknowledges limitations.

pith-pipeline@v1.3.0-alltime-deepseek · 47371 in / 12022 out tokens · 118644 ms · 2026-08-03T11:56:30.284145+00:00 · methodology

0 comments
read the original abstract

Machine-Generated Text (MGT) is becoming increasingly difficult to distinguish from Human-Written Text (HWT). This trend has exacerbated malicious activities such as fake news and online fraud. The generalization ability of fine-tuned detectors relies heavily on dataset quality, and simply expanding the sources of MGT may become increasingly insufficient. Further augmentation of the generation process is required. Based on HC-Var's theory, enhancing the human-like alignment of MGT not only facilitates robustness testing of existing detectors but also boosts the generalization ability of detectors fine-tuned on such aligned MGT datasets. Therefore, we propose the \textbf{M}achine-\textbf{A}ugment-\textbf{G}enerated Text via \textbf{A}lignment (MAGA) Detection Benchmark. MAGA integrates several alignment methods, ranging from prompt construction to \textbf{G}enerator-\textbf{D}etector \textbf{A}dversarial \textbf{R}einforcement \textbf{L}earning (GDARL) and the reasoning process. In our experiments, the RoBERTa detector fine-tuned on MAGA achieves an average improvement of 4.60\% in generalization AUC. Conversely, the aligned MGTs in MAGA also lead to an average decrease of 8.13\% in the AUC of selected detectors. We hope the MAGA Benchmark will provide valuable insights for future research on the generalization ability of MGT detectors.

Figures

Figures reproduced from arXiv: 2601.04633 by Anyang Song, Rui Feng, Ying Cheng, Yiqian Xu.

Figure 1
Figure 1. Figure 1: More aligned MGT not only evades detection [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: MAGA-Bench Overview. Our dataset construction contains 20 domains and 12 generators. We evaluate [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: MAGA Pipeline 2.1 Domains Various domains raise detection difficulty (Li et al., 2024; Dugan et al., 2023b) and improve the fine-tuned RoBERTa’s performance (Wang et al., 2024c). Taking English as an example, our sources cover social conversations (Reddit), summarization (S2ORC), general knowledge (Wikipedia, wikiHow), reviews (Trustpilot, Amazon Reviews), direct Q&A (Yahoo Answers, Natural Questions), and… view at source ↗
Figure 4
Figure 4. Figure 4: RLDF (Reinforce Learning from Detectors Feedback). Fine-tune the LLM via RL using detector D as the RM, resulting in harder-to-detect and better human-aligned generated text. Fine-tune detector D (RoBERTa) on the dataset constructed from HWT and its corresponding better-aligned MGT, to obtain a detector D with stronger generalized detection capability, thus enabling multi-round adversarial fine-tuning. RLD… view at source ↗
Figure 5
Figure 5. Figure 5: RLDF-CD matrix analysis for attack and generalization. The horizontal axis represents the RM domain, and the vertical axis represents the LLM domain. the upper limit of detection capability of such metric-based methods is not as high as that of neural-based methods, their generalization ability is indeed strong, and our model cannot be compared with them. Our enhanced model, R-B MAGA, significantly outperf… view at source ↗
Figure 7
Figure 7. Figure 7: log-ppl-core there is little difference in lexical and semantic similarity relative to HWT; these metrics may be more indicative of text quality rather than the fundamental distinctions between human and machine-generated texts. MGB-M exhibits higher lexical diversity than HWT, and MAGA-M shows even greater diversity. In [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 10
Figure 10. Figure 10: Ngram overlap rate with HWT. 1-round 2-round 3-round 4-round 79.5 80.0 80.5 81.0 81.5 82.0 82.5 83.0 Model Qwen3-8B Deepseek-8B Hunyuan-7B Llama-3.1-8B gemma-3-12b Ministral-8B [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: RLDF-CM generalization AUC by RLDF round. The CM group configuration use the most significant increase in [PITH_FULL_IMAGE:figures/full_fig_p021_11.png] view at source ↗
Figure 9
Figure 9. Figure 9: Direct log ppl computed on the base/chat large [PITH_FULL_IMAGE:figures/full_fig_p021_9.png] view at source ↗
Figure 12
Figure 12. Figure 12: RLDF-CD-matrix results. Top: Attack performance; Bottom: Generalization performance. [PITH_FULL_IMAGE:figures/full_fig_p029_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: RLDF-CM matrix analysis for attack and generalization [PITH_FULL_IMAGE:figures/full_fig_p029_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: T-SNE visualization results of different domains and models. From left to right: HWT domain, MGB-M [PITH_FULL_IMAGE:figures/full_fig_p029_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Team DACTYL at PAN 2026: Bayesian Data Mixing and Empirical X-risk Minimization for AI-text Detection

    cs.CL 2026-07 conditional novelty 5.0

    A Bayesian-uncertainty text filter, partial-AUROC training, and MCGrad calibration produce the second-ranked AI-text detector (0.974 mean score) on the PAN 2026 leaderboard.

Reference graph

Works this paper leans on

86 extracted references · 3 canonical work pages · cited by 1 Pith paper

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Aaditya Bhat . 2023. https://doi.org/10.57967/hf/0326 Gpt-wiki-intro

    work page doi:10.57967/hf/0326 2023

  4. [4]

    Mervat Abassy, Kareem Elozeiri, Alexander Aziz, Minh Ngoc Ta, Raj Vardhan Tomar, Bimarsha Adhikari, Saad El Dine Ahmed, Yuxia Wang, Osama Mohammed Afzal, Zhuohan Xie, and 1 others. 2024. Llm-detectaive: a tool for fine-grained machine-generated text detection. arXiv preprint arXiv:2408.04284

    Pith/arXiv arXiv 2024

  5. [5]

    AI@Meta. 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card

    2024

  6. [6]

    Ekaterina Artemova, Jason S Lucas, Saranya Venkatraman, Jooyoung Lee, Sergei Tilga, Adaku Uchendu, and Vladislav Mikhailov. 2025. Beemo: Benchmark of expert-edited machine-generated outputs. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: ...

    2025

  7. [7]

    Navid Ayoobi, Sadat Shahriar, and Arjun Mukherjee. 2023. The looming threat of fake and llm-generated linkedin profiles: Challenges and opportunities for detection and prevention. In Proceedings of the 34th ACM conference on hypertext and social media, pages 1--10

    2023

  8. [8]

    Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. 2023. https://arxiv.org/abs/2310.05130 Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature . Preprint, arXiv:2310.05130

    Pith/arXiv arXiv 2023

  9. [9]

    T Brown and 1 others. 2020. Language models are few-shot learners. advances in neural information. In 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada. Retrived from https://proceedings. neurips. cc/pa

    2020

  10. [10]

    Tuhin Chakrabarty, Philippe Laban, and Chien-Sheng Wu. 2024. Can ai writing be salvaged? mitigating idiosyncrasies and improving human-ai alignment in the writing process through edits. arXiv preprint arXiv:2409.14509

    Pith/arXiv arXiv 2024

  11. [11]

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, and 1 others. 2024. A survey on evaluation of large language models. ACM transactions on intelligent systems and technology, 15(3):1--45

    2024

  12. [12]

    Nuo Chen, Yan Wang, Yang Deng, and Jia Li. 2024. The oscars of ai theater: A survey on role-playing with language models. arXiv preprint arXiv:2407.11484

    Pith/arXiv arXiv 2024

  13. [13]

    Yutian Chen, Hao Kang, Vivian Zhai, Liangze Li, Rita Singh, and Bhiksha Raj. 2023. Token prediction as implicit classification to identify llm-generated text. arXiv preprint arXiv:2311.08723

    Pith/arXiv arXiv 2023

  14. [14]

    Jiale Cheng, Xiao Liu, Kehan Zheng, Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, and Minlie Huang. 2023. Black-box prompt optimization: Aligning large language models without model training. arXiv preprint arXiv:2311.04155

    Pith/arXiv arXiv 2023

  15. [15]

    Joseph Cornelius, Oscar Lithgow-Serrano, Sandra Mitrovi \'c , Ljiljana Dolamic, and Fabio Rinaldi. 2024. Bust: Benchmark for the evaluation of detectors of llm-generated text. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8029--8057

    2024

  16. [16]

    Crothers, Nathalie Japkowicz, and Herna L

    Evan N. Crothers, Nathalie Japkowicz, and Herna L. Viktor. 2023. https://doi.org/10.1109/ACCESS.2023.3294090 Machine-generated text: A comprehensive survey of threat models and detection methods . IEEE Access, 11:70977--71002

    arXiv 2023

  17. [17]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Jun-Mei Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiaoling Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 179 others. 2025. https://api.semanticscholar.org/CorpusID:275789950 Deepseek-r1: Incentivizing reasoning capability in llms...

    Pith/arXiv arXiv 2025

  18. [18]

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bing-Li Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dong-Li Ji, Erhang Li, Fangyun Lin, Fucong Dai, and 179 others. 2024. https://api.semanticscholar.org/CorpusID:275118643 Deepseek-v3 technical report . ArXiv, abs/2412.19437

    Pith/arXiv arXiv 2024

  19. [19]

    dirtycomputer . 2023. https://huggingface.co/datasets/dirtycomputer/douban_movie_review douban\_movie\_review

    2023

  20. [20]

    Liam Dugan, Alyssa Hwang, Filip Trhlik, Josh Magnus Ludan, Andrew Zhu, Hainiu Xu, Daphne Ippolito, and Chris Callison-Burch. 2024. Raid: A shared benchmark for robust evaluation of machine-generated text detectors. arXiv preprint arXiv:2405.07940

    Pith/arXiv arXiv 2024

  21. [21]

    Liam Dugan, Daphne Ippolito, Arun Kirubarajan, Sherry Shi, and Chris Callison-Burch. 2023 a . Real or fake text?: Investigating human ability to detect boundaries between human-written and machine-generated text. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 12763--12771

    2023

  22. [22]

    Liam Dugan, Daphne Ippolito, Arun Kirubarajan, Sherry Shi, and Chris Callison-Burch. 2023 b . https://doi.org/10.1609/aaai.v37i11.26501 Real or fake text? investigating human ability to detect boundaries between human-written and machine-generated text . In Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Confe...

    doi:10.1609/aaai.v37i11.26501 2023

  23. [23]

    Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. https://doi.org/10.18653/v1/P19-3019 GLTR : Statistical detection and visualization of generated text . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111--116, Florence, Italy. Association for Computational Linguistics

    doi:10.18653/v1/p19-3019 2019

  24. [24]

    Google DeepMind . 2024. https://blog.google/technology/google-deepmind/google-gemini-ai-update-december-2024/#gemini-2-0-flash Google gemini ai update: Introducing gemini 2.0 flash

    2024

  25. [25]

    Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. https://arxiv.org/abs/2301.07597 How close is chatgpt to human experts? comparison corpus, evaluation, and detection . Preprint, arXiv:2301.07597

    Pith/arXiv arXiv 2023

  26. [26]

    Felix Hamborg, Norman Meuschke, Corinna Breitinger, and Bela Gipp. 2017. https://doi.org/10.5281/zenodo.4120316 news-please: A generic news crawler and extractor . In Proceedings of the 15th International Symposium of Information Science, pages 218--223

    doi:10.5281/zenodo.4120316 2017

  27. [27]

    Hans WA Hanley and Zakir Durumeric. 2024. Machine-made media: Monitoring the mobilization of machine-generated articles on misinformation and mainstream news websites. In Proceedings of the international AAAI conference on web and social media, volume 18, pages 542--556

    2024

  28. [28]

    Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. https://arxiv.org/abs/2401.12070 Spotting llms with binoculars: Zero-shot detection of machine-generated text . Preprint, arXiv:2401.12070

    Pith/arXiv arXiv 2024

  29. [29]

    Xinlei He, Xinyue Shen, Zeyuan Chen, Michael Backes, and Yang Zhang. 2023. https://arxiv.org/abs/2303.14822 Mgtbench: Benchmarking machine-generated text detection . Preprint, arXiv:2303.14822

    Pith/arXiv arXiv 2023

  30. [30]

    Yongxin He, Shan Zhang, Yixuan Cao, Lei Ma, and Ping Luo. 2025. https://api.semanticscholar.org/CorpusID:282209720 Detree: Detecting human-ai collaborative texts via tree-structured hierarchical representation learning . ArXiv, abs/2510.17489

    arXiv 2025

  31. [31]

    HongzheBi . 2023. https://huggingface.co/datasets/HongzheBi/DuReader2.0 Dureader2.0

    2023

  32. [32]

    Xiaomeng Hu, Pin-Yu Chen, and Tsung-Yi Ho. 2023. Radar: Robust ai-text detection via adversarial learning. Advances in Neural Information Processing Systems

    2023

  33. [33]

    Guanhua Huang, Yuchen Zhang, Zhe Li, Yongjian You, Mingze Wang, and Zhouwang Yang. 2024. https://api.semanticscholar.org/CorpusID:270216354 Are ai-generated text detectors robust to adversarial perturbations? ArXiv, abs/2406.01179

    Pith/arXiv arXiv 2024

  34. [34]

    OpenAI Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Mkadry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alexander Kirillov, Alex Nichol, Alex Paino, and 397 others. 2024. https://api.semanticscholar.org/CorpusID:273662196 G...

    Pith/arXiv arXiv 2024

  35. [35]

    Jingyi Zheng, Tianyi Hu, Yule Liu, Zhen Sun, Zongmin Zhang, Wenhan Dong, Zifan Peng, Xinlei He . 2025. Chasm: Unveiling covert advertisements on chinese social media. https://huggingface.co/datasets/Jingyi77/CHASM-Covert_Advertisement_on_RedNote

    2025

  36. [36]

    u chemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan G \

    Enkelejda Kasneci, Kathrin Se ler, Stefan K \"u chemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan G \"u nnemann, Eyke H \"u llermeier, and 1 others. 2023. Chatgpt for good? on opportunities and challenges of large language models for education. Learning and individual differences, 103:102274

    2023

  37. [37]

    Kerassy . 2025. https://huggingface.co/datasets/Kerassy/trustpilot-reviews-123k trustpilot-reviews-123k

    2025

  38. [38]

    Varshney, Caiming Xiong, and Richard Socher

    Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. 2019. https://arxiv.org/abs/1909.05858 Ctrl: A conditional transformer language model for controllable generation . Preprint, arXiv:1909.05858

    Pith/arXiv arXiv 2019

  39. [39]

    Mahnaz Koupaee and William Yang Wang. 2018. https://api.semanticscholar.org/CorpusID:53046555 Wikihow: A large scale text summarization dataset . ArXiv, abs/1810.09305

    Pith/arXiv arXiv 2018

  40. [40]

    lars1234 . 2025. https://huggingface.co/datasets/lars1234/baidu-baike-dataset baidu-baike-dataset

    2025

  41. [41]

    Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, and Yue Zhang. 2024. https://arxiv.org/abs/2305.13242 Mage: Machine-generated text detection in the wild . Preprint, arXiv:2305.13242

    Pith/arXiv arXiv 2024

  42. [42]

    Yudong Li, Yuqing Zhang, Zhe Zhao, Linlin Shen, Weijie Liu, Weiquan Mao, and Hui Zhang. 2022. https://aclanthology.org/2022.coling-1.344 CSL : A large-scale C hinese scientific literature dataset . In Proceedings of the 29th International Conference on Computational Linguistics, pages 3917--3923, Gyeongju, Republic of Korea. International Committee on Com...

    2022

  43. [43]

    Tencent Hunyuan Team Ao Liu, Botong Zhou, Can Xu, Chayse Zhou, Chenchen Zhang, Chengcheng Xu, Chenhao Wang, Decheng Wu, Dengpeng Wu, Dian Jiao, Dong Du, Dong Wang, Feng Zhang, Fengzong Lian, Guanghui Xu, Guanwei Zhang, Hai Wang, Haipeng Luo, Han Hu, and 231 others. 2025. https://api.semanticscholar.org/CorpusID:278783111 Hunyuan-turbos: Advancing large la...

    arXiv 2025

  44. [44]

    Xiaojun Liu, Chuang Zhang, Xiaojun Chen, Yanan Cao, and Jinpeng Li. 2020. https://api.semanticscholar.org/CorpusID:265039148 Clts: A new chinese long text summarization dataset . In Natural Language Processing and Chinese Computing

    2020

  45. [45]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. https://arxiv.org/abs/1907.11692 Roberta: A robustly optimized bert pretraining approach . Preprint, arXiv:1907.11692

    Pith/arXiv arXiv 2019

  46. [46]

    Kyle Lo, Lucy Lu Wang, Mark Neumann, Rodney Kinney, and Daniel Weld. 2020. https://doi.org/10.18653/v1/2020.acl-main.447 S 2 ORC : The semantic scholar open research corpus . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4969--4983, Online. Association for Computational Linguistics

    doi:10.18653/v1/2020.acl-main.447 2020

  47. [47]

    LZYFirecn . 2025. https://huggingface.co/datasets/LZYFirecn/weibo-hot-search weibo-hot-search

    2025

  48. [48]

    Dominik Macko, Jakub Kopal, Robert Moro, and Ivan Srba. 2024. https://arxiv.org/abs/2406.12549 Multisocial: Multilingual benchmark of machine-generated text detection of social-media texts . Preprint, arXiv:2406.12549

    Pith/arXiv arXiv 2024

  49. [49]

    Dominik Macko, Robert Moro, Adaku Uchendu, Jason Lucas, Michiharu Yamashita, Mat \'u s Pikuliak, Ivan Srba, Thai Le, Dongwon Lee, Jakub Simko, and Maria Bielikova. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.616 MULTIT u DE : Large-scale multilingual machine-generated text detection benchmark . In Proceedings of the 2023 Conference on Empirical Meth...

    doi:10.18653/v1/2023.emnlp-main.616 2023

  50. [50]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, and 1 others. 2023. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534--46594

    2023

  51. [51]

    Mistral AI . 2025. https://mistral.ai/news/mistral-medium-3 Mistral medium 3: Medium is the new large

    2025

  52. [52]

    Mistral AI Team , Albert Jiang, Alexandre Abou Chahine, Alexandre Sablayrolles, Alexis Tacnet, Alodie Boissonnet, Alok Kothari, Amélie Héliou, Andy Lo, Anna Peronnin, Antoine Meunier, Antoine Roux, Antonin Faure, Aritra Paul, Arthur Darcet, Arthur Mensch, Audrey Herblin-Stoop, Augustin Garreau, Austin Birky, and 95 others. 2024. https://huggingface.co/mis...

    2024

  53. [53]

    Edoardo Mosca, Mohamed Hesham Ibrahim Abdalla, Paolo Basso, Margherita Musumeci, and Georg Groh. 2023. https://doi.org/10.18653/v1/2023.trustnlp-1.17 Distinguishing fact from fiction: A benchmark dataset for identifying machine-generated scientific papers in the LLM era. In Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustN...

    doi:10.18653/v1/2023.trustnlp-1.17 2023

  54. [54]

    Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. https://api.semanticscholar.org/CorpusID:202621357 Justifying recommendations using distantly-labeled reviews and fine-grained aspects . In Conference on Empirical Methods in Natural Language Processing

    2019

  55. [55]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730--27744

    2022

  56. [56]

    Areg Mikael Sarvazyan, Jos\' e \' A ngel Gonz\' a lez, Paolo Rosso, and Marc Franco-Salvador. 2023 a . https://doi.org/10.1007/978-3-031-42448-9_11 Supervised machine-generated text detectors: Family and scale matters . In Experimental IR Meets Multilinguality, Multimodality, and Interaction: 14th International Conference of the CLEF Association, CLEF 202...

    work page doi:10.1007/978-3-031-42448-9_11 2023

  57. [57]

    Areg Mikael Sarvazyan, José Ángel González, Marc Franco-Salvador, Francisco Rangel, Berta Chulvi, and Paolo Rosso. 2023 b . https://arxiv.org/abs/2309.11285 Overview of autextification at iberlef 2023: Detection and attribution of machine-generated text in multiple domains . Preprint, arXiv:2309.11285

    Pith/arXiv arXiv 2023

  58. [58]

    sentence-transformers . 2021. https://huggingface.co/datasets/sentence-transformers/reddit-title-body reddit-title-body

    2021

  59. [59]

    sentence-transformers . 2024 a . https://huggingface.co/datasets/sentence-transformers/natural-questions natural-questions

    2024

  60. [60]

    sentence-transformers . 2024 b . https://huggingface.co/datasets/sentence-transformers/npr npr

    2024

  61. [61]

    sentence-transformers . 2024 c . https://huggingface.co/datasets/sentence-transformers/yahoo-answers yahoo-answers

    2024

  62. [62]

    Tatiana Shamardina, Vladislav Mikhailov, Daniil Chernianskii, Alena Fenogenova, Marat Saidov, Anastasiya Valeeva, Tatiana Shavrina, Ivan Smurov, Elena Tutubalina, and Ekaterina Artemova. 2022. https://doi.org/10.28995/2075-7182-2022-21-497-511 Findings of the the ruatd shared task 2022 on artificial text detection in russian . In Computational Linguistics...

    work page doi:10.28995/2075-7182-2022-21-497-511 2022

  63. [63]

    Namrata Shivagunde, Vladislav Lialin, and Anna Rumshisky. 2023. Larger probes tell a different story: Extending psycholinguistic datasets via in-context learning. arXiv preprint arXiv:2303.16445

    Pith/arXiv arXiv 2023

  64. [64]

    Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, Miles McCain, Alex Newhouse, Jason Blazakis, Kris McGuffie, and Jasmine Wang. 2019. https://doi.org/10.48550/ARXIV.1908.09203 Release strategies and the social impacts of language models . arXiv preprint

    Pith/arXiv arXiv doi:10.48550/arxiv.1908.09203 2019

  65. [65]

    Chris Stokel-Walker. 2022. Ai bot chatgpt writes smart essays-should professors worry? Nature

    2022

  66. [66]

    Zhenpeng Su, Xing Wu, Wei Zhou, Guangyuan Ma, and Songlin Hu. 2024. https://arxiv.org/abs/2309.02731 Hc3 plus: A semantic-invariant human chatgpt comparison corpus . Preprint, arXiv:2309.02731

    Pith/arXiv arXiv 2024

  67. [67]

    Gemma Team. 2025. https://goo.gle/Gemma3Report Gemma 3

    2025

  68. [68]

    Tencent . 2025. https://huggingface.co/tencent/Hunyuan-7B-Instruct Hunyuan-7b-instruct

    2025

  69. [69]

    Yuchuan Tian, Hanting Chen, Xutao Wang, Zheyuan Bai, Qinghua Zhang, Ruifeng Li, Chaoxi Xu, and Yunhe Wang. 2023. https://api.semanticscholar.org/CorpusID:258960584 Multiscale positive-unlabeled detection of ai-generated texts . ArXiv, abs/2305.18149

    Pith/arXiv arXiv 2023

  70. [70]

    twn39 . 2025. https://huggingface.co/datasets/twn39/dianping dianping

    2025

  71. [71]

    Adaku Uchendu, Zeyu Ma, Thai Le, Rui Zhang, and Dongwon Lee. 2021. https://arxiv.org/abs/2109.13296 Turingbench: A benchmark environment for turing test in the age of neural text generation . Preprint, arXiv:2109.13296

    Pith/arXiv arXiv 2021

  72. [72]

    Jian Wang, Shangqing Liu, Xiaofei Xie, and Yi Li. 2023 a . https://arxiv.org/abs/2304.05193 Evaluating aigc detectors on code content . Preprint, arXiv:2304.05193

    Pith/arXiv arXiv 2023

  73. [73]

    Xinglin Wang, Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Boyuan Pan, Heda Wang, Yao Hu, and Kan Li. 2024 a . https://api.semanticscholar.org/CorpusID:271903190 Coglm: Tracking cognitive development of large language models . ArXiv, abs/2408.09150

    Pith/arXiv arXiv 2024

  74. [74]

    Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohammed Afzal, Tarek Mahmoud, Giovanni Puccetti, Thomas Arnold, and 1 others. 2024 b . M4gt-bench: Evaluation benchmark for black-box machine-generated text detection. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volu...

    2024

  75. [75]

    Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohammed Afzal, Tarek Mahmoud, Giovanni Puccetti, Thomas Arnold, and 1 others. 2024 c . Semeval-2024 task 8: Multidomain, multimodel and multilingual machine-generated text detection. arXiv preprint arXiv:2404.14183

    Pith/arXiv arXiv 2024

  76. [76]

    Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Alham Fikri Aji, and Preslav Nakov. 2023 b . https://arxiv.org/abs/2305.14902 M4: Multi-generator, multi-domain, and multi-lingual black-box machine-generated text detection . Preprint, arXiv:2305.14902

    Pith/arXiv arXiv 2023

  77. [77]

    Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, and 1 others. 2021. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359

    Pith/arXiv arXiv 2021

  78. [78]

    Wong, Shu Yang, Xuebo Liu, Lidia S

    Junchao Wu, Runzhe Zhan, Derek F. Wong, Shu Yang, Xuebo Liu, Lidia S. Chao, and Min Zhang. 2024. https://api.semanticscholar.org/CorpusID:269614196 Who wrote this? the key to zero-shot llm-generated text detection is gecscore . ArXiv, abs/2405.04286

    Pith/arXiv arXiv 2024

  79. [79]

    Kangxi Wu, Liang Pang, Huawei Shen, Xueqi Cheng, and Tat-Seng Chua. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.139 LLMD et: A third party large language models generated text detection tool . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2113--2133, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.findings-emnlp.139 2023

  80. [80]

    Han Xu, Jie Ren, Pengfei He, Shenglai Zeng, Yingqian Cui, Amy Liu, Hui Liu, and Jiliang Tang. 2023. https://arxiv.org/abs/2310.01307 On the generalization of training-based chatgpt detection methods . Preprint, arXiv:2310.01307

    Pith/arXiv arXiv 2023

Showing first 80 references.