Pith. sign in

REVIEW 2 major objections 7 minor 62 references

HACo-Det: A Study Towards Fine-Grained Machine-Generated Text Detection under Human-AI Coauthoring

T0 review · 2 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current machine-generated-text detectors, when pushed to word-level attribution in human-AI coauthored text, mostly fail: metric-based methods average 0.462 F1, while a finetuned DeBERTa reaches 0.831.

desk verdict A useful new benchmark for fine-grained MGT detection under coauthoring, but its lexical-overlap labels need human validation before the authorship claims hold. read the letter →

arxiv 2506.02959 v1 pith:Z6Z3NNSF submitted 2025-06-03 cs.CL cs.AI

Zhixiong Su , Yichen Wang , Herun Wan , Zhaohan Zhang , Minnan Luo This is my paper
classification cs.CLcs.AI
keywords machine-generatedtextdetectionfine-grainedhuman-AIcoauthoringword-levelattributionLLMparaphrasesequencelabelingAIratiobenchmarkdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that binary document-level machine-generated-text detection misses how text is actually produced today, where humans and LLMs revise each other's writing. To study the missing case, the authors build HACo-Det, an 11,200-text benchmark with word-level authorship labels, created by having instruction-tuned LLMs paraphrase selected spans of human text over multiple rounds. They retrofit seven common detectors to word-level sequence labeling and evaluate them in-domain and out-of-domain. The result is that metric-based detectors fail at fine-grained attribution, averaging 0.462 word-level F1, while a finetuned DeBERTa reaches 0.831 word-level F1, yet generalization across domains and revision modes remains unsolved. The paper's point is that fine-grained detection is a distinct, harder problem worth benchmarking separately.

What carries the argument

The central object is the HACo-Det dataset and its construction pipeline: raw human texts from four domains are split into sentences, and an instruct LLM repeatedly paraphrases randomly selected sentence spans, replacing them in the text. Word-level gold labels are assigned by a lexical-overlap rule: every word in a revised span is labeled machine-generated unless that word also appeared in the same tense in the original human span. The task is formalized as sequence labeling at the word level, with sentence labels obtained by majority voting over word predictions. This machinery turns authorship attribution into a measurable benchmark and is the basis for all reported detector scores.

What would settle it

Take a random sample of HACo-Det texts, have independent human annotators mark which words they believe the LLM wrote or changed, then measure agreement with the lexical-overlap labels, for example with Cohen's kappa. If agreement is poor, the benchmark's gold labels do not reflect human notions of authorship, and detector rankings on it would need to be re-evaluated against human labels.

Watch

Extended reading notes

Core claim

The paper's central discovery is that fine-grained, word-level attribution of authorship in human-AI coauthored text can be cast as sequence labeling, and that current document-level detectors do not transfer to it. On the HACo-Det benchmark, metric-based detectors hover near random prediction, with an average word-level F1 of 0.462, while finetuned DeBERTa reaches 0.831 word-level F1 and 0.966 sentence-level F1 in-domain, with better but still limited cross-domain and cross-model generalization. The paper also shows that context window size is a bottleneck: chunking long coauthored documents hurts detection, while larger input chunks consistently improve both word- and sentence-level performance. It further demonstrates that aggregating word or sentence labels into a document-level AI ratio is easier than exact attribution, with finetuned methods achieving about a 1.8% ratio error in-domain.

Load-bearing premise

Every score in the paper depends on the assumption that a word's authorship is determined by lexical overlap: any word in a machine-revised span that did not already appear in the same tense in the human span is called machine-generated, and this label is treated as ground truth.

Editorial extensions

If this is right

  • Word-level detection cannot be obtained by naively applying document-level metrics; on this benchmark their word-level F1 is near random.
  • Finetuned sequence taggers, especially DeBERTa, provide a usable in-domain baseline for future methods with 0.831 word-level and 0.966 sentence-level F1.
  • Longer context windows improve fine-grained detection, so chunking long coauthored documents is a real cost for current detectors.
  • Aggregated word or sentence predictions can estimate a document-level AI ratio, and finetuned methods reach about 1.8% error in-domain, suggesting this coarser use is more tractable.
  • Cross-domain and cross-model generalization is partial: training on longer, more diverse domains such as papers and Wikipedia helps, and training on GPT-4o-family revisions helps more than training on lighter revisers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The lexical-overlap labeling rule is a design choice; if readers believe authorship shifts even when words are reused, HACo-Det's scores describe revision artifacts rather than perceived authorship, and a human-judgment validation study would settle this.
  • Because detectors are evaluated against this automatic rule, 'machine-generated' here effectively means 'newly introduced wording'; a detector could score well by flagging uncommon or out-of-style words without any real authorship understanding.
  • The benchmark suggests a practical path: instead of demanding exact word attribution, downstream applications could rely on AI-ratio estimates, which remain informative even when word-level labels are noisy.
  • Extending HACo-Det to multilingual texts and multi-model collaboration would stress-test whether the finetuned advantage persists; the paper lists these as limitations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. The paper introduces HACo-Det, a benchmark for fine-grained (word- and sentence-level) machine-generated text detection in human-AI coauthored documents. It constructs 11,200 coauthored texts by having LLMs (Llama-3, Mixtral, GPT-4o, GPT-4o mini) paraphrase spans of human-written news, stories, papers, and Wikipedia articles over multiple turns. Word-level labels are assigned automatically: words in a revised span that also appeared in the same tense in the original span are labeled human; all other words in the span are labeled MGT. Seven existing detectors (finetuned DeBERTa, SeqXGPT, and metric-based methods) are adapted and evaluated in in-domain and out-of-domain settings. The central empirical finding is that metric-based detectors perform near chance (average F1 around 0.462 across word/sentence levels), while finetuned DeBERTa achieves word-level F1 0.831 and sentence-level F1 0.966, though a substantial performance gap remains. Additional analyses cover out-of-domain/out-of-model generalization, AI-ratio prediction, zero-shot detection, and the effect of context window.

Significance. The benchmark addresses a real gap: existing MGT detection mostly assumes document-level binary labels, while real human-AI writing is collaborative and fine-grained. The construction pipeline is systematic (four domains, four generators, multiple turns), and the adaptation of diverse baselines is thorough. If the automatic labeling rule is accepted as a reasonable proxy for authorship, the reported results establish a clear ranking: finetuned DeBERTa is far superior, metric-based methods are near random, and the task remains unsolved. However, the benchmark's validity rests on the lexical-overlap heuristic, which has not been validated against human judgments of authorship. The authors explicitly acknowledge in footnotes 1 and 3 that the definition is not holistic, but the abstract and conclusion state the findings without this qualification. The absence of a human annotation study is therefore a load-bearing gap. Also, all quantitative results are single-run, which limits the reliability of the finer-grained OOD comparisons.

major comments (2)
  1. [4.2, Figure 2, Tables 2–5, Abstract] The word-level ground truth in HACo-Det is defined by the Section 4.2 rule: in each LLM-revised span, a word is labeled MGT unless it 'has occurred in the same tense' in the original human span. Every reported performance number, including the abstract's '0.462 average F1' and the claim that finetuned models are superior, is measured against this automatic rule. The authors note in footnotes 1 and 3 that this is one possible definition and that a holistic definition is out of scope, but the central claim in the abstract is not similarly qualified, and the Limitations section does not list this as a limitation. The rule is not a trivial operationalization: a machine can retain a human phrase while contributing surrounding structure, and a human can accept a machine suggestion verbatim, so lexical overlap is not the same as perceived authorship. Without a human-annotation study (e.g., a subset of instances labeled by multiple raters under explicit authorship instructions) and a report of agreement with the automatic labels, the benchmark numbers may measure lexical replacement rather than coauthorship as experienced by readers. I ask the authors to either add such a validation study or visibly reframe the claims as being about detecting lexical changes introduced by LLM paraphrase.
  2. [6.1, Tables 2–4] All F1 and AUC numbers in Tables 2–4 (and the AI-ratio errors in Table 5) are reported for a single run, with no standard deviation, confidence interval, or significance test. The large gap between DeBERTa and the metric-based methods is likely robust, but several OOD comparisons rest on small differences (e.g., Table 3, News column: NPR at 0.444 vs. log prob at 0.485 for F1-W; Table 4, Llama3 column: log rank at 0.459 vs. rank at 0.461), and the arrows relative to in-domain performance are also based on single runs. Please report results over at least three random seeds for the finetuned models and at least three threshold or perturbation configurations for the metric-based methods, with means and standard deviations, and state whether the headline comparisons are statistically significant.
minor comments (7)
  1. [4.2] In the sentence 'Then we each turn select a continuous sentences span', the phrase should be 'in each turn we select' and 'sentences' should be 'sentence span' or similar.
  2. [Table 1] In the 'Domain' column, 'Wrting_prompts2,800' should read 'WritingPrompts 2,800' with a space, and 'Dagpap24' is more commonly spelled 'DagPap24'.
  3. [Footnote 4 vs. Appendix B.2] Footnote 4 states that metric-based methods without perturbation are not reformed because they are compatible with word-level detection, yet Appendix B.2 describes adapting LRR for sequence labeling; please clarify which detectors are used off-the-shelf and which are adapted, and correct the footnote if needed.
  4. [Figures 3 and 4] Figures 3 and 4 render poorly in the current PDF, with garbled characters and low resolution; please provide higher-quality figures so the length distributions, revision strengths, and cross-revision-mode results are legible.
  5. [Abstract] The phrase '0.462 average F1 score' is not defined; the main text should specify that this is the mean of word-level and sentence-level F1 over the metric-based detectors in Table 2.
  6. [6.2] The sentence 'We propose that it may be because texts from these domains are longer' is awkward; use 'We hypothesize' or 'This may be because'.
  7. [Table 5 caption] The row labels 'error-S' and 'error-W' should be explicitly defined in the caption as sentence-aggregated and word-aggregated absolute percentage error, respectively.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: detector scores are empirical measurements against an explicitly disclosed labeling rule, not predictions derived from that rule.

full rationale

The only candidate for circularity is the word-level labeling rule in Section 4.2, where a revised-span word is labeled MGT unless it 'has occurred in the same tense' in the original human span. Detector F1/AUC values are then measured against these labels. This is a benchmark operationalization rather than a circular derivation: the labels are a fixed function of the raw HWT and the LLM revision, and detector outputs are not used to define them. The paper explicitly states in footnote 1 that 'the definition of text modification and the threshold of its strength to shift authorship can be subjective' and in footnote 3 that the attribution rule is 'one of the many potential definitions of authorship transformation' and 'not holistic.' No fitted parameter is renamed as a prediction; threshold selection on the training set and evaluation on the test set is standard supervised evaluation. The AI-ratio experiment aggregates predicted labels and compares them to the same disclosed rule, which is the experiment's stated target, not a hidden input. The self-citations in the related-work section (Liu et al. 2023/2024, Wang et al. 2024, Li et al. 2025) are background references and are not load-bearing for the benchmark construction or the empirical conclusions. The remaining concern, whether lexical overlap corresponds to human-perceived authorship, is a construct-validity question about the benchmark's meaning, not a circularity in the paper's derivation chain.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No invented entities are listed because the paper's new artifact is a dataset, not a postulated physical or theoretical entity. The free parameters are the threshold and generation choices that shape the benchmark and its measured scores.

free parameters (2)
  • Metric-based detector decision threshold = not reported; selected per method via Youden index on the training split
    Section 5: all reported F1 and AUC values for metric-based detectors depend on this threshold choice.
  • Generation hyperparameters (temperature, top-p, span length) = temperature 0.6/0.8, top-p 0.9, min/max span 10/15 sentences, context windows 1024/2048
    Table 7: hand-chosen generation settings determine paraphrase strength and therefore the difficulty of the benchmark.
assumptions (4)
  • domain assumption Word-level authorship can be determined by lexical overlap: words in a revised span that appeared in the same tense in the original span are human; all other revised-span words are MGT.
    Section 4.2 and footnote 3; the entire ground truth depends on this heuristic.
  • domain assumption LLM-paraphrased sentence spans are machine-generated text whose authorship has transferred to the AI.
    Section 4.2 and Algorithm 1; this defines MGT operationally in the benchmark.
  • domain assumption The four selected corpora (XSum, WritingPrompts, Dagpap24, Wikipedia_en) and four LLMs (Llama-3, Mixtral, GPT-4o, GPT-4o mini) are representative of realistic human-AI coauthoring.
    Sections 4.1 and 4.2; the benchmark's external validity rests on this representativeness.
  • domain assumption Majority voting over word labels gives valid sentence-level ground truth.
    Section 3, equation for L_si; used to define sentence-level labels and the sentence-level detection task.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HACo-Det: A Study Towards Fine-Grained Machine-Generated Text Detection under Human-AI Coauthoring." pith.science (2026). https://pith.science/paper/Z6Z3NNSF

@misc{pith2026250602959,
  author       = {Pith},
  title        = {Pith review of: HACo-Det: A Study Towards Fine-Grained Machine-Generated Text Detection under Human-AI Coauthoring},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z6Z3NNSF}},
  note         = {Machine review of arXiv:2506.02959}
}
read the original abstract

The misuse of large language models (LLMs) poses potential risks, motivating the development of machine-generated text (MGT) detection. Existing literature primarily concentrates on binary, document-level detection, thereby neglecting texts that are composed jointly by human and LLM contributions. Hence, this paper explores the possibility of fine-grained MGT detection under human-AI coauthoring. We suggest fine-grained detectors can pave pathways toward coauthored text detection with a numeric AI ratio. Specifically, we propose a dataset, HACo-Det, which produces human-AI coauthored texts via an automatic pipeline with word-level attribution labels. We retrofit seven prevailing document-level detectors to generalize them to word-level detection. Then we evaluate these detectors on HACo-Det on both word- and sentence-level detection tasks. Empirical results show that metric-based methods struggle to conduct fine-grained detection with a 0.462 average F1 score, while finetuned models show superior performance and better generalization across domains. However, we argue that fine-grained co-authored text detection is far from solved. We further analyze factors influencing performance, e.g., context window, and highlight the limitations of current methods, pointing to potential avenues for improvement.

Figures

Figures reproduced from arXiv: 2506.02959 by the authors.

Figure 1
Figure 1. An overview of the study pipeline in HACo-Det. Firstly, we sample texts according to a curated rule from the raw human-written texts. Then we use instruct LLM to paraphrase them multiple rounds. We label the texts at the word level and sentence level, according to the detection task setting (§4.2). In the main experiment (§5), the detectors do sequence prediction at the word level, then ensemble the results to the s… view at source ↗
Figure 2
Figure 2. Grounded attribution labeling process on the word level and the sentence level setting in data construction. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. (Left) Length distribution of the texts from [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: F1 score of revision mode generalization for [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: F1 score of DeBERTa and SeqXGPT with dif [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Case of unqualified raw text (Type 1) From the stand, Rick watched. The scene seemed to be moving in slow motion. The talking, the cheering, the screaming, all seemed to be happening far away; in a movie, or an old, childhood story that comes alive inside your head whe…
Figure 7
Figure 7. Figure 7: Case of unqualified raw text (Type 2) [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Case of unqualified raw text (Type 3) This is a list of playoff records set by various teams in various categories in the National Football League during the Super Bowl Era. Wins Most Postseason Games Won, All-Time, 37 Dallas Cowboys, 1963-2019 Most Postseason Home Gam…
Figure 9
Figure 9. Figure 9: Case of unqualified raw text (Type 4) [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]
Figure 10
Figure 10. Figure 10: Case of unqualified raw text (Type 5) system: "You are a <role>." user: "polish the following text to make it more human-like, only output the polished version of the text: <passage> Here is the polished text:" [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 11
Figure 11. Figure 11: Instruction template for the generation of GPT-4o and GPT-4o-mini. [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Instruction template for the generation with Llama3. [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Instruction template for the generation with Mixtral. [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 31 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    AI@Meta. 2024. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md Llama 3 model card

    2024

  3. [3]

    Anthropic. 2024. https://www.anthropic.com/news/claude-3-5-sonnet Claude 3.5 sonnet

    2024

  4. [4]

    Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. 2023. Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature. In The Twelfth International Conference on Learning Representations

    2023

  5. [5]

    O'Reilly Media, Inc

    Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python: analyzing text with the natural language toolkit. " O'Reilly Media, Inc."

    2009

  6. [6]

    David Castillo-Bolado, Joseph Davidson, Finlay Gray, and Marek Rosa. 2024. Beyond prompts: Dynamic conversational benchmarking of large language models. arXiv preprint arXiv:2409.20222

    arXiv 2024

  7. [7]

    Tuhin Chakrabarty, Vishakh Padmakumar, and He He. 2022. Help me write a poem: Instruction tuning as a vehicle for collaborative poetry writing. arXiv preprint arXiv:2210.13669

    arXiv 2022

  8. [8]

    Savvas Chamezopoulos, Drahomira Herrmannova, Anita De Waard, Drahomira Herrmannova, Domenic Rosati, and Yury Kashnitsky. 2024. https://aclanthology.org/2024.sdp-1.2 Overview of the D ag P ap24 shared task on detecting automatically generated scientific paper . In Proceedings of the Fourth Workshop on Scholarly Document Processing (SDP 2024), pages 7--11, ...

    work page 2024

Show all 62 references
  1. [9]

    Le, and Christopher D

    Kevin Clark, Minh - Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. https://openreview.net/forum?id=r1xMH1BtvB ELECTRA: pre-training text encoders as discriminators rather than generators . In 8th International Conference on Learning Representations, ICLR 2020, Addi...

    2020

  2. [10]

    J Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, volume 2019, page 4171

    2018

  3. [11]

    Liam Dugan, Alyssa Hwang, Filip Trhl \' k, Andrew Zhu, Josh Magnus Ludan, Hainiu Xu, Daphne Ippolito, and Chris Callison-Burch. 2024. https://doi.org/10.18653/v1/2024.acl-long.674 RAID : A shared benchmark for robust evaluation of machine-generated text detectors . In Proceedi...

    2024 doi

  4. [12]

    Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hierarchical neural story generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889--898

    2018

  5. [13]

    Chujie Gao, Dongping Chen, Qihui Zhang, Yue Huang, Yao Wan, and Lichao Sun. 2024. Llm-as-a-coauthor: The challenges of detecting llm-human mixcase. arXiv preprint arXiv:2401.05952

    2024 arXiv

  6. [14]

    Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. https://doi.org/10.18653/v1/P19-3019 GLTR : Statistical detection and visualization of generated text . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrat...

    2019 doi

  7. [15]

    Google. 2024. https://ai.google.dev/gemini-api/tutorials/wordcraft Build an ai writing assistant with wordcraft

    2024

  8. [16]

    Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. arXiv preprint arXiv:2301.07597

    2023 arXiv

  9. [17]

    Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023 a . https://openreview.net/forum?id=sE7-XhLxHA De BERT av3: Improving de BERT a using ELECTRA -style pre-training with gradient-disentangled embedding sharing . In The Eleventh International Conference on Learning Representations

    2023

  10. [18]

    Xinlei He, Xinyue Shen, Zeyuan Chen, Michael Backes, and Yang Zhang. 2023 b . Mgtbench: Benchmarking machine-generated text detection. arXiv preprint arXiv:2303.14822

    2023 arXiv

  11. [19]

    Xiaomeng Hu, Pin-Yu Chen, and Tsung-Yi Ho. 2023. Radar: Robust ai-text detection via adversarial learning. Advances in neural information processing systems, 36:15077--15095

    2023

  12. [20]

    Adeeb M Jarrah, Yousef Wardat, and Patricia Fidalgo. 2023. https://api.semanticscholar.org/CorpusID:260860417 Using chatgpt in academic writing is (not) a form of plagiarism: What does the literature say? Online Journal of Communication and Media Technologies

    2023

  13. [21]

    Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. 2023. Understanding the effects of rlhf on llm generalisation and diversity. arXiv preprint arXiv:2310.06452

    2023 arXiv

  14. [22]

    Ryuto Koike, Masahiro Kaneko, and Naoaki Okazaki. 2023. How you prompt matters! even task-oriented constraints in instructions affect llm-generated text detection. arXiv preprint arXiv:2311.08369

    2023 arXiv

  15. [23]

    Laida Kushnareva, Tatiana Gaintseva, German Magai, Serguei Barannikov, Dmitry Abulkhanov, Kristian Kuznetsov, Irina Piontkovskaya, and Sergey Nikolenko. 2023. Artificial text boundary detection with topological data analysis and sliding window techniques. arXiv preprint arXiv:...

    2023 arXiv

  16. [24]

    Laida Kushnareva, Tatiana Gaintseva, German Magai, Serguei Barannikov, Dmitry Abulkhanov, Kristian Kuznetsov, Eduard Tulchinskii, Irina Piontkovskaya, and Sergey Nikolenko. 2024. Ai-generated text boundary detection with roft. In 1st Conference on Language Modeling (COLM), volume 2024

    2024

  17. [25]

    Mina Lee, Percy Liang, and Qian Yang. 2022. Coauthor: Designing a human-ai collaborative writing dataset for exploring language model capabilities. In Proceedings of the 2022 CHI conference on human factors in computing systems, pages 1--19

    2022

  18. [26]

    Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jianyun Nie, and Ji rong Wen. 2023. https://api.semanticscholar.org/CorpusID:258832847 Halueval: A large-scale hallucination evaluation benchmark for large language models . ArXiv, abs/2305.11747

    2023 arXiv

  19. [27]

    Yafu Li, Zhilin Wang, Leyang Cui, Wei Bi, Shuming Shi, and Yue Zhang. 2024 a . Spotting ai's touch: Identifying llm-paraphrased spans in text. arXiv preprint arXiv:2405.12689

    2024 arXiv

  20. [28]

    Yafu Li, Zhilin Wang, Leyang Cui, Wei Bi, Shuming Shi, and Yue Zhang. 2024 b . https://doi.org/10.18653/v1/2024.findings-acl.423 Spotting AI ' s touch: Identifying LLM -paraphrased spans in text . In Findings of the Association for Computational Linguistics: ACL 2024, pages 70...

    2024 doi

  21. [29]

    Yuanfan Li, Zhaohan Zhang, Chengzhengxu Li, Chao Shen, and Xiaoming Liu. 2025. Iron sharpens iron: Defending against attacks in machine-generated text detection with adversarial training. arXiv preprint arXiv:2502.12734

    2025 arXiv

  22. [30]

    Zhuoyan Li, Chen Liang, Jing Peng, and Ming Yin. 2024 c . The value, benefits, and concerns of generative ai-powered assistance in writing. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pages 1--25

    2024

  23. [31]

    Shengchao Liu, Xiaoming Liu, Yichen Wang, Zehua Cheng, Chengzhengxu Li, Zhaohan Zhang, Yu Lan, and Chao Shen. 2024. Does detectgpt fully utilize perturbation? bridging selective perturbation to fine-tuned contrastive learning detector would be better. In Proceedings of the 62n...

    2024

  24. [32]

    Xiaoming Liu, Zhaohan Zhang, Yichen Wang, Hang Pu, Yu Lan, and Chao Shen. 2023. Coco: Coherence-enhanced machine-generated text detection under low resource with contrastive learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pa...

    2023

  25. [33]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    2019 arXiv

  26. [34]

    Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, and Reza Shokri. 2022. Quantifying privacy risks of masked language models using membership inference attacks. arXiv preprint arXiv:2203.03929

    2022 arXiv

  27. [35]

    MistralAI. 2023. https://mistral.ai/news/mixtral-of-experts/ Mixtral of experts

    2023

  28. [36]

    Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In International Conference on Machine Learning, pages 24950--24962. PMLR

    2023

  29. [37]

    Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1797--1807

    2018

  30. [38]

    Notion. 2023. https://www.notion.so/product/ai Meet the new notion ai

    2023

  31. [39]

    OpenAI. 2024 a . https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/ Gpt-4o mini: advancing cost-efficient intelligence

    2024

  32. [40]

    OpenAI. 2024 b . https://openai.com/index/gpt-4o-system-card/ Gpt-4o system card

    2024

  33. [41]

    OpenAI. 2024 c . https://openai.com/index/introducing-canvas/ Introducing canvas

    2024

  34. [42]

    Xiao Pu, Jingyu Zhang, Xiaochuang Han, Yulia Tsvetkov, and Tianxing He. 2023. On the zero-shot generalization of machine-generated text detectors. arXiv preprint arXiv:2310.05165

    2023 arXiv

  35. [43]

    Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, Pengfei Liu, and Dong Yu. 2024. Infobench: Evaluating instruction following ability in large language models. arXiv preprint arXiv:2401.03601

    2024 arXiv

  36. [44]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21:1--67

    2020

  37. [45]

    Mohi Reza, Nathan M Laundry, Ilya Musabirov, Peter Dushniku, Zhi Yuan “Michael” Yu, Kashish Mittal, Tovi Grossman, Michael Liut, Anastasia Kuzminykh, and Joseph Jay Williams. 2024. Abscribe: Rapid exploration & organization of multiple writing variations in human-ai co-writing...

    2024

  38. [46]

    Mohi Reza, Jeb Thomas-Mitchell, Peter Dushniku, Nathan Laundry, Joseph Jay Williams, and Anastasia Kuzminykh. 2025. Co-writing with ai, on human terms: Aligning research with user demands across the writing process. arXiv preprint arXiv:2504.12488

    2025 arXiv

  39. [47]

    Zhouxing Shi, Yihan Wang, Fan Yin, Xiangning Chen, Kai-Wei Chang, and Cho-Jui Hsieh. 2024. Red teaming language model detectors with language models. Transactions of the Association for Computational Linguistics, 12:174--189

    2024

  40. [48]

    Lei Shu, Liangchen Luo, Jayakumar Hoskere, Yun Zhu, Yinxiao Liu, Simon Tong, Jindong Chen, and Lei Meng. 2024. Rewritelm: An instruction-tuned large language model for text rewriting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18970--18980

    2024

  41. [49]

    Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, et al. 2019. Release strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203

    2019 arXiv

  42. [50]

    Jinyan Su, Terry Zhuo, Di Wang, and Preslav Nakov. 2023. Detectllm: Leveraging log rank information for zero-shot detection of machine-generated text. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 12395--12412

    2023

  43. [51]

    Zhen Tao, Zhiyu Li, Runyu Chen, Dinghao Xi, and Wei Xu. 2024. Unveiling large language models generated texts: A multi-level fine-grained detection framework. arXiv preprint arXiv:2410.14231

    2024 arXiv

  44. [52]

    Nafis Irtiza Tripto, Saranya Venkatraman, Dominik Macko, Robert Moro, Ivan Srba, Adaku Uchendu, Thai Le, and Dongwon Lee. 2023. A ship of theseus: Curious cases of paraphrasing in llm-generated texts. arXiv preprint arXiv:2311.08374

    2023 arXiv

  45. [53]

    Pengyu Wang, Linyang Li, Ke Ren, Botian Jiang, Dong Zhang, and Xipeng Qiu. 2023. Seqxgpt: Sentence-level ai-generated text detection. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1144--1156

    2023

  46. [54]

    Yichen Wang, Shangbin Feng, Abe Bohan Hou, Xiao Pu, Chao Shen, Xiaoming Liu, Yulia Tsvetkov, and Tianxing He. 2024. Stumbling blocks: Stress testing the robustness of machine-generated text detectors under attacks. arXiv preprint arXiv:2402.11638

    2024 arXiv

  47. [55]

    Wikimedia. 2024. https://dumps.wikimedia.org Wikimedia downloads

    2024

  48. [56]

    Zhilin Yang. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. arXiv preprint arXiv:1906.08237

    2019 arXiv

  49. [57]

    Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. Advances in neural information processing systems, 32

    2019

  50. [58]

    Qihui Zhang, Chujie Gao, Dongping Chen, Yue Huang, Yixin Huang, Zhenyang Sun, Shilin Zhang, Weiye Li, Zhengyan Fu, Yao Wan, and Lichao Sun. 2024 a . https://doi.org/10.18653/v1/2024.findings-naacl.29 LLM -as-a-coauthor: Can mixed human-written and machine-generated text be det...

    2024 doi

  51. [59]

    Zhongping Zhang, Wenda Qin, and Bryan Plummer. 2024 b . https://doi.org/10.18653/v1/2024.findings-acl.495 Machine-generated text localization . In Findings of the Association for Computational Linguistics ACL 2024, pages 8357--8371, Bangkok, Thailand and virtual meeting. Assoc...

    2024 doi

  52. [60]

    Zhongping Zhang, Wenda Qin, and Bryan A Plummer. 2024 c . Machine-generated text localization. arXiv preprint arXiv:2402.11744

    2024 arXiv

  53. [61]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  54. [62]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.