Pith. sign in

REVIEW 3 major objections 4 minor 57 references

Assessing LLM Text Detection in Educational Contexts: Does Human Contribution Affect Detection?

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current LLM-text detectors frequently mislabel lightly edited student essays as machine-written, so they are not safe for academic-integrity decisions.

desk verdict A transparent, useful benchmark that supports its main claim about detectors failing on intermediate contribution levels; the dataset is the real contribution, and the soft spots are mostly about missing confidence intervals and unvalidated simulation details. read the letter →

arxiv 2508.08096 v1 pith:ILIP3C63 submitted 2025-08-11 cs.CL cs.LG

Lukas Gehring , Benjamin Paa{ss}en This is my paper · ORCID
classification cs.CLcs.LG
keywords LLM-generatedtextdetectioneducationalassessmentcontributionlevelsfalsepositivesGEDEdatasetzero-shotsupervisedacademicintegrity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper benchmarks six widely used LLM-text detectors on a new educational dataset, GEDE, built from over 900 human-written essays and more than 12,500 machine-generated ones. The central question is whether detectors can tell how much a student contributed when an essay falls between fully human and fully machine-written. The authors define eight contribution levels, from untouched human text through LLM grammar fixes and rewrites to fully generated and deliberately 'humanized' text, and test DetectGPT, Fast-DetectGPT, Intrinsic-Dim, Ghostbuster, RoBERTa, and GPTZero on each. The main finding is that performance degrades sharply at the intermediate levels: even light LLM polishing makes detectors label human text as machine-written, and false positives persist under threshold settings aimed at avoiding false accusations. A sympathetic reader should care because false positives in education can trigger academic-integrity penalties, and this paper gives concrete evidence that current detectors are not dependable for that decision.

What carries the argument

The central machinery is the contribution-level taxonomy used to build GEDE. It orders essays along a spectrum of student effort: Human, Improve-Human (LLM corrects grammar), Rewrite-Human (LLM rewrites), Summary (LLM writes from a T5-generated summary meant to simulate student notes), Task+Summary, Task (LLM writes from the assignment prompt alone), Rewrite-LLM (a second LLM paraphrases generated text), and Humanize (DIPPER paraphrases to evade detection). This taxonomy turns the binary detector problem into a graded one, so that benchmark performance can be measured not as one number but as a curve across contribution levels and policy boundaries. The detectors themselves use standard scor

What would settle it

A decisive test would be a field study where students submit essays and then self-report exactly how much LLM editing they used; run Fast-DetectGPT on the lightly edited subset with a 5% false-positive threshold. If the false-positive rate on genuinely human-written essays turns out not to rise when the LLM merely polishes grammar, or if it rises far more than in GEDE, the paper's central claim would need revision.

Watch

Extended reading notes

Core claim

On the GEDE dataset, the paper claims, all tested detectors separate fully human from fully generated essays reasonably well, but their scores degrade sharply once the boundary includes essays with intermediate human contribution. Fast-DetectGPT achieves the highest overall ROC-AUC (0.90 on the full dataset) among open detectors and also beats the commercial GPTZero on the subsample tested, yet it still misclassifies a meaningful share of lightly improved human texts. The error pattern is consistently one of false positives: detectors over-rely on surface wording, so minor LLM edits to an otherwise student-written essay push the text into the machine-generated class. The paper further shows

Load-bearing premise

The load-bearing premise is that GEDE's simulated contribution levels reproduce how real students actually use LLMs, especially the use of T5-generated summaries in place of student-written notes and the fixed prompt templates; if real prompting differs, the reported false-positive rates may not transfer.

Editorial extensions

If this is right

  • If current detectors are used with a boundary that treats human and lightly LLM-improved essays as acceptable, overall ROC-AUC for the best detector (Fast-DetectGPT) drops to about 0.90 on the full dataset and false positives remain frequent.
  • Supervised detectors (RoBERTa, Ghostbuster) generalize poorly to contribution levels they were not trained on; any detector deployed in education should be trained or calibrated on intermediate levels, not just fully human versus fully generated text.
  • Threshold choices that cap false positives at 5% cut F1 scores sharply, so a school that wants to avoid false accusations will necessarily miss many true positives.
  • Detectors transfer better across datasets when they are zero-shot; mixing generative models makes detection harder, so real-world deployments with varied student tools will be harder than in-house benchmarks.
  • GPTZero shows no clear advantage over the open detectors on the tested subsample, suggesting costlier proprietary tools do not automatically solve the reliability problem.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The gradient of contribution levels is a reusable evaluation template: the same eight-level construction could calibrate detectors in other high-stakes settings, such as hiring assessments or professional certification, where false positives are costly.
  • If real students prompt models more idiosyncratically than the paper's fixed templates, real-world detection is likely even harder than GEDE indicates, since adding generative-model diversity already degrades scores.
  • A practical extension would be to collect 'natural' LLM-improved essays from students who self-report their editing, replacing the T5-simulated summaries with authentic student notes and comparing false-positive rates directly.
  • The paper's policy recommendation implies shifting institutional effort from detection toward assessment design (drafts, oral exams, in-class writing), though the authors do not quantify the cost of that shift.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces GEDE, a new benchmark dataset for evaluating LLM-generated-text detectors in educational contexts. GEDE contains more than 900 human-written essays and more than 12,500 LLM-generated essays organized into eight 'contribution levels' that range from fully human-written text through lightly LLM-improved and LLM-rewritten text, summary-based generation, fully LLM-generated text, and adversarial humanized text. The authors evaluate six detectors (DetectGPT, Fast-DetectGPT, Intrinsic-Dim, Ghostbuster, fine-tuned RoBERTa, and GPTZero) across these levels, varying label boundaries, generative models, threshold optimization methods, datasets, and text lengths. The central empirical claim is that most detectors perform well only on fully generated text and degrade on intermediate contribution levels, producing unacceptably high false-positive rates for lightly edited human text. The authors conclude that current detectors are not suitable for high-stakes educational deployment and make the dataset and code publicly available.

Significance. If the findings hold, this is a useful contribution to the LLM-text-detection literature: GEDE is substantially broader than prior educational benchmarks in terms of contribution levels, domains, and generative models, and the public release of dataset and code supports reproducibility. The explicit treatment of the human label boundary is a particularly valuable design, because it frames detection as a policy choice rather than a single binary task. The paper also responsibly includes a commercial detector (GPTZero) and an adversarial attack level. The main limitations are that several load-bearing methodological details and external-validity assumptions need to be addressed before the strong educational-practice conclusion can be accepted.

major comments (3)
  1. [Tables 2–5, Sec. 4.3] No confidence intervals are reported for any ROC-AUC or F1 point estimate. Several comparative claims rest on small differences, e.g., the statement in Sec. 4.3 that performance is on average 0.03 higher for Llama-generated texts, or the boundary decreases in Table 3. With the sample sizes involved, some of these differences may be within sampling error. Please provide bootstrap or DeLong confidence intervals for the AUC values and, at minimum, for the comparisons that drive the narrative.
  2. [Sec. 3.2, Discussion] The external-validity assumption behind the contribution-level taxonomy is acknowledged but not validated. Summary-based levels use T5-generated summaries 'to simulate the notes or bullet points students typically provide,' and the Improve/Rewrite levels use two fixed prompt templates. The Discussion admits that 'future research should investigate which information students actually provide to the LLM in such scenarios.' This is load-bearing because the paper's educational-practice conclusion is a transfer claim: if real student prompts and summaries differ materially from the simulation, the reported boundary-specific AUC/FPR numbers (Tables 3 and 9) may not transfer. The Improve-Human false-positive result is less affected by the T5-summary substitution, but the fixed-prompt issue remains. Please either validate with human-written summaries and varied prompt conditions, or moderate the
  3. [Sec. 4.1, Table 2] The AUC definition for the Improve-Human and Rewrite-Human rows in Table 2 is unconventional: for these levels, the negative class is the level itself and the positive class is Task, rather than comparing the level against Human. This makes the row not directly interpretable as 'can the detector detect this level against human text.' The paper later addresses the human label boundary in Sec. 4.2 and Table 3, which is the more relevant analysis, but the Table 2 caption and the accompanying text should be explicit to avoid over-reading. In particular, the claim in the abstract that detectors 'struggle to accurately classify texts of intermediate student contribution levels' should be tied to the boundary analysis and Table 3, not to the Task-pairwise rows of Table 2.
minor comments (4)
  1. [Throughout] Several typos and copyediting artifacts remain: 'BA WE' and 'PERSAUDE' in Sec. 4.5, 'Ghostbuter' in Sec. 4, 'GPTZeros'' in Sec. 4.7, and 'uni00A0' artifacts in Figure 3. The reference [50] lists the author as 'Liu Zhuang' rather than 'Zhuang Liu.' Please run a careful proofreading pass.
  2. [Sec. 4.4/Table 9] Table 9 includes 'static' threshold rows for Ghostbuster and RoBERTa that are not discussed in the text or in Figure 3a. Please either discuss them or remove them for clarity.
  3. [Sec. 4.1] The sentence 'For RoBERTa, we fine-tune a roberta-base model on texts from the Human (negative class) and Task (positive class) contribution levels' does not specify whether the evaluation in Table 2 uses the held-out 0.2 test split described in Appendix D or the full subset including training texts. Please state the evaluation split explicitly.
  4. [Sec. 4.7] GPTZero is evaluated on GEDESub (569 samples), while the other detectors are evaluated on the full dataset for Table 4. The comparison in Table 5 is valid only across detectors on GEDESub; the sentence in Sec. 4.7 comparing to 'the full dataset in Table 4' is unclear because GPTZero was not run on the full dataset. Please rephrase.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: detector performance is measured on externally constructed text levels; contribution levels are operational definitions, not fitted predictions.

full rationale

The paper's central claims are empirical measurements of off-the-shelf detectors on a newly constructed dataset. The contribution levels are operational definitions of text-generation pipelines (e.g., Improve-Human, Task, Humanize), not quantities fitted to detector outputs; the reported ROC-AUC, F1, specificity, and false-positive rates are computed against externally sourced human essays (AAE, PERSUADE, BAWE) and LLM-generated texts produced under stated prompts. No equation defines detector performance in terms of the dataset construction, and no parameter is fit to a subset and then 'predicted' on a closely related subset. The paper does not rely on a load-bearing self-citation: its references are external benchmarks, detection methods, and corpora, and no uniqueness theorem or prior result by the same authors is used to force the conclusion. The acknowledged limitations—using T5-generated summaries instead of human notes and fixed prompt templates—concern external validity and transfer to real student usage, not circularity: even if real-world prompts differ, the measured detector failures remain genuine outcomes of the stated generation protocol. The label-boundary choice (treating Rewrite-Human as human) is an explicit policy assumption that shapes how false positives are interpreted, but it is not a derivation of the result from its own inputs. The claim is falsifiable: if detectors performed well on intermediate contribution levels, the paper's conclusion would be wrong. Hence no significant circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 1 invented entities

The central claim depends on the constructed contribution levels and on the representativeness of the underlying corpora and prompt templates. These are declared domain assumptions rather than derived facts. There are no mathematical axioms beyond standard statistics.

free parameters (1)
  • FPR bound c for threshold optimization = 0.05 for the main evaluation; 0.0 to 0.5 in Figure 3b
    Chosen by hand as a policy constraint. The false-positive claim is sensitive to this value, and Figure 3b shows steep F1 decline for c < 0.1.
assumptions (4)
  • domain assumption AAE, PERSUADE, and BAWE essays are representative of student writing in educational contexts.
    Section 3.1 relies on these three corpora as realistic human-written student texts. If they are not representative, the benchmark results may not generalize.
  • domain assumption T5-generated summaries behave like real student notes or bullet points for summary-based generation.
    Introduced in Section 3.2 because the source corpora do not contain human-written summaries. The paper itself flags this in the Discussion.
  • domain assumption Fixed prompt templates capture how students actually use LLMs at each contribution level.
    Section 3.3 and Appendix C define one prompt per contribution level. Real student prompts vary, and the results depend on this simplification.
  • domain assumption DIPPER-based Humanize examples represent realistic attempts to evade detectors.
    Section 3.2 uses DIPPER as the only humanizer. Other evasion methods may behave differently, so the Humanize results are tied to this model choice.
invented entities (1)
  • Contribution levels taxonomy (Improve-Human, Rewrite-Human, Summary, Task+Summary, Task, Rewrite-LLM, Humanize)
    purpose: Define the spectrum of student LLM use studied in the benchmark.
    Definitional construct introduced by the authors. No external data validate that real students fall into these categories, and the paper itself flags the synthetic-summary limitation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Assessing LLM Text Detection in Educational Contexts: Does Human Contribution Affect Detection?." pith.science (2026). https://pith.science/paper/ILIP3C63

@misc{pith2026250808096,
  author       = {Pith},
  title        = {Pith review of: Assessing LLM Text Detection in Educational Contexts: Does Human Contribution Affect Detection?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ILIP3C63}},
  note         = {Machine review of arXiv:2508.08096}
}
read the original abstract

Recent advancements in Large Language Models (LLMs) and their increased accessibility have made it easier than ever for students to automatically generate texts, posing new challenges for educational institutions. To enforce norms of academic integrity and ensure students' learning, learning analytics methods to automatically detect LLM-generated text appear increasingly appealing. This paper benchmarks the performance of different state-of-the-art detectors in educational contexts, introducing a novel dataset, called Generative Essay Detection in Education (GEDE), containing over 900 student-written essays and over 12,500 LLM-generated essays from various domains. To capture the diversity of LLM usage practices in generating text, we propose the concept of contribution levels, representing students' contribution to a given assignment. These levels range from purely human-written texts, to slightly LLM-improved versions, to fully LLM-generated texts, and finally to active attacks on the detector by "humanizing" generated texts. We show that most detectors struggle to accurately classify texts of intermediate student contribution levels, like LLM-improved human-written texts. Detectors are particularly likely to produce false positives, which is problematic in educational settings where false suspicions can severely impact students' lives. Our dataset, code, and additional supplementary materials are publicly available at https://github.com/lukasgehring/Assessing-LLM-Text-Detection-in-Educational-Contexts.

Figures

Figures reproduced from arXiv: 2508.08096 by the authors.

Figure 1
Figure 1. Overview of the different contribution levels, from fully human-written to fully generated texts, and how the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. DetectGPT prediction scores across different [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. In-sample performance of DetectGPT, FastDetectGPT, Ghostbuster, and RoBERTa of the different threshold [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (9 more)
Figure 5
Figure 5. Figure 5: ROC-AUC scores for text with a maximum number of words on different detectors across a subset of contribution levels: Human texts are from the Human con￾tribution level, and LLM-generated texts are from the Task contribution level. Acknowledgement We gratefully acknowl…
Figure 4
Figure 4. Figure 4: Training set generalization for RoBERTa, [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 6
Figure 6. Figure 6: AAE Task vs Human statistics 250 300 350 400 450 score 0 50 100 150 200 Count Word Count Text Type Human LLM 5 10 15 20 score 0 50 100 150 200 Count Sentence Count Text Type Human LLM 20 30 40 50 score 0 50 100 150 200 250 Count Avg Sentence Length Text Type Human LLM …
Figure 7
Figure 7. Figure 7: BAWE Task vs Human statistics 250 300 350 score 0 10 20 30 Count Word Count Text Type Human LLM 10 20 score 0 10 20 30 40 50 Count Sentence Count Text Type Human LLM 20 40 60 80 score 0 20 40 60 Count Avg Sentence Length Text Type Human LLM 100 125 150 175 200 score 0 …
Figure 8
Figure 8. Figure 8: PERSUADE Task vs Human statistics PREPRINT VERSION AS PROVIDED BY THE AUTHORS [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Visualization of the changes made by both LLMs to a human-written text from the BAWE subset, belonging [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Cosine Similarity between Improve-Human and Rewrite-Human texts. [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Violin plot of the ROC-AUC over all prompt modes for all detectors on all datasets and generative models. [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: ROC Curve of different detectors at varying human label boundaries. [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 36 canonical work pages

  1. [1]

    Same same but different: Tex- tidentität, Autor_innenschaft und Schöpfung- shöhe im Angesicht von generativer KI , pages 169–182

    Amrei Bahr. Same same but different: Tex- tidentität, Autor_innenschaft und Schöpfung- shöhe im Angesicht von generativer KI , pages 169–182. De Gruyter, Berlin, Boston, 2024. doi:10.1515/9783111351490-012

    work page doi:10.1515/9783111351490-012 2024

  2. [2]

    Fast-detectgpt: Efficient zero- shot detection of machine-generated text via condi- tional probability curvature

    Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. Fast-detectgpt: Efficient zero- shot detection of machine-generated text via condi- tional probability curvature. In The Twelfth Inter- national Conference on Learning Representations , 2024

    work page 2024

  3. [3]

    Debby R. E. Cotton, Peter A. Cotton, and J. Reuben Shipway. Chatting and cheating: Ensuring academic integrity in the era of chatgpt. Innovations in Ed- ucation and Teaching International, 61(2):228–239,

    work page

  4. [4]

    Crossley, Yu Tian, Perpetual Baffour, Alex Franklin, Meg Benner, and Ulrich Boser

    Scott A. Crossley, Yu Tian, Perpetual Baffour, Alex Franklin, Meg Benner, and Ulrich Boser. A large- scale corpus for assessing written argumentation: Persuade 2.0. Assessing Writing, 61:100865, 2024. doi:10.1016/j.asw.2024.100865

    work page arXiv 2024

  5. [5]

    Bert: Pre-training of deep bidi- rectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidi- rectional transformers for language understanding. In Proceedings of the NAACL , pages 4171–4186,

    work page

  6. [6]

    Nguyen, and Isao Echizen

    Zhicheng Dou, Yuchen Guo, Ching-Chun Chang, Huy H. Nguyen, and Isao Echizen. Enhancing ro- bustness of llm-synthetic text detectors for academic writing: A comprehensive analysis. In Proceedings of the AINA, pages 266–277, 2024. doi:10.1007/978- 3-031-57916-5_23

    doi:10.1007/978- 2024

  7. [7]

    Ensuring academic integrity in the age of chatgpt: Rethinking exam de- sign, assessment strategies, and ethical ai policies in higher education

    Edmund De Leon Evangelista. Ensuring academic integrity in the age of chatgpt: Rethinking exam de- sign, assessment strategies, and ethical ai policies in higher education. Contemporary Educational Tech- nology, 17(1), 2025. doi:10.30935/cedtech/15775

    work page doi:10.30935/cedtech/15775 2025

  8. [8]

    Hello gpt! goodbye home examination? an ex- ploratory study of ai chatbots impact on university teachers’ assessment practices

    Alexandra Farazouli, Teresa Cerratto-Pargman, Klara Bolander-Laksov, and Cormac McGrath and. Hello gpt! goodbye home examination? an ex- ploratory study of ai chatbots impact on university teachers’ assessment practices. Assessment & Eval- uation in Higher Education , 49(3):363–375, 2024. doi:10.1080/02602938.2023.2241676

    work page arXiv 2024

Show all 57 references
  1. [9]

    An introduction to roc analysis

    Tom Fawcett. An introduction to roc analysis. Pattern Recognition Letters , 27(8):861–874, 2006. doi:https://doi.org/10.1016/j.patrec.2005.10.010. ROC Analysis in Pattern Recognition

    2006 doi

  2. [10]

    Prati, Bartosz Krawczyk, and Fran- cisco Herrera

    Alberto Fernández, Salvador García, Mikel Galar, Ronaldo C. Prati, Bartosz Krawczyk, and Fran- cisco Herrera. Performance Measures, pages 47–61. Springer International Publishing, Cham, Switzer- land, 2018. doi:10.1007/978-3-319-98074-4_3

    2018 doi

  3. [11]

    Keller, Olaf Köller, and Jens Möller

    Johanna Fleckenstein, Jennifer Meyer, Thor- ben Jansen, Stefan D. Keller, Olaf Köller, and Jens Möller. Do teachers spot ai? eval- uating the detectability of ai-generated texts among student essays. Computers and Edu- cation: Artificial Intelligence , 6:100209, 2024. doi:http...

    2024

  4. [12]

    Student Generative AI Survey

    Josh Freeman. Student Generative AI Survey

  5. [13]

    Watermark- ing pre-trained language models with backdoor- ing, 2023

    Chenxi Gu, Chengsong Huang, Xiaoqing Zheng, Kai-Wei Chang, and Cho-Jui Hsieh. Watermark- ing pre-trained language models with backdoor- ing, 2023. URL https://arxiv.org/abs/2210. 07543

    2023

  6. [14]

    Kwon, Ahmed Hosny Saleh Metwally, Ahmed Tlili, Simone Bassanelli, Antonio Bucchiarone, Sujit Gujar, Lennart E

    Reza Hadi Mogavi, Chao Deng, Justin Juho Kim, Pengyuan Zhou, Young D. Kwon, Ahmed Hosny Saleh Metwally, Ahmed Tlili, Simone Bassanelli, Antonio Bucchiarone, Sujit Gujar, Lennart E. Nacke, and Pan Hui. Chatgpt in education: A blessing or a curse? a qualitative study exploring e...

    2024

  7. [15]

    Mgtbench: Bench- marking machine-generated text detection

    Xinlei He, Xinyue Shen, Zeyuan Chen, Michael Backes, and Yang Zhang. Mgtbench: Bench- marking machine-generated text detection. In Pro- ceedings of the ACM CSS , page 2251–2265, 2024. doi:10.1145/3658644.3670344

    2024

  8. [16]

    Harvard undergraduate survey on generative ai, 2024

    Shikoh Hirabayashi, Rishab Jain, Nikola Jurkovi ´c, and Gabriel Wu. Harvard undergraduate survey on generative ai, 2024. URL https://arxiv.org/ abs/2406.00833

    2024 arXiv

  9. [17]

    Detecting chatgpt-generated code submis- sions in a cs1 course using machine learning models

    Muntasir Hoq, Yang Shi, Juho Leinonen, Damilola Babalola, Collin Lynch, Thomas Price, and Bita Akram. Detecting chatgpt-generated code submis- sions in a cs1 course using machine learning models. In Proceedings of the SIGCSE, page 526–532, 2024. doi:10.1145/3626252.3630826

    2024

  10. [18]

    Detecting ai assisted submissions in in- troductory programming via code anomaly

    Oscar Karnalim, Hapnes Toba, and Meliana Chris- tianti Johan. Detecting ai assisted submissions in in- troductory programming via code anomaly. Educa- tion and Information Technologies , 29(13):16841– 16866, 2024. doi:10.1007/s10639-024-12520-6

    2024 doi

  11. [19]

    A watermark for large language models

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In Pro- ceedings of the ICML , pages 17061–17084, 2023. URL https://proceedings.mlr.press/v202/ kirchenbauer23a.html

    2023

  12. [20]

    I don’t trust you (anymore)! – the effect of students’ llm use on lecturer-student trust in higher education, November 2024

    Simon Kloke, Matthew Bazanya, and Twaha Ka- teete. I don’t trust you (anymore)! – the effect of students’ llm use on lecturer-student trust in higher education, November 2024. URL http://dx.doi. org/10.58653/nche.v12i1.6

    2024 doi

  13. [21]

    Paraphrasing evades detectors of ai-generated text, but retrieval is an ef- fective defense

    Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. Paraphrasing evades detectors of ai-generated text, but retrieval is an ef- fective defense. In Advances in Neural Information Processing Systems, pages 27469–27500, 2023

    2023

  14. [22]

    Argugpt: evaluating, understanding and identifying argumentative essays generated by gpt models, 2023

    Yikang Liu, Ziyin Zhang, Wanyang Zhang, Shisen Yue, Xiaojing Zhao, Xinyuan Cheng, Yiwen Zhang, and Hai Hu. Argugpt: evaluating, understanding and identifying argumentative essays generated by gpt models, 2023

    2023

  15. [23]

    Multitude: Large-scale multilin- gual machine-generated text detection benchmark

    Dominik Macko, Robert Moro, Adaku Uchendu, Ja- son Lucas, Michiharu Yamashita, Matúš Pikuliak, Ivan Srba, Thai Le, Dongwon Lee, Jakub Simko, and Maria Bielikova. Multitude: Large-scale multilin- gual machine-generated text detection benchmark. In Proceedings of the EMNLP , pag...

  16. [24]

    Classification of human- and ai-generated texts: Investigating features for chatgpt

    Lorenz Mindner, Tim Schlippe, and Kristina Schaaff. Classification of human- and ai-generated texts: Investigating features for chatgpt. In Proceedings of the AIET , pages 152–170, 2023. doi:10.1007/978-981-99-7947-9_12

    2023 doi

  17. [25]

    Manning, and Chelsea Finn

    Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. De- tectgpt: Zero-shot machine-generated text detection using probability curvature. In Proceedings of the ICML’23, 2023. doi:10.5555/3618408.3619446

    2023

  18. [26]

    British academic written english cor- pus, 2008

    Hilary Nesi, Sheena Gardner, Paul Thompson, and Paul Wickens. British academic written english cor- pus, 2008. URL http://hdl.handle.net/20. 500.14106/2539. Literary and Linguistic Data Ser- vice

    2008

  19. [27]

    De- tecting llm-generated text in computing education: Comparative study for chatgpt cases

    Michael Sheinman Orenstrakh, Oscar Karnalim, Carlos Aníbal Suárez, and Michael Liut. De- tecting llm-generated text in computing education: Comparative study for chatgpt cases. In Pro- ceedings of the COMPSAC , pages 121–126, 2024. doi:10.1109/COMPSAC61105.2024.00027

    2024 arXiv

  20. [28]

    Assessing ai detectors in identifying ai- generated code: Implications for education

    Wei Hung Pan, Ming Jie Chok, Jonathan Leong Shan Wong, Yung Xin Shin, Yeong Shian Poon, Zhou Yang, Chun Yong Chong, David Lo, and Mei Kuan Lim. Assessing ai detectors in identifying ai- generated code: Implications for education. In Proceedings of the ICSE-SEET , page 1–11, 20...

    2024

  21. [29]

    Survey on ai- generated plagiarism detection: The impact of large language models on academic integrity

    Shushanta Pudasaini, Luis Miralles-Pechuán, David Lillis, and Marisa Llorens Salvador. Survey on ai- generated plagiarism detection: The impact of large language models on academic integrity. Journal of Academic Ethics , 2024. doi:10.1007/s10805-024- 09576-x

    2024 doi

  22. [30]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019

    2019

  23. [31]

    Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text trans- former. J. Mach. Learn. Res. , 21(1):5485–5551,

  24. [32]

    Cross-domain de- tection of gpt-2-generated technical text

    Juan Diego Rodriguez, Todd Hay, David Gros, Zain Shamsi, and Ravi Srinivasan. Cross-domain de- tection of gpt-2-generated technical text. In Pro- ceedings of the NAACL , pages 1213–1233, 2022. doi:10.18653/v1/2022.naacl-main.88

    2022 doi

  25. [33]

    Persistent homology and the upper box dimension

    Benjamin Schweinhart. Persistent homology and the upper box dimension. Discrete & Computational Geometry , 65(2):331–364, 2021. doi:10.1007/s00454-019-00145-3

    2021 doi

  26. [34]

    De- tecting and unmasking ai-generated texts through explainable artificial intelligence using stylistic fea- tures

    Aditya Shah, Prateek Ranka, Urmi Dedhia, Shruti Prasad, Siddhi Muni, and Kiran Bhowmick. De- tecting and unmasking ai-generated texts through explainable artificial intelligence using stylistic fea- tures. International Journal of Advanced Computer Science and Applications, 14...

    2023

  27. [35]

    Argu- ment annotated essays (version 2), 2017

    Christian Stab and Iryna Gurevych. Argu- ment annotated essays (version 2), 2017. URL https://tudatalib.ulb.tu-darmstadt.de/ handle/tudatalib/2422

    2017

  28. [36]

    Pars- ing argumentation structures in persuasive essays

    Christian Stab and Iryna Gurevych. Pars- ing argumentation structures in persuasive essays. Computational Linguistics , 43(3):619–659, 2017. doi:10.1162/COLI_a_00295

    2017 doi

  29. [37]

    Gptzero: Ai content detection tool

    Edward Tian. Gptzero: Ai content detection tool. https://gptzero.me, 2023. Accessed: 2025-04- 17

    2023

  30. [38]

    Detecting academic misconduct using learning analytics

    Kelly Trezise, Tracii Ryan, Paula de Barba, and Gre- gor Kennedy. Detecting academic misconduct using learning analytics. Journal of Learning Analytics, 6 (3):90–104, 2019. doi:10.18608/jla.2019.63.11

    2019 doi

  31. [39]

    Intrinsic dimension estimation for ro- bust detection of ai-generated texts

    Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Daniil Cherniavskii, Sergey Nikolenko, Evgeny Burnaev, Serguei Barannikov, and Irina Pio- ntkovskaya. Intrinsic dimension estimation for ro- bust detection of ai-generated texts. In Proceed- ings of the NeurIPS , pages ...

    2023

  32. [40]

    Turingbench: A benchmark environ- ment for turing test in the age of neural text genera- tion

    Adaku Uchendu, Zeyu Ma, Thai Le, Rui Zhang, and Dongwon Lee. Turingbench: A benchmark environ- ment for turing test in the age of neural text genera- tion. In Findings of the EMNLP, pages 2001–2016,

    2001

  33. [41]

    Ghostbuster: Detecting text ghost- written by large language models

    Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. Ghostbuster: Detecting text ghost- written by large language models. In Pro- ceedings of the NAACL , pages 1702–1717, 2024. doi:10.18653/v1/2024.naacl-long.95

    2024 doi

  34. [42]

    Seqxgpt: Sentence- level ai-generated text detection

    Pengyu Wang, Linyang Li, Ke Ren, Botian Jiang, Dong Zhang, and Xipeng Qiu. Seqxgpt: Sentence- level ai-generated text detection. In Proceedings of the EMNLP, pages 1144–1156, 2023. URL https: //aclanthology.org/2023.emnlp-main.73/. PREPRINT VERSION AS PROVIDED BY THE AUTHORS A...

    2023

  35. [43]

    M4: Multi-generator, multi-domain, and multi-lingual black-box machine-generated text detection

    Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, Thomas Arnold, Al- ham Fikri Aji, Nizar Habash, Iryna Gurevych, and Preslav Nakov. M4: Multi-generator, multi-domain, and ...

    2024

  36. [44]

    Detectrl: Benchmarking llm-generated text de- tection in real-world scenarios

    Junchao Wu, Runzhe Zhan, Derek F Wong, Shu Yang, Xinyi Yang, Yulin Yuan, and Lidia S Chao. Detectrl: Benchmarking llm-generated text de- tection in real-world scenarios. arXiv preprint arXiv:2410.23746, 2024

    2024 arXiv

  37. [45]

    A survey on llm-generated text detection: Necessity, methods, and future directions

    Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Lidia Sam Chao, and Derek Fai Wong. A survey on llm-generated text detection: Necessity, methods, and future directions. Computational Linguistics, 51 (1):275–338, 2025. doi:10.1162/coli_a_00549

    2025 doi

  38. [46]

    Zhenyu Xu and Victor S. Sheng. Detect- ing ai-generated code assignments using perplex- ity of large language models. In Proceed- ings of the AAAI’24 , pages 23155–23162, 2024. doi:10.1609/aaai.v38i21.30361

    2024 doi

  39. [47]

    Robust multi-bit natural language wa- termarking through invariant features

    KiYoon Yoo, Wonhyuk Ahn, Jiho Jang, and No- jun Kwak. Robust multi-bit natural language wa- termarking through invariant features. In Pro- ceedings of the ACL , pages 2092–2115, 2023. doi:10.18653/v1/2023.acl-long.117

    doi

  40. [48]

    William J. Youden. Index for rating diagnostic tests. Cancer, 3(1):32–35, 1950. doi:10.1002/1097- 0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2- 3

    1950 doi

  41. [49]

    Plagiarism in the context of education and evolving detection strategies

    Gasparyan Armen Yuri, Nurmashev Bekaidar, Seksenbayev Bakhytzhan, Trukhachev Vladimir I., Kostyukova Elena I., and Kitas George D. Plagiarism in the context of education and evolving detection strategies. Journal of Ko- rean Medical Science , 32(8):1220–1227, 2017. doi:10.3346...

    2017 doi

  42. [50]

    A robustly optimized bert pre-training approach with post-training

    Liu Zhuang, Lin Wayne, Shi Ya, and Zhao Jun. A robustly optimized bert pre-training approach with post-training. In Proceedings of the CCL , pages 1218–1227, 2021. URL https://aclanthology. org/2021.ccl-1.108/

    2021

  43. [51]

    Prosa: Assessing and understanding the prompt sensitiv- ity of llms

    Jingming Zhuo, Songyang Zhang, Xinyu Fang, Haodong Duan, Dahua Lin, and Kai Chen. Prosa: Assessing and understanding the prompt sensitiv- ity of llms. In Proceedings of the EMNLP , pages 1950–1976, 2024. doi:10.18653/v1/2024.findings- emnlp.108. PREPRINT VERSION AS PROVIDED BY...

    1950 doi

  44. [2019]

    PREPRINT VERSION AS PROVIDED BY THE AUTHORS Assessing LLM Text Detection in Educational Contexts 11

    doi:10.18653/v1/N19-1423. PREPRINT VERSION AS PROVIDED BY THE AUTHORS Assessing LLM Text Detection in Educational Contexts 11

    doi

  45. [2020]

    doi:10.5555/3455716.3455856

  46. [2021]

    doi:10.18653/v1/2021.findings-emnlp.172

    2021 doi

  47. [2023]

    PREPRINT VERSION AS PROVIDED BY THE AUTHORS Assessing LLM Text Detection in Educational Contexts 12

    doi:10.18653/v1/2023.emnlp-main.616. PREPRINT VERSION AS PROVIDED BY THE AUTHORS Assessing LLM Text Detection in Educational Contexts 12

    2023 doi

  48. [2024]

    doi:10.1080/14703297.2023.2190148

    2023

  49. [2025]

    URL https://www.hepi.ac.uk/2025/02/26/ student-generative-ai-survey-2025/

    Technical report, The Higher Educa- tion Policy Institute (HEPI), February 2025. URL https://www.hepi.ac.uk/2025/02/26/ student-generative-ai-survey-2025/

    2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.