Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Detecting LLM-Generated Short Answers and Effects on Learner Performance

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning GPT-4o on human-coded rubric labels detects LLM-generated short answers at 80% accuracy, beating GPTZero, and flagged LLM use more than doubles the odds of a correct posttest answer.

desk verdict Useful dataset and transparent fine-tuning pipeline, but the detection result measures agreement with a stylistic rubric, and the learning-effect claim is overstated. read the letter →

arxiv 2506.17196 v1 pith:Q6PLLDFW submitted 2025-06-20 cs.HC

Shambhavi Bhushan , Danielle R Thomas , Conrad Borchers , Isha Raghuvanshi , Ralph Abboud , Erin Gatz , Shivang Gupta , Kenneth Koedinger This is my paper
classification cs.HC
keywords LLM-generatedtextdetectionfine-tunedGPT-4oGPTZeroshort-answeronlinelearninghumanannotationrubricacademicintegrityoutcomes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a domain-specific detector of LLM-generated short answers is both feasible and more accurate than a generic one. In a low-stakes online tutoring lesson, two human coders labeled 1,635 open responses as LLM-generated, human-authored, or uncertain according to a stylistic rubric; a GPT-4o model fine-tuned on those labels reached 80% accuracy on held-out responses, versus GPTZero's 70%, and generalized to other lessons at 77%. The same analysis then asks whether learners suspected of LLM use actually learned less, and finds the opposite on the surface: they were more than twice as likely to answer the corresponding posttest multiple-choice question correctly. The authors read this as evidence that LLM use bypassed the learning process while inflating immediate assessment performance, rather than as genuine learning. If correct, the study offers a recipe for building accurate detectors from modest amounts of human-verified data and a warning that AI misuse can masquerade as strong performance.

What carries the argument

The machinery that carries the argument is a two-part pipeline. First, a codebook-based annotation rubric turns stylistic cues, such as over-polished and lengthy phrasing, bullet or list formatting, formulaic transitions, inverted commas, and repeated templated openings for LLM output, and misspellings, improper capitalization, casual tone, and first-person 'I' for human output, into labels assigned by two human coders, with disagreements saved as an 'uncertain' class. Second, GPT-4o is fine-tuned on 1,302 of those labeled responses and queried with a prompt aligned to the rubric on held-out responses. A 13-feature stylometric classifier serves as a transparent open-source baseline. For the learning analysis, the operating mechanism is a mixed-effects logistic regression that pairs each flagged open response with its corresponding posttest multiple-choice item, with the learner as a random intercept.

What would settle it

A concrete way to settle the central claim would be to gather responses whose true origin is known, for example by logging copy-paste events or having learners answer one item with and one without an LLM, and then compare the human coders' labels and the fine-tuned detector's predictions against that known origin; if either matches no better than chance, the detection result is an artifact of stylistic judgment rather than true authorship. A second check: if flagged learners already scored higher on earlier questions or a pretest, then the 2.37 odds ratio could reflect prior ability rather than LLM-driven bypass.

Watch

Extended reading notes

Core claim

The central claim is that a fine-tuned GPT-4o, trained on human-coded labels from a curated rubric, identifies LLM-generated short answers in an online tutoring lesson with 80% accuracy and weighted F1 of 0.78, outperforming the commercial detector GPTZero (70% accuracy, macro F1 of 0.50) and slightly beating an open-source stylometric baseline (77%). The same detector flags that learners whose open response was LLM-generated had 2.37 times higher odds (p < .001) of answering the paired posttest multiple-choice question correctly; the authors interpret this as LLM misuse that produces correct-looking performance without the corresponding learning. They additionally define their target: LLM-generated text means any LLM output used without paraphrasing or refinement, with ground truth established by human coders rather than by verified provenance.

Load-bearing premise

The load-bearing premise is that the two human coders' rubric-based labels identify which responses were truly produced by an LLM; if those labels track writing style rather than actual authorship, the detector comparison and the learning-outcome result both inherit that error.

Editorial extensions

If this is right

  • A fine-tuned detector built on a few hundred human-coded examples can reach roughly 80% accuracy in the same instructional domain, whereas a generic commercial detector misclassifies all ambiguous responses in this data.
  • Detected LLM use in one response item is a statistical marker that the learner may also obtain the correct answer on an adjacent question, so unadjusted posttest scores in self-paced online training can overstate true learning.
  • Stylometric features alone, without any LLM fine-tuning, reach 77% accuracy and outperform GPTZero, suggesting that simple transparent classifiers remain a competitive baseline for short-answer detection.
  • Adopting rubric-based human labels plus codebook-driven fine-tuning gives educators a reproducible path to context-specific detectors with lower false-positive rates than generic tools.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the ground truth is human stylistic judgment, this detector is best read as a formalization of what the coders' rubric considers LLM-like; it may not detect LLM output that is carefully paraphrased or naturally written, and it may flag polished human writing, especially writing with formal structure and correct punctuation.
  • The 2.37 odds ratio is observational and uncontrolled for prior ability; the same pattern would arise if stronger students are more likely both to use an LLM and to answer MCQs correctly, so a randomized or pretest-controlled design is needed to establish that LLM use itself bypasses learning.
  • A natural extension is to build training sets from verified provenance, for example by asking learners to answer with and without LLM assistance or by logging copy-paste events, and then compare human coder accuracy and detector accuracy on the same responses; this would quantify how much of the reported detection accuracy is about AI use versus writing style.
  • If the detector is deployed at scale, its bias toward flagging formally polished responses could create disparate treatment for students whose natural writing is formal; a fairness analysis across writing styles would be a cheap test of that risk.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper fine-tunes GPT-4o on 1,302 human-labeled short open responses from a low-stakes tutor-training lesson, compares detection accuracy and F1 against GPTZero and a stylometric logistic-regression baseline, and then uses mixed-effects logistic regression to relate detector flags of LLM use to performance on corresponding posttest multiple-choice questions. It reports 80% in-domain accuracy for the fine-tuned model versus 70% for GPTZero and 77% for the stylometric baseline, and an odds ratio of 2.37 for correct posttest answers when the open response was flagged as LLM-generated. The authors interpret these results as showing that domain-fine-tuned detectors outperform generic detectors and that suspected LLM misuse is associated with bypassing the learning process.

Significance. If the claims hold, the paper offers a practical, code-based pipeline for domain-specific LLM detection in short-answer learning contexts and a novel empirical link between suspected LLM use and immediate posttest performance. The contribution of labeled data, an explicit annotation rubric, and a reproducible fine-tuning procedure is a genuine strength, and the out-of-domain test set is a useful generalizability check. However, the headline comparison is limited by the absence of uncertainty quantification, and the central ground truth is human stylistic judgment rather than verified LLM provenance, so the significance of the detection result is currently more modest than the abstract suggests.

major comments (3)
  1. [Section 2.2 and Section 3.1] The ground truth labels are two coders' stylistic judgments (Cohen's kappa 0.64–0.68) with no external verification of actual LLM provenance, and the abstract itself defines LLM-generated text as 'evaluated by human coders.' Consequently, the headline 80%-versus-70% comparison measures how well each tool reproduces the annotation rubric, not how well it detects true LLM authorship. The fine-tuned model is trained on the same rubric, so its advantage over GPTZero may largely reflect rubric alignment rather than detector accuracy. I recommend either validating labels on a subsample with verified provenance (e.g., known human-written and known LLM-prompted responses) or reframing the RQ1 claim as 'agreement with expert human coders' and using that wording consistently in the abstract and conclusion.
  2. [Section 3.1, Tables 3–5] The reported accuracy differences (80% vs. 70% vs. 77%) are not accompanied by confidence intervals or significance tests. On a test set of 333 responses, the 3-point gap between the fine-tuned model and the stylometric baseline is plausibly within sampling noise, and even the 10-point gap versus GPTZero needs a paired test (e.g., McNemar's test) to rule out chance. The class-level comparisons are even less stable, especially the 51-case Uncertain class where GPTZero has F1=0.00 and the fine-tuned model has F1=0.34. Please report bootstrap confidence intervals or exact paired tests for overall accuracy and per-class F1.
  3. [Section 3.2] The RQ2 analysis uses the fine-tuned model's flags as 'suspected LLM misuse,' so the odds ratio (OR=2.37) inherits the rubric's stylistic signal: responses that are longer, more formal, better punctuated, or more structured are more likely to be flagged. These same features are plausibly correlated with writing proficiency, motivation, or content knowledge, which would confound the claimed link between LLM misuse and posttest MCQ performance. Because the rubric features overlap with general response quality, I ask the authors to add controls (e.g., response length, prior performance, lesson completion time) or to reanalyze using the human labels directly while discussing the residual confounding.
minor comments (5)
  1. [Section 2.2] The text says 'two researchers manually coded 1400+ responses' but then reports 1,635 labeled responses; please reconcile the counts.
  2. [Section 2.5] The method states that marginal R2 was used, but no marginal R2 value appears in Section 3.2; please report it or remove the claim.
  3. [Section 2.3] The fine-tuning section omits training hyperparameters (epochs, learning rate, batch size) and the exact prompt template, which are needed to reproduce the reported results; please add these to the paper or the repository.
  4. [Section 3.1] The sentence 'This baseline significantly outperformed GPTZero' uses 'significantly' without a statistical test; please soften or support the wording.
  5. [Section 2.3 heading] The heading 'Finetuning OpenAI's GPT-4o' should be written as 'Fine-Tuning OpenAI's GPT-4o.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the fine-tuned detector is evaluated on a held-out test set against an explicitly operationalized human-label ground truth, and the comparison with GPTZero and the stylometric baseline is externally grounded.

full rationale

The paper's central detection claim is a standard supervised-learning result: a GPT-4o model is fine-tuned on an 80% split of human-labeled responses and evaluated on a held-out 20% split (Section 2.3). The reported accuracy and F1 scores therefore measure generalization to unseen responses under the same labeling rubric, not memorization of the training labels. The ground truth is explicitly operationalized: 'we define LLM-generated text within open responses as those produced by any LLM without paraphrasing or refinement, as evaluated by human coders' (Abstract). This makes 'LLM-generated' a human-judgment construct by definition, which is a validity limitation that the authors acknowledge in Section 4 ('The study relies on human judgment as the ground truth, which may not always be correct'). But that limitation is not circular: the model's predictions are not derived from the test labels, and the comparison against GPTZero and an open-source stylometric baseline (Sections 2.4, 3.1) provides independent, external grounding for the relative performance claim. The RQ2 analysis uses detector flags to compare MCQ performance; this is a correlational analysis whose interpretation depends on the validity of the detector, but again it is not a case of a fitted parameter being renamed as a prediction. No load-bearing self-citation chain or imported uniqueness theorem appears; reference [30] is used only as a comparative benchmark for learning gains. Overall, the derivation chain is self-contained and does not reduce to its inputs by construction.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No numeric free parameters are reported. The central claim rests on the annotation rubric that defines the LLM-generated construct, the assumption that paraphrasing tools were not used, the mapping of GPTZero outputs to a three-class scheme, and the use of detector flags to infer MCQ cheating. No invented entities are introduced.

assumptions (4)
  • domain assumption Human coders can reliably distinguish LLM-generated from human-authored short answers using the rubric.
    Section 2.2 uses two coders' labels as ground truth despite Cohen's kappa of only 0.64 to 0.68; if this assumption fails, all detection and learning results inherit label error.
  • domain assumption Learners in this low-stakes training had no incentive to use paraphrasing tools, so LLM-generated text appears unmodified.
    Section 2.2 argues minimal incentive, but no data supports this; paraphrased LLM text would be mislabeled as human or uncertain.
  • ad hoc to paper GPTZero's AI/Human/Mixed categories are numerically equivalent to the 1/0/0.5 labels.
    Footnote 5 maps GPTZero outputs to the same numeric scheme; this mapping is a modeling choice that may disadvantage GPTZero on uncertain cases.
  • domain assumption Detector flags on open responses can serve as a proxy for LLM misuse on the corresponding multiple-choice question.
    Section 3.2 infers cheating across question types from open-response flags, but the MCQ itself is never assessed for AI use.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Detecting LLM-Generated Short Answers and Effects on Learner Performance." pith.science (2026). https://pith.science/paper/Q6PLLDFW

@misc{pith2026250617196,
  author       = {Pith},
  title        = {Pith review of: Detecting LLM-Generated Short Answers and Effects on Learner Performance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q6PLLDFW}},
  note         = {Machine review of arXiv:2506.17196}
}
read the original abstract

The increasing availability of large language models (LLMs) has raised concerns about their potential misuse in online learning. While tools for detecting LLM-generated text exist and are widely used by researchers and educators, their reliability varies. Few studies have compared the accuracy of detection methods, defined criteria to identify content generated by LLM, or evaluated the effect on learner performance from LLM misuse within learning. In this study, we define LLM-generated text within open responses as those produced by any LLM without paraphrasing or refinement, as evaluated by human coders. We then fine-tune GPT-4o to detect LLM-generated responses and assess the impact on learning from LLM misuse. We find that our fine-tuned LLM outperforms the existing AI detection tool GPTZero, achieving an accuracy of 80% and an F1 score of 0.78, compared to GPTZero's accuracy of 70% and macro F1 score of 0.50, demonstrating superior performance in detecting LLM-generated responses. We also find that learners suspected of LLM misuse in the open response question were more than twice as likely to correctly answer the corresponding posttest MCQ, suggesting potential misuse across both question types and indicating a bypass of the learning process. We pave the way for future work by demonstrating a structured, code-based approach to improve LLM-generated response detection and propose using auxiliary statistical indicators such as unusually high assessment scores on related tasks, readability scores, and response duration. In support of open science, we contribute data and code to support the fine-tuning of similar models for similar use cases.

Figures

Figures reproduced from arXiv: 2506.17196 by the authors.

Figure 1
Figure 1. Confusion matrices for the fine-tuned model (left) and GPTZero (right). [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OpenCoderRank: Personalized Technical Assessments with Generative AI

    cs.SE 2025-09 unverdicted novelty 4.0 of 10

    OpenCoderRank provides a self-hosted, customizable system for time-bound technical assessments with automatic grading via BERTScore and LLM evaluation.

Reference graph

Works this paper leans on

35 extracted references · 25 canonical work pages · cited by 1 Pith paper

  1. [1]

    In: Proceedings of the In- ternational Texas Congress on Advanced Scientific Research and Innovation

    Aktay, S.: Analysis of ChatGPT GPTs in Education. In: Proceedings of the In- ternational Texas Congress on Advanced Scientific Research and Innovation. pp. 49–55. Houston, TX (Aug 2024)

    work page 2024

  2. [2]

    Academic Honesty and Integrity (2023)

    Andrews, E.: Comparing AI detection tools: One instructor’s experience. Academic Honesty and Integrity (2023)

    work page 2023

  3. [3]

    Mayo Clinic Proceedings Digital Health3(1), 100184 (Nov 2024)

    Anisuzzaman, D., Malins, J.G., Friedman, P.A., Attia, Z.I.: Fine-Tuning LLMs for specialized use cases. Mayo Clinic Proceedings Digital Health3(1), 100184 (Nov 2024). https://doi.org/10.1016/j.mcpdig.2024.11.005

    doi:10.1016/j.mcpdig.2024.11.005 2024

  4. [4]

    In: Intelligent Tutoring Systems: 8th International Conference, ITS 2006, Jhongli, Taiwan, June 26-30, 2006

    Baker, R.S.d., Corbett, A.T., Koedinger, K.R., Evenson, S., Roll, I., Wagner, A.Z., Naim, M., Raspat, J., Baker, D.J., Beck, J.E.: Adapting to when students game an intelligent tutoring system. In: Intelligent Tutoring Systems: 8th International Conference, ITS 2006, Jhongli, Taiwan, June 26-30, 2006. Proceedings 8. pp. 392–

    work page 2006

  5. [5]

    Bates, D., Mächler, M., Bolker, B., Walker, S.: Package Lme4: Linear Mixed-Effects Models Using Eigen and S4 (01 2014)

    work page 2014

  6. [6]

    Ecological statistics: contemporary theory and application pp

    Bolker, B.M.: Linear and generalized linear mixed models. Ecological statistics: contemporary theory and application pp. 309–333 (2015)

    work page 2015

  7. [7]

    Journal of Applied Learning & Teaching6(2) (Jul 2023)

    Chaka, C.: Detecting AI content in responses generated by ChatGPT, YouChat, and Chatsonic: The case of five AI content detection tools. Journal of Applied Learning & Teaching6(2) (Jul 2023). https://doi.org/10.37074/jalt.2023.6.2.12

    work page doi:10.37074/jalt.2023.6.2.12 2023

  8. [8]

    Journal of English Teaching (2024), https://ejournal.uki.ac.id/index.php/jet/article/view/5619

    Chanpradit, T., Samran, P., Saengpinit, S., Subkasin, P.: Paraphrasing Strategies and Levels of Proficiency of an AI-generated QuillBot and Paraphrasing Tool: Case Study of Scientific Research Abstracts. Journal of English Teaching (2024), https://ejournal.uki.ac.id/index.php/jet/article/view/5619

    work page 2024

Show all 35 references
  1. [9]

    The International Journal of Management Education21(2), 100822 (May 2023)

    Dalalah, D., Dalalah, O.M.: The false positives and false negatives of generative AI detection tools in education and academic research: The case of ChatGPT. The International Journal of Management Education21(2), 100822 (May 2023). https://doi.org/10.1016/j.ijme.2023.100822

    2023

  2. [10]

    International Journal for Educational Integrity19(1), 15 (2023)

    Elkhatat, A.M.: Evaluating the authenticity of ChatGPT responses: a study on text-matching capabilities. International Journal for Educational Integrity19(1), 15 (2023). https://doi.org/https://doi.org/10.1007/s40979-023-00137-0

    2023 doi

  3. [11]

    International Journal for Educational Integrity 19(1) (Aug 2023)

    Elkhatat, A.M., Elsaid, K., Almeer, S.: Evaluating the efficacy of AI con- tent detection tools in differentiating between human and AI-generated text. International Journal for Educational Integrity 19(1) (Aug 2023). https://doi.org/10.1007/s40979-023-00140-5 Detecting LLM-Ge...

    2023 doi

  4. [12]

    British Journal of Educational Technology (Dec 2024)

    Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., Gašević, D.: Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance. British Journal of Educational Technology (Dec 2024). http...

    2024 doi

  5. [13]

    Teaching of Psychology (May 2024)

    Fisk, G.D.: AI or Human? Finding and Responding to Artificial In- telligence in Student Work. Teaching of Psychology (May 2024). https://doi.org/10.1177/00986283241251855

    2024 doi

  6. [14]

    Computers and Education Artificial Intelligence6, 100209 (Jan 2024)

    Fleckenstein, J., Meyer, J., Jansen, T., Keller, S.D., Köller, O., Möller, J.: Do teachers spot AI? Evaluating the detectability of AI-generated texts among stu- dent essays. Computers and Education Artificial Intelligence6, 100209 (Jan 2024). https://doi.org/10.1016/j.caeai.2...

    2024

  7. [15]

    NPJ Digital Medicine6(1), 75 (2023)

    Gao, C.A., Howard, F.M., Markov, N.S., Dyer, E.C., Ramesh, S., Luo, Y., Pearson, A.T.: Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. NPJ Digital Medicine6(1), 75 (2023)

    2023

  8. [16]

    Authorea Preprints3 (2023)

    Hadi, M.U., Qureshi, R., Shah, A., Irfan, M., Zafar, A., Shaikh, M.B., Akhtar, N., Wu, J., Mirjalili, S., et al.: A survey on large language models: Applications, challenges, limitations, and practical usage. Authorea Preprints3 (2023)

    2023

  9. [17]

    https://doi.org/10.1038/s41598-023-45644-9

    Herbold, S., Hautli-Janisz, A., Heuer, U., Kikteva, Z., Trautsch, A.: A large-scale comparisonofhuman-writtenversusChatGPT-generatedessays.ScientificReports 13(1) (Oct 2023). https://doi.org/10.1038/s41598-023-45644-9

    2023 doi

  10. [18]

    Jia, L., Qi, C., Wei, Y., Sun, H., Yang, X.: Fine-Tuning large language models for educational support: Leveraging Gagne’s Nine Events of Instruction for lesson planning (Mar 2025), https://arxiv.org/abs/2503.09276

    2025 arXiv

  11. [19]

    Indian Journal of Psychological Medicine (May 2024)

    Kar, S.K., Bansal, T., Modi, S., Singh, A.: How sensitive are the free AI-detector tools in detecting AI-generated texts? A comparison of popu- lar AI-detector tools. Indian Journal of Psychological Medicine (May 2024). https://doi.org/10.1177/02537176241247934

    2024 doi

  12. [20]

    Kirchner, J.H., Ahmad, L., Aaronson, S., Leike, J.: New AI Classifier for Indicat- ing AI-Written Text (March 2024), https://openai.com/index/new-ai-classifier-for- indicating-ai-written-text/

    2024

  13. [21]

    chatting with ChatGPT

    Menon, D., Shilpa, K.: “chatting with ChatGPT”: Analyzing the factors influencing users’ intention to Use the Open AI’s ChatGPT using the UTAUT model. Heliyon 9(11), e20962 (Oct 2023). https://doi.org/10.1016/j.heliyon.2023.e20962

    2023 doi

  14. [22]

    Explaining decisions of machine learning model for detecting short ChatGPT- generated text (Jan 2023), https://arxiv.org/abs/2301.13852

    Mitrović, S., Andreoletti, D., Ayoub, O.: ChatGPT or human? Detect and explain. Explaining decisions of machine learning model for detecting short ChatGPT- generated text (Jan 2023), https://arxiv.org/abs/2301.13852

    2023 arXiv

  15. [23]

    In: Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part II 21

    Mogessie, M., Elizabeth Richey, J., McLaren, B.M., Andres-Bray, J.M.L., Baker, R.S.: Confrustion and gaming while learning with erroneous examples in a deci- mals game. In: Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–...

    2020

  16. [24]

    Opara, C.: StyloAI: Distinguishing AI-Generated Content with Stylometric Anal- ysis (May 2024), https://arxiv.org/abs/2405.10129

    2024 arXiv

  17. [25]

    In: Proceedings of the third international conference on learning analytics and knowledge

    Pardos, Z.A., Baker, R.S., San Pedro, M.O., Gowda, S.M., Gowda, S.M.: Affective states and state tests: Investigating how affect throughout the school year predicts end of year learning outcomes. In: Proceedings of the third international conference on learning analytics and k...

    2013

  18. [26]

    IEEE International Conference on Com- puter Vision (Aug 2019), https://arxiv.org/abs/1908.07086 14 S

    Peterson, J.C., Battleday, R.M., Griffiths, T.L., Russakovsky, O.: Human uncer- tainty makes classification more robust. IEEE International Conference on Com- puter Vision (Aug 2019), https://arxiv.org/abs/1908.07086 14 S. Bhushan et al

    2019 arXiv

  19. [27]

    Proceedings of the 2022 Conference on Em- pirical Methods in Natural Language Processing p

    Plank, B.: The “Problem” of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation. Proceedings of the 2022 Conference on Em- pirical Methods in Natural Language Processing p. 10671–10682 (Dec 2022), https://aclanthology.org/2022.emnlp-main.731.pdf

    2022

  20. [28]

    Transactions of the Association for Computational Linguistics 12, 174–189 (Jan 2024), https://doi.org/10.1162/tacl_a_00639

    Shi, Z., Wang, Y., Yin, F., Chen, X., Chang, K.W., Hsieh, C.J.: Red Teaming Language Model Detectors with Language Models. Transactions of the Association for Computational Linguistics 12, 174–189 (Jan 2024), https://doi.org/10.1162/tacl_a_00639

    2024 doi

  21. [29]

    Communications of the ACM67(4), 50–59 (2024)

    Tang, R., Chuang, Y.N., Hu, X.: The science of detecting LLM-generated text. Communications of the ACM67(4), 50–59 (2024)

    2024

  22. [30]

    Thomas, D.R., Borchers, C., Kakarla, S., Lin, J., Bhushan, S., Guo, B., Gatz, E., Koedinger, K.R.: Do Tutors Learn from Equity Training and Can Generative AI Assess It? arXiv preprint arXiv:2412.11255 (2024)

    2024 arXiv

  23. [31]

    Tian, E., Cui, A.: GPTZero: Towards detection of AI-generated text using zero- shot and supervised methods" (2023), https://gptzero.me

    2023

  24. [32]

    1–66 (2025)

    Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L.S., Wong, D.F.: A survey on LLM- generatedtextdetection:Necessity,methods,andfuturedirections.Computational Linguistics pp. 1–66 (2025)

    2025

  25. [33]

    gaming the system

    Xia, M., Asano, Y., Williams, J.J., Qu, H., Ma, X.: Using information visualization to promote students’ reflection on" gaming the system" in online learning. In: Proceedings of the seventh ACM conference on learning@ scale. pp. 37–49 (2020)

    2020

  26. [34]

    Computersand Education ArtificialIntelligence p.100399 (Mar 2025)

    Yamtinah, S., Wiyarsi, A., Widarti, H.R., Shidiq, A.S., Ramadhani, D.G.: Fine- Tuning AI models for enhanced consistency and precision in chemistry educa- tional assessments. Computersand Education ArtificialIntelligence p.100399 (Mar 2025). https://doi.org/10.1016/j.caeai.2025.100399

    2025

  27. [35]

    Smart Learning Environments 11(1) (Jun 2024)

    Zhai, C., Wibowo, S., Li, L.D.: The effects of over-reliance on AI dialogue systems on students’ cognitive abilities: a systematic review. Smart Learning Environments 11(1) (Jun 2024). https://doi.org/10.1186/s40561-024-00316-7

    2024 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.