Spotlights and Blindspots: Evaluating Machine-Generated Text Detection

Kailash Patil; Kevin Stowe

REVIEW 3 major objections 2 minor 2 cited by

No single machine-generated text detector excels across all datasets and metrics, with high variance and poor results on novel human texts.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-05-10 08:43 UTC

load-bearing objection Detector rankings flip a lot with dataset and metric choice, and all of them struggle on fresh creative human text, but the paper needs tighter controls on how the test sets were picked. the 3 major comments →

arxiv 2604.16607 v2 submitted 2026-04-17 cs.CL cs.AI

Spotlights and Blindspots: Evaluating Machine-Generated Text Detection

Kevin Stowe , Kailash Patil This is my paper

classification cs.CL cs.AI

keywords machine-generated text detectionevaluation benchmarksdetector performance variancedataset effectsmetric sensitivityAI text detectorshuman-written text evaluation

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper evaluates fifteen detection models from six systems plus seven trained models on seven English textual test sets and three creative human-written datasets. It demonstrates that no system performs best everywhere, that nearly all detectors succeed on some tasks, and that reported performance depends heavily on which datasets and metrics are chosen. Model rankings shift substantially across different test conditions, and detectors generally perform poorly when faced with new human-written texts from high-risk domains. A reader would care because inconsistent evaluation practices make it hard to know which tools can be trusted for applications like content moderation or academic integrity checks.

Core claim

No single system excels in all areas and nearly all are effective for certain tasks, with the representation of model performance critically linked to dataset and metric choices. There is high variance in model ranks based on datasets and metrics, and overall poor performance on novel human-written texts in high-risk domains. Methodological choices that are often assumed or overlooked are essential for clearly and accurately reflecting model performance.

What carries the argument

Comparative evaluation of 22 detection models across seven textual test sets and three creative human-written datasets using multiple metrics to expose performance differences and dependencies.

Load-bearing premise

The selected seven textual test sets and three creative human-written datasets together with the chosen metrics are representative enough to reveal general strengths and weaknesses without selection bias.

What would settle it

A new study using an independent collection of novel human-written texts from additional high-risk domains that shows one or more detectors achieving consistently high accuracy across several standard metrics would falsify the claim of overall poor performance and high variance.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Detector rankings flip a lot with dataset and metric choice, and all of them struggle on fresh creative human text, but the paper needs tighter controls on how the test sets were picked.

read the letter

The main point is that rankings of these detectors change substantially depending on the dataset and metric used, no single system wins everywhere, and performance is weak on novel creative human-written text. They tested 15 models from six systems plus seven trained detectors across seven English textual sets and three creative human datasets. That scale gives a useful empirical map of where different approaches hold up or fall down, and it makes a solid case that evaluation choices shape the apparent strengths of any given detector. The variance in ranks across conditions is the clearest new observation here, and it lines up with the broader complaint that the field has scattered benchmarks that are hard to compare directly. The paper also flags that nearly every system has at least some narrow task where it performs decently, which is a fair corrective to blanket claims about any one detector being reliable. On the soft side, the abstract gives no information on how the seven plus three datasets were selected or whether text length, domain overlap, or generation method were balanced. Without those details it is difficult to know whether the reported variance reflects general properties or just the particular mix of sources. The same goes for statistical testing and data splits; those are missing from the summary and would need to be checked in the full methods. The creative datasets are called novel and high-risk, but the criteria for novelty and risk are not spelled out, so the generalization to other high-risk domains stays tentative. This is the kind of paper that people working on detection systems or AI safety benchmarks would want to read for the side-by-side numbers. It is not a theoretical fix, but the comparisons are concrete enough to be worth having in the literature. I would send it to peer review so referees can verify the dataset construction and controls; the empirical scope is large enough to justify the effort even if revisions are needed on the methods section.

Referee Report

3 major / 2 minor

Summary. The paper evaluates 15 detection models (from six systems plus seven trained variants) on seven English textual test sets and three creative human-written datasets. It claims that no single system excels across all tasks, that nearly all detectors are effective in specific settings, that model performance rankings exhibit high variance depending on dataset and metric choices, and that detectors show overall poor performance on novel human-written texts in high-risk domains. The authors conclude that often-overlooked methodological choices are essential for accurate assessment of detector effectiveness.

Significance. If the empirical findings hold after addressing controls and representativeness, the work is significant for the MGT detection field: it supplies concrete evidence that single-benchmark claims of superiority are unreliable and that detector blind spots are dataset-dependent. This could encourage more cautious deployment and standardized multi-dataset evaluation protocols.

major comments (3)

[Abstract and §3] Abstract and §3 (Datasets): The central claims of high rank variance and poor performance on 'novel human-written texts in high-risk domains' rest on the representativeness of the seven textual test sets plus three creative datasets. No selection criteria, domain coverage analysis, or controls for text length/domain overlap are described, so the observed variance could be an artifact of the particular collection rather than a general property.
[§4 and §5] §4 (Evaluation) and §5 (Results): The reported high variance in model ranks across datasets and metrics is presented without statistical significance tests, confidence intervals, or ablation on confounders such as length and topic. This makes it impossible to determine whether the variance is robust or driven by small per-dataset sample sizes.
[§5] §5 (Results, high-risk domains subsection): The claim of 'overall poor performance on novel human-written texts in high-risk domains' is load-bearing for the 'blindspots' narrative, yet the paper does not define the high-risk domains, specify how novelty was ensured, or compare against matched human-written controls from the same sources.

minor comments (2)

[§4] Table captions and metric definitions in §4 should explicitly state whether accuracy, F1, or AUROC is used per dataset and whether thresholds are fixed or optimized.
[§3] The abstract states 'seven trained models' but the methods section should clarify whether these are fine-tuned on subsets of the test sets or held-out data to avoid leakage.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback, which helps clarify key aspects of our evaluation methodology. We address each major comment below and indicate the revisions planned for the next version of the manuscript.

read point-by-point responses

Referee: [Abstract and §3] Abstract and §3 (Datasets): The central claims of high rank variance and poor performance on 'novel human-written texts in high-risk domains' rest on the representativeness of the seven textual test sets plus three creative datasets. No selection criteria, domain coverage analysis, or controls for text length/domain overlap are described, so the observed variance could be an artifact of the particular collection rather than a general property.

Authors: We agree that the dataset selection process merits more explicit documentation to support the generalizability of our claims. The seven textual test sets were selected to cover diverse domains including news, academic writing, web content, and social media, while the three creative human-written datasets were included specifically to evaluate performance on novel, non-standard text. In the revised manuscript, we will add a dedicated subsection to §3 that details the selection criteria, provides a domain coverage analysis (e.g., via topic modeling or category breakdown), and describes controls applied for text length and domain overlap. revision: yes
Referee: [§4 and §5] §4 (Evaluation) and §5 (Results): The reported high variance in model ranks across datasets and metrics is presented without statistical significance tests, confidence intervals, or ablation on confounders such as length and topic. This makes it impossible to determine whether the variance is robust or driven by small per-dataset sample sizes.

Authors: This observation is correct and we will strengthen the statistical rigor of our analysis. In the revised §4 and §5, we will add bootstrap-derived confidence intervals for all reported metrics, apply non-parametric tests (such as the Friedman test followed by Nemenyi post-hoc comparisons) to assess the statistical significance of rank differences, and include ablation experiments that control for text length and topic by length-matching and topic-stratified subsampling across datasets. revision: yes
Referee: [§5] §5 (Results, high-risk domains subsection): The claim of 'overall poor performance on novel human-written texts in high-risk domains' is load-bearing for the 'blindspots' narrative, yet the paper does not define the high-risk domains, specify how novelty was ensured, or compare against matched human-written controls from the same sources.

Authors: We will revise the high-risk domains subsection to provide clearer definitions and supporting details. High-risk domains are operationalized as settings where undetected machine-generated text carries substantial societal consequences (education, journalism, and scientific publishing); the creative datasets serve as proxies for novel human text in these areas. Novelty was ensured via temporal separation (texts created after the detectors' training cutoffs) and source verification. We will add explicit definitions, a description of the novelty protocol, and, where source-matched human controls are available in our collection, direct comparisons to better ground the performance claims. revision: partial

Circularity Check

0 steps flagged

No circularity: purely empirical evaluation of existing detectors on external datasets.

full rationale

The paper performs direct empirical benchmarking of 15+ detection models (from six systems plus seven trained ones) on seven English textual test sets and three creative human-written datasets. No derivations, equations, fitted parameters, or predictions are present; all claims about rank variance, lack of universal best system, and poor performance on novel high-risk texts follow immediately from the reported evaluation results. No self-citations are invoked as load-bearing uniqueness theorems or ansatzes. The analysis is self-contained against external model outputs and datasets, satisfying the default expectation for non-circular empirical work.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The work is an empirical benchmarking study that relies on pre-existing detection models and standard NLP evaluation practices rather than new theoretical constructs or fitted parameters.

axioms (1)

domain assumption The selected English-language test sets and creative human datasets adequately sample the space of machine-generated and human text that detectors will encounter in practice.
Invoked implicitly by treating performance on these sets as diagnostic of general detector quality.

pith-pipeline@v0.9.0 · 5457 in / 1193 out tokens · 38405 ms · 2026-05-10T08:43:28.489051+00:00 · methodology

0 comments

read the original abstract

With the rise of generative language models, machine-generated text detection has become a critical challenge. A wide variety of models is available, but inconsistent datasets, evaluation metrics, and assessment strategies obscure comparisons of model effectiveness. To address this, we evaluate 15 different detection models from six distinct systems, as well as seven trained models, across seven English-language textual test sets and three creative human-written datasets. We provide an empirical analysis of model performance, the influence of training and evaluation data, and the impact of key metrics. We find that no single system excels in all areas and nearly all are effective for certain tasks, and the representation of model performance is critically linked to dataset and metric choices. We find high variance in model ranks based on datasets and metrics, and overall poor performance on novel human-written texts in high-risk domains. Across datasets and metrics, we find that methodological choices that are often assumed or overlooked are essential for clearly and accurately reflecting model performance.

Figures

Figures reproduced from arXiv: 2604.16607 by Kailash Patil, Kevin Stowe.

**Figure 1.** Figure 1: F1 and AUC (marked with x) for each model on each dataset. For the pretrained transformers, these were trained on in-domain training data from the respective dataset. other partitions from these resources: before evaluation, we have ensured that there is no overlap between the training data and our evaluation sets. For our training, we create training sets of 10k texts for each dataset by random sampling … view at source ↗

**Figure 2.** Figure 2: F1 and AUROC scores for cross-trained models. Bold models are those trained on the in-domain [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Figure 3: Model error as a function of word count (log), punctuation (%), repeated words (%), and perplexity [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

**Figure 4.** Figure 4: F1, AUROC, and TPR@FPR 1% performance for imbalanced datasets. The x-axis reflects the [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗

discussion (0)

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

ARB: A Matched Authorship-Rewriting Benchmark Dataset for AI-Text Detector Evaluation
cs.CL 2026-07 conditional novelty 6.0

A matched four-regime benchmark shows AI-text detectors catch direct LLM output but lose most of their recall on human text rewritten by an LLM.
When Words Predict Workload
cs.DC 2026-07 conditional novelty 6.0

Linguistic features plus a closed-form latency threshold route patent-claim LLM requests before edge GPU allocation, cutting misroutes from 0.85 to ~0.09 while bounding VRAM at 4.82 GiB.

Reference graph

Works this paper leans on

17 extracted references · 17 canonical work pages · cited by 2 Pith papers · 1 internal anchor

[1]

Spotlights and Blindspots: Evaluating Machine-Generated Text Detection

Introduction Recent years have witnessed remarkable advance- ments in generative systems capable of produc- ing realistic video, audio, and text. While these technologies offer significant benefits, they also introduce serious challenges in verifying the ori- gin of content. Distinguishing between machine- generatedandhuman-writtentext,inparticular,has be...

work page internal anchor Pith review Pith/arXiv arXiv 2025
[2]

Ta- ble 1 summarizes the selected model variants, de- tailingtheiroriginalevaluationdata, metrics, andre- ported performance

Systems We assemble a diverse collection of contemporary machine-generated text detection systems, includ- ing zero-shot and trained public models, pretrained transformers, and feature-based approaches. Ta- ble 1 summarizes the selected model variants, de- tailingtheiroriginalevaluationdata, metrics, andre- ported performance. Our goal is to explore impac...

work page 2025
[3]

Data To ensure standardized, robust evaluations, we establish a unified dataset comprising seven test sets derived from four benchmark datasets: MAGE (aka Deepfake) (Li et al., 2024):This datasetconsistsofa447khumanandAI-generated text samples covering diverse models and method- ologies. We extract three test sets: (1) a class- balanced sample of 10k from...

work page 2024
[4]

All positive

Experiments We evaluate each system on each of the eight datasets described above. For comparison include a trivial baseline system (All positive) that assigns every sample a score of 1 (machine-generated).3 For our initial evaluation, we use F1 score (thresh- old=0.5)andAUROC;wediscussmoreonmetrics in Section 5. Results are shown in Figure 1. We start by...

work page 2025
[5]

Threshold by EER

Metrics Our dataset analysis reveals divergences between F1 and AUROC metrics: while Binoculars, BiS- cope, and Fast-DetectGPT achieve strong AUROC scores despite comparatively low F1 scores, De- Threshold 0.5 Threshold by EER Threshold invariant Model Precision Recall F1 Accuracy AvgRec Precision Recall F1 Accuracy AvgRec AUROC TPR@FPR 1% TPR@FPR .01% Va...

work page
[6]

(2024), who explicitly advocate for AUROC’s threshold- agnostic benefits, and Hans et al

Analysis To investigate performance differences, we ex- plore four key textual attributes commonly used for machine-generated text detection: length (in words), punctuation, repetition, and perplexity under the facebook/opt-1.3b (Zhang et al., 2022),definedinAppendixD.Weploteachmodels’ error rates against each of these attributes (Figure 5With the notable...

work page 2022
[7]

Model behavior patterns in different ways for each attribute

with the goal of identifying trends in these at- tributes that may contribute to disparity in model performance. Model behavior patterns in different ways for each attribute. For word count, models exhibit highly variable performance on low-word count texts, but typically improve as the text length in- creases. However, certain of the DeTeCtive vari- ants...

work page
[8]

Model performance varies sub- stantially across different evaluation frameworks, with each system under specific metric and data conditions

Conclusions In this work we demonstrate that the choice of datasets and metrics critically influences our as- sessment of machine-generated text detection sys- tem capabilities. Model performance varies sub- stantially across different evaluation frameworks, with each system under specific metric and data conditions. This underscores the necessity of cont...

work page 2024
[9]

Ethical Considerations The primary ethical consideration surrounding this work is the ethical application of machine- generated text detection models. We aim to avoid making claims about the general usage of these models, and whether it is appropriate, but under- stand there are ethical implications for employing automated systems for decision making that...

work page
[10]

Limitations While our evaluation spans a diverse range of mod- els, it is not exhaustive—many systems and bench- marks fall outside our analysis. We demonstrate that our core findings (evaluation variance on differ- ent datasets, importance of metrics, and relatively weak performance on human-written texts) hold across a representative sample of prevalent...

work page
[11]

References Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Co- jocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023. The fal- con series of open language models. Tal August, Maarten Sap, Elizabeth C...

work page 2023
[12]

Amrita Bhattacharjee, Raha Moraffah, Joshua Gar- land, and Huan Liu

Longformer: The long-document trans- former. Amrita Bhattacharjee, Raha Moraffah, Joshua Gar- land, and Huan Liu. 2024. Eagle: A domain generalization framework for ai-generated text detection. Steven Bird, Edward Loper, and Ewan Klein. 2009.Natural Language Processing with Python. O’Reilly Media Inc. Scott A. Crossley, Yu Tian, Perpetual Baffour, Alex Fr...

work page 2024
[13]

ZeyanLiu, ZijunYao, FengjunLi, andBoLuo.2024

Openorca: An open dataset of gpt aug- mented flan reasoning traces. ZeyanLiu, ZijunYao, FengjunLi, andBoLuo.2024. On the detectability of chatgpt content: Bench- marking, methodology, and evaluation through the lens of academic writing. InProceedings of the 2024 on ACM SIGSAC Conference on Com- puter and Communications Security, CCS ’24, page 2236–2250, N...

work page 2024
[14]

Does human collaboration enhance the accuracy of identifying llm-generated deepfake texts? Adaku Uchendu, Zeyu Ma, Thai Le, Rui Zhang, and Dongwon Lee. 2021. TURINGBENCH: A benchmarkenvironmentforTuringtestintheage of neural text generation. InFindings of the As- sociation for Computational Linguistics: EMNLP 2021, pages 2001–2016, Punta Cana, Domini- can...

work page 2021
[15]

InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3964–3992, Bangkok, Thailand

M4GT-bench: Evaluation benchmark for black-box machine-generated text detection. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3964–3992, Bangkok, Thailand. Association for Computa- tional Linguistics. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Ant...

work page 2020
[16]

Kangxi Wu, Liang Pang, Huawei Shen, Xueqi Cheng, and Tat-Seng Chua

Detectrl: Benchmarkingllm-generatedtext detection in real-world scenarios. Kangxi Wu, Liang Pang, Huawei Shen, Xueqi Cheng, and Tat-Seng Chua. 2023. LLMDet: A third party large language models generated text detection tool. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 2113–2133, Singapore. Association for Compu- tational ...

work page 2023
[17]

short" text types (Yelp and Arxiv), and two

ArobustlyoptimizedBERTpre-trainingap- proach with post-training. InProceedings of the 20th Chinese National Conference on Compu- tational Linguistics, pages 1218–1227, Huhhot, China. Chinese Information Processing Society of China. A. Metrics We here define the metrics used for evaluation. We use thenumpy and scikit-learn packages to calculate these metri...

work page 2024

[1] [1]

Spotlights and Blindspots: Evaluating Machine-Generated Text Detection

Introduction Recent years have witnessed remarkable advance- ments in generative systems capable of produc- ing realistic video, audio, and text. While these technologies offer significant benefits, they also introduce serious challenges in verifying the ori- gin of content. Distinguishing between machine- generatedandhuman-writtentext,inparticular,has be...

work page internal anchor Pith review Pith/arXiv arXiv 2025

[2] [2]

Ta- ble 1 summarizes the selected model variants, de- tailingtheiroriginalevaluationdata, metrics, andre- ported performance

Systems We assemble a diverse collection of contemporary machine-generated text detection systems, includ- ing zero-shot and trained public models, pretrained transformers, and feature-based approaches. Ta- ble 1 summarizes the selected model variants, de- tailingtheiroriginalevaluationdata, metrics, andre- ported performance. Our goal is to explore impac...

work page 2025

[3] [3]

Data To ensure standardized, robust evaluations, we establish a unified dataset comprising seven test sets derived from four benchmark datasets: MAGE (aka Deepfake) (Li et al., 2024):This datasetconsistsofa447khumanandAI-generated text samples covering diverse models and method- ologies. We extract three test sets: (1) a class- balanced sample of 10k from...

work page 2024

[4] [4]

All positive

Experiments We evaluate each system on each of the eight datasets described above. For comparison include a trivial baseline system (All positive) that assigns every sample a score of 1 (machine-generated).3 For our initial evaluation, we use F1 score (thresh- old=0.5)andAUROC;wediscussmoreonmetrics in Section 5. Results are shown in Figure 1. We start by...

work page 2025

[5] [5]

Threshold by EER

Metrics Our dataset analysis reveals divergences between F1 and AUROC metrics: while Binoculars, BiS- cope, and Fast-DetectGPT achieve strong AUROC scores despite comparatively low F1 scores, De- Threshold 0.5 Threshold by EER Threshold invariant Model Precision Recall F1 Accuracy AvgRec Precision Recall F1 Accuracy AvgRec AUROC TPR@FPR 1% TPR@FPR .01% Va...

work page

[6] [6]

(2024), who explicitly advocate for AUROC’s threshold- agnostic benefits, and Hans et al

Analysis To investigate performance differences, we ex- plore four key textual attributes commonly used for machine-generated text detection: length (in words), punctuation, repetition, and perplexity under the facebook/opt-1.3b (Zhang et al., 2022),definedinAppendixD.Weploteachmodels’ error rates against each of these attributes (Figure 5With the notable...

work page 2022

[7] [7]

Model behavior patterns in different ways for each attribute

with the goal of identifying trends in these at- tributes that may contribute to disparity in model performance. Model behavior patterns in different ways for each attribute. For word count, models exhibit highly variable performance on low-word count texts, but typically improve as the text length in- creases. However, certain of the DeTeCtive vari- ants...

work page

[8] [8]

Model performance varies sub- stantially across different evaluation frameworks, with each system under specific metric and data conditions

Conclusions In this work we demonstrate that the choice of datasets and metrics critically influences our as- sessment of machine-generated text detection sys- tem capabilities. Model performance varies sub- stantially across different evaluation frameworks, with each system under specific metric and data conditions. This underscores the necessity of cont...

work page 2024

[9] [9]

Ethical Considerations The primary ethical consideration surrounding this work is the ethical application of machine- generated text detection models. We aim to avoid making claims about the general usage of these models, and whether it is appropriate, but under- stand there are ethical implications for employing automated systems for decision making that...

work page

[10] [10]

Limitations While our evaluation spans a diverse range of mod- els, it is not exhaustive—many systems and bench- marks fall outside our analysis. We demonstrate that our core findings (evaluation variance on differ- ent datasets, importance of metrics, and relatively weak performance on human-written texts) hold across a representative sample of prevalent...

work page

[11] [11]

References Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Co- jocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023. The fal- con series of open language models. Tal August, Maarten Sap, Elizabeth C...

work page 2023

[12] [12]

Amrita Bhattacharjee, Raha Moraffah, Joshua Gar- land, and Huan Liu

Longformer: The long-document trans- former. Amrita Bhattacharjee, Raha Moraffah, Joshua Gar- land, and Huan Liu. 2024. Eagle: A domain generalization framework for ai-generated text detection. Steven Bird, Edward Loper, and Ewan Klein. 2009.Natural Language Processing with Python. O’Reilly Media Inc. Scott A. Crossley, Yu Tian, Perpetual Baffour, Alex Fr...

work page 2024

[13] [13]

ZeyanLiu, ZijunYao, FengjunLi, andBoLuo.2024

Openorca: An open dataset of gpt aug- mented flan reasoning traces. ZeyanLiu, ZijunYao, FengjunLi, andBoLuo.2024. On the detectability of chatgpt content: Bench- marking, methodology, and evaluation through the lens of academic writing. InProceedings of the 2024 on ACM SIGSAC Conference on Com- puter and Communications Security, CCS ’24, page 2236–2250, N...

work page 2024

[14] [14]

Does human collaboration enhance the accuracy of identifying llm-generated deepfake texts? Adaku Uchendu, Zeyu Ma, Thai Le, Rui Zhang, and Dongwon Lee. 2021. TURINGBENCH: A benchmarkenvironmentforTuringtestintheage of neural text generation. InFindings of the As- sociation for Computational Linguistics: EMNLP 2021, pages 2001–2016, Punta Cana, Domini- can...

work page 2021

[15] [15]

InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3964–3992, Bangkok, Thailand

M4GT-bench: Evaluation benchmark for black-box machine-generated text detection. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3964–3992, Bangkok, Thailand. Association for Computa- tional Linguistics. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Ant...

work page 2020

[16] [16]

Kangxi Wu, Liang Pang, Huawei Shen, Xueqi Cheng, and Tat-Seng Chua

Detectrl: Benchmarkingllm-generatedtext detection in real-world scenarios. Kangxi Wu, Liang Pang, Huawei Shen, Xueqi Cheng, and Tat-Seng Chua. 2023. LLMDet: A third party large language models generated text detection tool. InFindings of the Association for Computational Linguistics: EMNLP 2023, pages 2113–2133, Singapore. Association for Compu- tational ...

work page 2023

[17] [17]

short" text types (Yelp and Arxiv), and two

ArobustlyoptimizedBERTpre-trainingap- proach with post-training. InProceedings of the 20th Chinese National Conference on Compu- tational Linguistics, pages 1218–1227, Huhhot, China. Chinese Information Processing Society of China. A. Metrics We here define the metrics used for evaluation. We use thenumpy and scikit-learn packages to calculate these metri...

work page 2024