Pith. sign in

REVIEW 3 major objections 6 minor 48 references

DivScore: Zero-Shot Detection of LLM-Generated Text in Specialized Domains

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A zero-shot detector that divides one entropy by another, sharpened by domain distillation, beats all tested baselines on machine-written medical and legal text, and its largest margins come at near-zero false-positive rates.

desk verdict The detector recipe is interesting but the headline gains are inflated by teacher/source overlap, and the theory in Appendix A does not prove what it claims. read the letter →

arxiv 2506.06705 v1 pith:LQTLGH2Q submitted 2025-06-07 cs.CL cs.AI

Zhihui Chen , Kai He , Yucheng Huang , Yunxiao Zhu , Mengling Feng This is my paper
classification cs.CLcs.AI
keywords LLM-generatedtextdetectionzero-shotmedicalandlegaldomainsentropynormalizationdomainknowledgedistillationadversarialrobustnessshiftmachinebenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Existing zero-shot detectors of machine-written text assume the scoring model knows the text's domain well enough that machine text looks statistically unusual; the paper argues that medical and legal writing breaks this assumption, collapsing the score gap. It proposes DivScore, which scores a text as the ratio of the entropy a domain-adapted language model assigns to it over the cross-entropy between that adapted model and a general-purpose model, and sharpens the contrast by distilling domain knowledge from a large teacher model into the scorer without any human labels. On a new benchmark pairing 4,000 professional medical and legal documents with outputs from four current LLMs, DivScore is reported to beat state-of-the-art detectors by 14.4% in AUROC and 64.0% in recall at a 0.1% false-positive threshold, and to keep the advantage under paraphrase and word-substitution attacks. If the claim holds, machine-generated content in high-stakes fields can be flagged with far fewer false alarms than current zero-shot tools allow.

What carries the argument

The load-bearing object is the DivScore ratio $D_{M,M^*}(x) = H_{M^*}(x)/\mathrm{CE}_{M,M^*}(x)$, in which the mean per-token entropy $H_{M^*}$ of a domain-adapted language model plays against the cross-entropy $\mathrm{CE}_{M,M^*}$ between that adapted model and its general-purpose base model; the division normalizes away the common difficulty of the text so that only the model-disagreement signal remains. The ratio is motivated by Theorem 1, which asserts that any likelihood-based zero-shot detector's AUROC increases monotonically with the effective KL divergence $\delta_{\mathrm{KL}} = D_{\mathrm{KL}}(P\parallel Q') - D_{\mathrm{KL}}(Q\parallel Q')$, the amount by which the human distribution $P$ diverges more from the true machine-text source $Q'$ than the detector distribution $Q$ does. The second mechanism is unsupervised domain knowledge distillation: a large teacher model generates domain Q&A content, and the student scoring model is LoRA-finetuned on it, which the paper argues shrinks $D_{\mathrm{KL}}(Q\parallel Q')$, enlarges $\delta_{\mathrm{KL}}$, and thereby restores the score gap that general-domain detectors lose under domain shift.

What would settle it

On the released benchmark, measure the rank correlation between each text's likelihood under the detector model and its mean entropy; if the two orderings disagree for a large share of texts, the monotonicity premise behind Theorem 1 fails and the claimed AUROC-to-$\delta_{\mathrm{KL}}$ link is unsupported. Alternatively, run DivScore on a specialized domain excluded from distillation, such as financial filings: if its AUROC advantage over Binoculars collapses, the reported gains come from distillation coverage rather than from the entropy-ratio scoring itself.

Watch

Extended reading notes

Core claim

DivScore detects LLM-generated text in specialized domains using the score $D_{M,M^*}(x) = H_{M^*}(x)/\mathrm{CE}_{M,M^*}(x)$, where $H_{M^*}(x)$ is the mean per-token entropy of the candidate text under a domain-adapted language model $M^*$ and $\mathrm{CE}_{M,M^*}(x)$ is the per-token cross-entropy between $M^*$ and a general-purpose model $M$. Low scores mark machine text, because adapted models find LLM output predictable (low entropy) while the two models disagree sharply on it (high cross-entropy), whereas human text stays unfamiliar to both. The adapted model is built without labels by having a large teacher model generate domain question-answer knowledge and LoRA-finetuning a small student model on it, a step the authors tie to Theorem 1: AUROC rises monotonically with the effective KL divergence $\delta_{\mathrm{KL}} = D_{\mathrm{KL}}(P\parallel Q') - D_{\mathrm{KL}}(Q\parallel Q')$, and distillation reduces $D_{\mathrm{KL}}(Q\parallel Q')$. Across four medical and legal datasets and four source LLMs, DivScore outperforms eleven baselines, with its clearest advantage at the low false-positive rates that matter in clinical and legal settings, and it degrades less than other detectors under paraphrase and word-substitution attacks.

Load-bearing premise

The theory linking detector accuracy to the divergence gap $\delta_{\mathrm{KL}}$ assumes that human-written and LLM-written texts each show bell-shaped entropy distributions and that ranking texts by likelihood always puts them in the same order as ranking by entropy, so if either fails for real text the stated foundation of the method is not proven.

Editorial extensions

If this is right

  • DivScore raises the true-positive rate at a 0.1% false-positive threshold by 64.0% over the strongest prior zero-shot detector, so machine-written medical and legal text can be caught while still allowing only about one false alarm per thousand human documents.
  • One adapted scorer covers several specialized fields: the combined medical-and-legal distillation performs as well as single-domain variants, indicating cross-domain generalization within the adapted scope.
  • Adapting the detector to a new domain needs no human-labeled examples, only a capable teacher model to generate domain knowledge and a finetuned student, which is what makes the zero-shot claim operational.
  • Under moderate paraphrase and word-substitution attacks, DivScore retains an average 22.8% AUROC advantage over Binoculars, consistent with the claim that distillation anchors the scorer to domain content that attacks must preserve.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's ablations suggest distillation, not the entropy ratio itself, drives most of the gain; a direct test would be to apply the same distillation recipe to Binoculars or to plain entropy scoring and compare the lift, which the paper does not run.
  • Because adaptation must target a domain in advance, DivScore is zero-label but not zero-preparation; the honest stress test is a domain deliberately absent from distillation, such as financial filings, where the authors' own reasoning predicts the theoretical motivation, not the gap itself, is what transfers.
  • The cross-entropy component separates machine from human text far more sharply than entropy alone, so the ratio may mostly be a disagreement measure between a general and a domain-adapted model; if so, cheaper surrogates such as perplexity gaps could recover most of the benefit.
  • With AUROC often above 99.9% on this benchmark, a realistic contamination test — human documents with a single inserted LLM-written paragraph — would show whether the large reported margins survive mixed authorship, which the current paired-text setup does not measure.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes DivScore, a zero-shot detector for LLM-generated text in specialized domains (medicine and law). The method scores a candidate text as the ratio of the mean token entropy under a domain-adapted LLM M* to the cross-entropy between a general LLM M and M*. M* is built by LoRA-finetuning Mistral-7B on 26.5M tokens of teacher-generated domain QA, with DeepSeek-R1 as the teacher. The authors claim a theoretical result (Theorem 1) linking detector AUROC to a difference of KL divergences, and report large empirical gains over Binoculars and other baselines on four datasets and four source LLMs, plus robustness results under paraphrase and word-substitution attacks. The paper also releases a benchmark and code.

Significance. If the empirical results hold under a leakage-controlled evaluation, the contribution is valuable: domain-adapted, label-free detection in specialized domains is practically important, and the reported gains over Binoculars are large and consistent across most cells. The release of a medical/legal detection benchmark with distillation resources is a useful community asset. The ablation studies (detector architecture, scoring components, distillation versus human-text fine-tuning) are thorough and give credit to the empirical design. However, the theoretical foundation in Theorem 1 is not sound as written, and the 'zero-shot' framing is weakened by the use of 26.5M teacher-generated tokens for fine-tuning plus the overlap between the teacher model and one of the evaluated source LLMs.

major comments (3)
  1. [Appendix A, Eqs. (11)-(13) and (19)] The proof of Theorem 1 is not valid. Eq. (11) defines H_Q(x) as a sum of p_Q(x_i|x_{<i}) log p_Q(x_i|x_{<i}), which is the negative log-likelihood of the observed text under Q, not the entropy of the predictive distribution (which would average over all possible next tokens). The claimed means in Eq. (13) are not the expectations of this quantity: for x sampled from Q', E[H_Q(X)] is a cross-entropy of the form H(Q') + D_KL(Q'||Q), not H(Q) + D_KL(Q||Q'), and similarly for x sampled from P. Consequently the expression δKL = D_KL(P||Q') - D_KL(Q||Q') in Eq. (16) does not follow. In addition, Eq. (19) asserts that likelihood p_Q(x) and entropy H_Q(x) rank every pair of texts identically; this monotonicity does not hold for arbitrary sequences, so the ROC-equivalence argument in Eqs. (20)-(24) collapses. Since Section 3.1 presents Theorem 1 as the theoretical motivation for DivScore, the central theoretical claim is unsupported as written and should be re-derived, or explicitly reframed as a heuristic.
  2. [Section 3.3 and Table 1] The headline 'zero-shot' claim is contaminated by teacher/source overlap. The detector M* is trained on 26.5M tokens of DeepSeek-R1-generated domain QA (Section 3.3, Appendix B.3, Table 12), while Table 1 evaluates detection of texts produced by DeepSeek-R1 and DeepSeek-V3. For the DeepSeek-R1 columns, the low-entropy/high-cross-entropy signal may reflect the detector having been adapted to the generating model's output style rather than domain specialization. The reported 14.4% AUROC and 64.0% TPR gains are averages over all 16 cells, including these overlap columns. Please recompute the averages excluding DeepSeek-R1 (and ideally DeepSeek-V3) cells, report per-source results separately, and clarify whether 'zero-shot' means 'no human labels' (in which case the 26.5M-token teacher fine-tuning should be stated prominently) or 'no training on the target domain' (in which case the method is not zero-shot by that definition).
  3. [Table 1, OALC/GPT-4o row] The Binoculars AUROC for OALC with GPT-4o is 38.47, far below its performance on most other cells (e.g., 98.34 on MIMIC/GPT-4o). This single outlier cell contributes disproportionately to the average gain: DivScore improves it by 60.41 AUROC points. Please verify this value by checking the Binoculars configuration and OALC preprocessing, and report the average gains with and without this cell, together with confidence intervals or multiple seeds. Without such a leakage/outlier-controlled analysis, the abstract's 'consistently outperforms' claim is stronger than the evidence presented.
minor comments (6)
  1. [Section 3.2, Eqs. (4)-(5)] Eq. (4) is not the entropy of the predictive distribution: -1/L Σ p_{M*}(x_i|x_{<i}) log p_{M*}(x_i|x_{<i}) is the average negative log-likelihood of the observed tokens (a cross-entropy), not the entropy. Please rename the quantity (e.g., 'mean token negative log-likelihood') or correct the formula.
  2. [Section 4.4, Table 3] The claimed '36.2%' increase in cross-entropy centroid distance does not match the reported means: the Avg. row gives 1.79 vs. 2.23 without distillation (distance 0.44) and 4.33 vs. 3.64 with distillation (distance 0.69), which is a 56.8% increase. Please recompute and correct this number.
  3. [Appendix A, Eq. (17)] The variable φ(z) is called the 'Cumulative Density Function' but is used as the standard normal density; ϕ is the PDF, while the CDF is Φ. Please fix the notation.
  4. [Section 3.1, Eq. (2)] The notation ∝+ is non-standard. Define it explicitly as 'AUROC is a monotonically increasing function of δKL' and state the assumptions under which the monotonicity is claimed to hold.
  5. [Table 3 and Figure 3] The 'Avg.' row averages means and standard deviations across four datasets of different lengths and genres; please clarify whether these are pooled statistics or simple averages, since the latter can be misleading when dataset sizes or variances differ.
  6. [Related Work] The related-work section does not discuss recent zero-shot detectors such as GECScore, although it is cited in the references. Add a sentence positioning DivScore relative to that line of work.

Circularity Check

2 steps flagged · score 4.0 of 10

Theorem 1's AUROC–KL link is assumed by construction, and teacher/source overlap inflates the reported gains; the central empirical claim still has independent support.

  1. self definitional [Appendix A, Proof of Theorem 1, Eqs. (12)–(17)]
    "Assume the text variable X's LLM entropy values follow normal distributions: ... with means derived from cross-entropy among detector distribution Q, human distribution P and LLM text generation distribution Q′: μ_Q = H(Q)+D_KL(Q∥Q′), μ_P = H(P)+D_KL(P∥Q′) ... δKL = μ_P − μ_Q = D_KL(P∥Q′)−D_KL(Q∥Q′)"

    The proof defines the means of the assumed normal entropy distributions so that their difference is exactly δKL, then derives AUROC = Φ(δKL / sqrt(σ_P^2 + σ_Q'^2)). Hence the theorem's conclusion AUROC ∝ δKL is true by construction under the normality assumption: it restates that AUROC increases with the mean-score gap after naming that gap δKL. No independent link between AUROC and the KL divergences is established; the 'means derived from cross-entropy' also do not equal expected entropy, since E_{Q'}[H_Q(X)] is not H(Q)+D_KL(Q||Q'). The theoretical analysis therefore assumes the very relationship it claims to prove.

  2. fitted input called prediction [Section 3.3 and Section 4.1, Tables 1 and 12]
    "We select DeepSeek-R1 (DeepSeek-AI, 2025) as the teacher LLM ... The resulting outputs are used to finetune the detector's general LLM with Low Rank Adaptation (LoRA). ... The source LLMs for LLM-generated text are chosen from two model families: GPT-4o (OpenAI, 2024), O3-mini (OpenAI, 2025), DeepSeek-V3 (DeepSeek-AI, 2024) and DeepSeek-R1 (DeepSeek-AI, 2025)."

    M* is LoRA-finetuned on 26.5M tokens of DeepSeek-R1-generated QA (Table 12), and the benchmark includes DeepSeek-R1 (and DeepSeek-V3) as text sources. For those columns the detector is adapted to the teacher's output distribution and then 'predicts' whether text comes from that same family, so part of the reported 14.4% AUROC and 64.0% TPR advantage is teacher-identity matching rather than an independent zero-shot prediction. The GPT-4o and O3-mini columns, and the general-domain HC3 experiments, provide independent evidence, so the circularity is partial rather than total.

full rationale

The DivScore scoring formula itself is not fitted to detection labels: it is a fixed ratio of mean token entropy under M* to cross-entropy between M and M*, and the evaluation against external corpora (MIMIC, PubMedQA, OALC, LawStack, HC3) is a genuine empirical comparison. There is no load-bearing self-citation chain: the cited seed-knowledge sources are from other groups, and the method is benchmarked against standard baselines. However, the paper's stated theoretical foundation is circular. Theorem 1 is advertised as showing that zero-shot detector failure is 'fundamentally linked to the KL divergence,' but its proof assigns normal-distribution means whose difference is defined to equal δKL, making AUROC monotone in δKL by construction. Separately, the claimed headline gains are partially self-referential because the detection model is distilled from DeepSeek-R1 outputs and then evaluated on DeepSeek-R1/V3-generated test text; the ablation on GPT-4o and O3-mini sources and the general-domain results show the approach has independent content. Overall, the empirical core is self-contained, but the theoretical derivation reduces to its own definitional assumptions and the headline average is inflated by teacher/source overlap, warranting a moderate circularity score of 4.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The detector score itself has no fitted scalar parameters; the tuning burden is concentrated in the choice of domain-adapted model M* and its hyperparameters, plus the evaluation benchmark construction. The theorem's assumptions are the largest unstated load.

free parameters (4)
  • Domain distillation LoRA hyperparameters = rank=64, alpha=128, lr=2e-4, batch=12, seq_len=2048
    Chosen by design and ablation; these hyperparameters control the adapted model M* that the detector depends on.
  • Finetuning epoch count = 8
    Selected because all variants reach peak AUROC at epoch 8 in Figure 5, which tunes the predictor on the evaluation data.
  • Detector base LLM architecture = Mistral-7B-v0.2
    Chosen over Falcon-7B, Qwen2.5-8B, and Llama3-8B based on best averaged AUROC in Table 4, a model selection on the same task.
  • Adversarial attack parameter settings = DIPPER lexical diversity 20%, content reordering 20%; BERT word substitution top 20% frequent words in 50-token windows
    Hand-set attack intensities; robustness claims are conditional on these moderate strengths.
assumptions (5)
  • ad hoc to paper Entropy scores of human and LLM texts are normally distributed and their means are H(Q)+D_KL(Q||Q') and H(P)+D_KL(P||Q')
    Used in Appendix A Eqs. 12-13 to derive Theorem 1; the mean formula describes expected negative log-likelihood, not expected entropy, and normality is not justified.
  • ad hoc to paper Likelihood p_Q(x) and entropy H_Q(x) are monotonically related, so rankings by entropy and by likelihood coincide
    Used in Eq. 19 of Appendix A to extend the result from entropy detectors to any monotonic score function; this relationship is false in general.
  • domain assumption Domain knowledge distillation from DeepSeek-R1 makes M* better aligned with distribution of LLM-generated text, increasing δKL
    Core motivation of Section 3.3; supported only by the paper's own ablation, not by an external benchmark.
  • domain assumption LLM continuation generated from the first third of a human document is a representative sample of real LLM-generated text in that domain
    Benchmark construction in Appendix B.1; if real LLM usage differs, such as full generation or editing, measured accuracy may not transfer.
  • domain assumption The supervised RoBERTa detectors are evaluated out-of-domain without adaptation, providing fair baselines
    Main experiment treats them as zero-shot baselines; this is standard but favorable to DivScore.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DivScore: Zero-Shot Detection of LLM-Generated Text in Specialized Domains." pith.science (2026). https://pith.science/paper/LQTLGH2Q

@misc{pith2026250606705,
  author       = {Pith},
  title        = {Pith review of: DivScore: Zero-Shot Detection of LLM-Generated Text in Specialized Domains},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LQTLGH2Q}},
  note         = {Machine review of arXiv:2506.06705}
}
read the original abstract

Detecting LLM-generated text in specialized and high-stakes domains like medicine and law is crucial for combating misinformation and ensuring authenticity. However, current zero-shot detectors, while effective on general text, often fail when applied to specialized content due to domain shift. We provide a theoretical analysis showing this failure is fundamentally linked to the KL divergence between human, detector, and source text distributions. To address this, we propose DivScore, a zero-shot detection framework using normalized entropy-based scoring and domain knowledge distillation to robustly identify LLM-generated text in specialized domains. We also release a domain-specific benchmark for LLM-generated text detection in the medical and legal domains. Experiments on our benchmark show that DivScore consistently outperforms state-of-the-art detectors, with 14.4% higher AUROC and 64.0% higher recall (0.1% false positive rate threshold). In adversarial settings, DivScore demonstrates superior robustness than other baselines, achieving on average 22.8% advantage in AUROC and 29.5% in recall. Code and data are publicly available.

Figures

Figures reproduced from arXiv: 2506.06705 by the authors.

Figure 1
Figure 1. Fast-DetectGPT (Bao et al., 2023) and Binoc￾ulars (Hans et al., 2024) detector scores of LLM/Human texts in specialized (legal, medical) and general domain. In specialized domain, both detectors failed in creating scoring gap necessary for detecting LLM-generated text data they were trained on, such supervised detec￾tors show clear shortcomings when faced with new domains or more advanced language models. For instan… view at source ↗
Figure 2
Figure 2. The overall framework of DivScore a specialised domain, setting the theoretical stage for DivScore. Guided by this analysis, we derive our core detector and then introduce a domain￾knowledge distillation step that narrows the gap between the detector and specialised language, markedly strengthening robustness and accuracy. The framework of DivScore is shown in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Text statistical signatures in DivScore for [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Comparison of detector AUROC scores across [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Detector performance (AUROC %) of LLM distillation (proposed) and human text finetuning across finetuning epochs. AUROC scores are averaged across all evaluation datasets. LLM domain knowledge distillation significantly enhances detection performance, with respective A…
Figure 6
Figure 6. Figure 6: ROC Curves of Baseline Detectors and DivScore in the Main Experiment [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 17 canonical work pages

  1. [1]

    Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023. https://arxiv.org/abs/2311.16867 The falcon series of open language models . Preprint, arXiv:2...

    arXiv 2023

  2. [2]

    Anton Bakhtin, Sam Gross, Myle Ott, Yuntian Deng, Marc'Aurelio Ranzato, and Arthur Szlam. 2019. Real or fake? learning to discriminate machine from human generated text. arXiv preprint arXiv:1906.03351

    arXiv 2019

  3. [3]

    Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. 2023. Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature. In The Twelfth International Conference on Learning Representations

    work page 2023

  4. [4]

    Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. 2021. https://doi.org/10.5281/zenodo.5297715 GPT-Neo : Large scale autoregressive language modeling with mesh-tensorflow . Version 1.0

    doi:10.5281/zenodo.5297715 2021

  5. [5]

    Umar Butler. 2023. Open australian legal qa. https://huggingface.co/datasets/isaacus/open-australian-legal-qa. Version 2.0.0, Hugging Face, DOI: 10.57967/hf/1479. Accessed: 2025-05-09

    doi:10.57967/hf/1479 2023

  6. [6]

    Umar Butler. 2025. Open australian legal corpus. https://huggingface.co/datasets/isaacus/open-australian-legal-corpus. Accessed: 2025-05-09

    work page 2025

  7. [7]

    Ilias Chalkidis, Nicolas Garneau, Catalina Goanta, Daniel Katz, and Anders S gaard. 2023. https://aclanthology.org/2023.acl-long.865 L e XF iles and L egal LAMA : Facilitating E nglish multinational legal language model development . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1...

    work page 2023

  8. [8]

    Andong Chen, Feng Yao, Xinyan Zhao, Yating Zhang, Changlong Sun, Yun Liu, and Weixing Shen. 2023. https://doi.org/10.1145/3594536.3595159 Equals: A real-world dataset for legal question answering via reading chinese laws . In Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, ICAIL '23, page 71–80, New York, NY, USA...

    arXiv 2023

Show all 48 references
  1. [9]

    Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. 2024. https://arxiv.org/abs/2412.18925 Huatuogpt-o1, towards medical complex reasoning with llms . Preprint, arXiv:2412.18925

    2024 arXiv

  2. [10]

    DeepSeek-AI. 2024. DeepSeek-V3 Technical Report . https://arxiv.org/abs/2412.19437. Accessed: 2025-05-09

    2024 arXiv

  3. [11]

    DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning . https://arxiv.org/abs/2501.12948. Accessed: 2025-05-09

    2025 arXiv

  4. [12]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://arxiv.org/abs/1810.04805 Bert: Pre-training of deep bidirectional transformers for language understanding . Preprint, arXiv:1810.04805

    2019 arXiv

  5. [13]

    Liam Dugan, Alyssa Hwang, Filip Trhl \'i k, Andrew Zhu, Josh Magnus Ludan, Hainiu Xu, Daphne Ippolito, and Chris Callison-Burch. 2024. https://doi.org/10.18653/v1/2024.acl-long.674 RAID : A shared benchmark for robust evaluation of machine-generated text detectors . In Proceed...

    2024 doi

  6. [14]

    Angela Fan, Mike Lewis, and Yann Dauphin. 2018. https://doi.org/10.18653/v1/P18-1082 Hierarchical neural story generation . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889--898, Melbourne, Australia....

    2018 doi

  7. [15]

    Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. https://doi.org/10.18653/v1/P19-3019 GLTR : Statistical detection and visualization of generated text . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrat...

    2019 doi

  8. [16]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  9. [17]

    Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. arXiv preprint arXiv:2301.07597

    2023 arXiv

  10. [18]

    Tianyu Han, Lisa C Adams, Jens-Michalis Papaioannou, Paul Grundmann, Tom Oberhauser, Alexander L \"o ser, Daniel Truhn, and Keno K Bressem. 2023. Medalpaca--an open-source collection of medical conversational ai models and training data. arXiv preprint arXiv:2304.08247

    2023 arXiv

  11. [19]

    Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. 2024. Spotting llms with binoculars: Zero-shot detection of machine-generated text. In International Conference on Machine Learning, pages 1...

    2024

  12. [20]

    Krass, Lucia Zheng, Neel Guha, Christopher D

    Peter Henderson, Mark S. Krass, Lucia Zheng, Neel Guha, Christopher D. Manning, Dan Jurafsky, and Daniel E. Ho. 2022. https://arxiv.org/abs/2207.00220 Pile of law: Learning responsible data filtering from the law and a 256gb open-source legal dataset . Preprint, arXiv:2207.00220

    2022 arXiv

  13. [21]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  14. [22]

    Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. https://aclanthology.org/D19-1259/ Pubmedqa: A dataset for biomedical research question answering . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9...

    2019

  15. [23]

    Alistair EW Johnson, Lucas Bulgarelli, Lu Shen, Alvin Gayles, Ayad Shammout, Steven Horng, Tom J Pollard, Sicheng Hao, Benjamin Moody, Brian Gow, et al. 2023. Mimic-iv, a freely accessible electronic health record dataset. Scientific data, 10(1):1

    2023

  16. [24]

    Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. 2023. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. Advances in Neural Information Processing Systems, 36:27469--27500

    2023

  17. [25]

    Thomas Lavergne, Tanguy Urvoy, and Fran c ois Yvon. 2008. Detecting fake content with relative entropy scoring. In Proceedings of the 2008 International Conference on Uncovering Plagiarism, Authorship and Social Software Misuse - Volume 377, PAN'08, page 27–31, Aachen, DEU. CE...

    2008

  18. [26]

    Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang. 2023. Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge. Cureus, 15(6)

    2023

  19. [27]

    Gongbo Liang, Jesus Guerrero, and Izzat Alsmadi. 2023. Mutation-based adversarial attacks on neural text detectors. arXiv preprint arXiv:2302.05794

    2023 arXiv

  20. [28]

    Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In International Conference on Machine Learning, pages 24950--24962. PMLR

    2023

  21. [29]

    Yasmin Moslem. 2025. Law-StackExchange (Revision 6a14705) . https://huggingface.co/datasets/ymoslem/Law-StackExchange. Accessed: 2025-05-09

    2025

  22. [30]

    Cohen, and Mirella Lapata

    Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. https://doi.org/10.18653/v1/D18-1206 Don`t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization . In Proceedings of the 2018 Conference on Empirical Methods in Natura...

    2018 doi

  23. [31]

    OpenAI. 2024. GPT-4o System Card . https://arxiv.org/abs/2410.21276. Accessed: 2025-05-09

    2024 arXiv

  24. [32]

    OpenAI. 2025. OpenAI o3 and o4-mini System Card . https://cdn.openai.com/pdf/2221c875-02dc-4789-800b-e7758f3722c1/o3-and-o4-mini-system-card.pdf. Accessed: 2025-05-09

    2025

  25. [33]

    Xinlin Peng, Ying Zhou, Ben He, Le Sun, and Yingfei Sun. 2023. Hidding the ghostwriters: An adversarial evaluation of ai-generated student essay detection. In The 2023 Conference on Empirical Methods in Natural Language Processing

    2023

  26. [34]

    Qwen. 2025. Qwen2.5 technical report. https://arxiv.org/abs/2412.15115. Accessed: 2025-05-09

    2025 arXiv

  27. [35]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(1)

    2020

  28. [36]

    Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. 2023. Can ai-generated text be reliably detected? arXiv preprint arXiv:2303.11156

    2023 arXiv

  29. [37]

    Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, et al. 2019. Release strategies and the social impacts of language models. arXiv preprint arXiv:1908.09203

    2019 arXiv

  30. [38]

    Jinyan Su, Terry Zhuo, Di Wang, and Preslav Nakov. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.827 D etect LLM : Leveraging log rank information for zero-shot detection of machine-generated text . In Findings of the Association for Computational Linguistics: EMNLP 20...

    2023 doi

  31. [39]

    Mickael Tordjman, Zelong Liu, Murat Yuce, Valentin Fauveau, Yunhao Mei, Jerome Hadjadj, Ian Bolger, Haidara Almansour, Carolyn Horst, Ashwin Singh Parihar, et al. 2025. Comparative benchmarking of the deepseek large language model on medical tasks and clinical reasoning. Natur...

    2025

  32. [40]

    Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B : A 6 billion parameter autoregressive language model. https://github.com/kingoflolz/mesh-transformer-jax. Accessed: 2025-04-20

    2021

  33. [41]

    Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Lidia Sam Chao, and Derek Fai Wong. 2025 a . https://doi.org/10.1162/coli_a_00549 A survey on llm-generated text detection: Necessity, methods, and future directions . Computational Linguistics, 51(1):275--338

    2025 doi

  34. [42]

    Wong, Shu Yang, Xuebo Liu, Lidia S

    Junchao Wu, Runzhe Zhan, Derek F. Wong, Shu Yang, Xuebo Liu, Lidia S. Chao, and Min Zhang. 2025 b . https://aclanthology.org/2025.coling-main.684/ Who wrote this? the key to zero-shot LLM -generated text detection is GECS core . In Proceedings of the 31st International Confere...

    2025

  35. [43]

    Guangzhi Xiong, Qiao Jin, Zhiyong Lu, and Aidong Zhang. 2024. https://aclanthology.org/2024.findings-acl.372/ Benchmarking retrieval-augmented generation for medicine . In Findings of the Association for Computational Linguistics ACL 2024, pages 6233--6251

    2024

  36. [44]

    Yashaswat. 2024. Indian legal text - abstractive summarization dataset. https://huggingface.co/datasets/Yashaswat/Indian-Legal-Text-ABS. Accessed: 2025-05-17

    2024

  37. [45]

    Xiao Yu, Kejiang Chen, Qi Yang, Weiming Zhang, and Nenghai Yu. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.885 Text fluoroscopy: Detecting LLM -generated text through intrinsic features . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Proc...

    2024 doi

  38. [46]

    Yaoyao Yu, Leilei Gan, Yinghao Hu, Bin Wei, Kun Kuang, and Fei Wu. 2025. Evaluating test-time scaling llms for legal reasoning: Openai o1, deepseek-r1, and beyond. arXiv preprint arXiv:2503.16040

    2025

  39. [47]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  40. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.