Pith. sign in

REVIEW 4 minor 59 references

A statistical power formula turns LLM watermark hyperparameters into a one-dimensional optimization problem that finds Pareto-optimal detectability-distortion tradeoffs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 03:36 UTC pith:LHYYC3OP

load-bearing objection Clean reduction of KGW (γ,δ) selection to a one-dimensional power-vs-KL optimizer that actually lands on the empirical Pareto front; Dirichlet prior is a modeling choice, not a load-bearing flaw.

arxiv 2607.05694 v1 pith:LHYYC3OP submitted 2026-07-06 stat.ML cs.LG

Beyond Heuristic Tuning: Power-Calibrated LLM Watermarking

Xiaopu Wang , Zelin He , Chengyuan Liu , Runze Li This is my paper
classification stat.ML cs.LG
keywords LLM watermarkinglogit-based watermarkdetection powerKL distortionhyperparameter calibrationKGWPareto frontier
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Logit-based watermarking marks LLM text by slightly biasing next-token logits toward a secret green list, then testing whether green tokens appear more often than chance. Stronger bias raises detection power but also moves the model distribution farther from the original, harming semantic fidelity. Existing analyses leave the two governing knobs (green-list fraction and logit bias) to be tuned by hand. This paper derives closed-form expressions that map those knobs directly onto asymptotic detection power and expected per-token KL distortion, converting design into a guided optimization under an explicit power or distortion budget. The resulting procedure is solved once before generation and is shown, across several open models and domains, to recover operating points that dominate heuristic baselines on the power-versus-distortion frontier.

Core claim

Under a non-informative prior on next-token probabilities and mild dependence conditions, the green-token rate under watermarking admits an explicit formula γ′(γ, δ). Combined with a normal approximation for the sequence-level green-token count, this yields a closed-form power function π*(γ, δ) and a matching closed-form KL distortion D_KL(γ, δ). Because KL is strictly monotone in the bias for fixed γ, any distortion budget reduces the two-parameter search to a one-dimensional numerical optimization whose solutions lie on the empirical Pareto frontier.

What carries the argument

The power-calibrated mapping (Theorem 3.5 + Lemma 3.9 + the normal power formula): γ′ is expressed via a hypergeometric function (or its large-vocabulary limit), D_KL becomes δγ′ − log(1 + γ(e^δ − 1)), and detection power is the corresponding normal tail probability; monotonicity of KL then collapses design to max_γ π*(γ, δ(γ, K0)).

Load-bearing premise

The closed-form green-token probability rests on treating every next-token probability vector as an independent draw from the uniform Dirichlet distribution; if real models produce far more peaked or context-dependent distributions, both the predicted power and the optimal settings can move.

What would settle it

Generate short sequences under the theoretically recommended (γ, δ) pairs and under dense grid-search alternatives on the same models and prompts; if the theory-selected points systematically fall inside the convex hull of the grid-search power-versus-KL cloud rather than on its outer frontier, the claimed optimality fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 4 minor

Summary. The paper develops a power-calibrated statistical framework for logit-based KGW watermarking. Under a keyed green-list model and a non-informative Dirichlet prior on next-token probabilities, it derives a closed-form green-token probability γ' (Theorem 3.5, hypergeometric / large-vocabulary plug-in), an asymptotic normal power formula π*(γ,δ) under α-mixing dependence (Eq. 1, Theorem 3.8), and a closed-form token-wise KL distortion D_KL(γ,δ) that is monotone in δ (Lemma 3.9). These relations reduce two-parameter hyperparameter search to a one-dimensional constrained optimization (maximize power under a KL budget, or the dual). Experiments on GPT-2, OPT-125M, Pythia-160M (and Gemma-2 9B) across C4, LFQA, and Wikipedia show that the calibrated parameters lie on the empirical Pareto frontier against DP, OPT-style, and grid-search baselines under both KL and surface quality metrics (BLEU/ROUGE/BERTScore), with high agreement between predicted and observed green rates and KL (R² ≥ 0.98).

Significance. If the results hold, the paper replaces heuristic (γ,δ) tuning with a statistically grounded, pre-generation calibration procedure that has explicit power and distortion guarantees. The closed-form power and KL expressions, the reduction to a one-dimensional optimization whose argmax is independent of the long-run variance constant c, the Q–Q normality checks, the high R² validation of γ' and D_KL, and the consistent Pareto dominance under multiple quality metrics are concrete, falsifiable contributions that improve the practical deployability of KGW-style watermarks. The framework is modular with respect to the detector aggregation rule and includes a signal-removal robustness discussion, which strengthens its applied value.

minor comments (4)
  1. Assumption 3.4 (uniform Dirichlet NTP prior) is the modeling step that yields the closed-form γ' and plug-in KL. The large-vocabulary concentration argument and the R² ≥ 0.98 empirical checks already support it; a short additional remark on how a more concentrated or mixture prior would shift the numerical optimum (without changing the optimization procedure) would further clarify the scope.
  2. Appendix C estimates the variance inflation c and notes model/dataset dependence; a brief statement in the main text that c cancels from the argmax (already in the remark after Eq. 1) but can be estimated for absolute power prediction would help practitioners.
  3. Figures 5–8 and the Pareto curve fits use a logistic form for visualization only; stating this explicitly in the captions (as done in the text) would avoid any impression that the functional form is part of the method.
  4. A few typographic issues remain (e.g., garbled figure labels in the source, occasional missing spaces around math). A light copy-edit pass would improve readability.

Circularity Check

0 steps flagged

No significant circularity: closed-form power and KL follow from stated assumptions and are validated out-of-sample, not fitted or self-defined.

full rationale

The derivation chain is self-contained. Lemma 3.2 follows from Assumption 3.1 (keyed green lists independent of tokens). Lemma 3.3 is an algebraic identity from the softmax logit bias. Theorem 3.5 obtains the closed-form γ' (hypergeometric, then large-|V| plug-in) solely from the non-informative Dirichlet prior (Assumption 3.4) and Beta aggregation; it is not fitted to any language-model outputs. Theorem 3.8 supplies the H1 CLT under geometric information decay and non-degenerate long-run variance (Assumptions 3.6–3.7). The normal-approximation power (Eq. 1) and the plug-in KL (Lemma 3.9) are then algebraic consequences of those expressions. The remark after Eq. 1 explicitly notes that the long-run constant c cancels from the argmax, so setting c=1 for calibration does not force the claimed optima. Parameter selection therefore reduces to a one-dimensional numerical optimization under an explicit distortion budget; the subsequent multi-model, multi-dataset R^{2}≥0.98 checks (Figure 4, Table 5) and Pareto comparisons against DP, OPT-style and grid search are independent empirical tests, not tautologies. No load-bearing uniqueness theorem, self-citation chain, or fitted-then-predicted quantity appears. The modeling assumptions are transparent and externally falsifiable; the paper does not redefine its targets in terms of its own outputs.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The central claim rests on four modeling axioms that convert an otherwise intractable next-token process into closed-form power and KL expressions, plus one free variance-inflation constant that is set rather than estimated for the optimization. No new physical entities are postulated; the framework re-uses the existing KGW green-list mechanism.

free parameters (2)
  • long-run variance inflation c = 1 (default)
    Appears in the asymptotic variance under the alternative; set to 1 for the closed-form calibration used in experiments, with an estimation procedure given only in the appendix.
  • search initializer γ* = n/(n+z_{1-α}^{2})
    Upper bound γ ≤ n/(n + z_{1-α}^{2}) used to start the numerical search for the higher root γ_h; derived from the power > 0.5 requirement under a relaxed γ' = 1.
axioms (4)
  • domain assumption Green lists {G_i} are i.i.d. uniform and independent of the generated token sequence (Assumption 3.1).
    Standard modeling of keyed hash-based green lists; used to obtain i.i.d. Bernoulli(γ) indicators under H0.
  • ad hoc to paper Next-token probability vectors are i.i.d. uniform Dirichlet Dir(1,…,1) (Assumption 3.4).
    Non-informative surrogate that yields the hypergeometric expression for γ' and the plug-in KL; not claimed to be the true NTP law.
  • domain assumption Mutual information between past and future green indicators decays geometrically (Assumption 3.6) and long-run variance is positive (Assumption 3.7).
    Enables the α-mixing CLT for the sequence-level statistic under the alternative.
  • domain assumption Large-vocabulary plug-in P(x ∈ G) ≈ γ for the KL formula (Lemma 3.9).
    Simplifies the exact hypergeometric expression to a closed algebraic form used in optimization.

pith-pipeline@v1.1.0-grok45 · 33363 in / 2915 out tokens · 32688 ms · 2026-07-11T03:36:53.969723+00:00 · methodology

0 comments
read the original abstract

Logit-based watermarking is a widely used mechanism for identifying LLM generated content, yet its effectiveness is governed by a fundamental trade-off between detectability and semantic distortion. Existing analyses provide limited guidance for principled hyperparameter selection, leaving practical deployments reliant on heuristic tuning. In this work, we develop a power-calibrated statistical framework that establishes explicit quantitative relationships between watermark hyperparameters, detection power, and distortion. This characterization transforms watermark design into a guided optimization problem. Building on these results, we derive practical parameter selection procedures that achieve optimal tradeoffs under constraints. Extensive experiments across multiple language models and datasets validate the theory and demonstrate that the proposed framework consistently identifies Pareto-optimal points.

Figures

Figures reproduced from arXiv: 2607.05694 by Chengyuan Liu, Runze Li, Xiaopu Wang, Zelin He.

Figure 1
Figure 1. Figure 1: Information flow between tokens, model state, and green lists. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Figure (a) illustrates the power curves of the two solutions [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Q–Q plots of the standardized green-token count statistic under both hypotheses. The empirical [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison between theoretical and empirical green-token rates. The fitted model is [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Statistical power (measured by TPR) versus semantic metrics, including BLEU, ROUGE, and [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Statistical power (measured by TPR) versus semantic distortion (measured by KL divergence). [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Statistical power (measured by TPR) versus semantic distortion (measured by KL divergence) [PITH_FULL_IMAGE:figures/full_fig_p031_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Complete statistical power (TPR) versus semantic quality results under BLEU, ROUGE, and [PITH_FULL_IMAGE:figures/full_fig_p032_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Smoothed inflation constant distribution, categorized by dataset [PITH_FULL_IMAGE:figures/full_fig_p034_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Statistical power (measured by TPR) versus semantic distortion (measured by KL divergence). [PITH_FULL_IMAGE:figures/full_fig_p035_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Statistical power (measured by TPR) versus semantic distortion (measured by KL divergence). [PITH_FULL_IMAGE:figures/full_fig_p036_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Statistical power (measured by TPR) versus semantic distortion (measured by KL divergence). [PITH_FULL_IMAGE:figures/full_fig_p037_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

59 extracted references · 59 canonical work pages · 20 internal anchors

  1. [1]

    Three Bricks to Consolidate Watermarks for Large Language Models , year=

    Fernandez, Pierre and Chaffin, Antoine and Tit, Karim and Chappelier, Vivien and Furon, Teddy , booktitle=. Three Bricks to Consolidate Watermarks for Large Language Models , year=

    work page

  2. [3]

    2024 , eprint=

    Towards Better Statistical Understanding of Watermarking LLMs , author=. 2024 , eprint=

    work page 2024

  3. [4]

    2024 , eprint=

    Beyond the Black Box: A Statistical Model for LLM Reasoning and Inference , author=. 2024 , eprint=

    work page 2024

  4. [5]

    2024 , eprint=

    A Watermark for Large Language Models , author=. 2024 , eprint=

    work page 2024

  5. [6]

    2024 , eprint=

    On the Reliability of Watermarks for Large Language Models , author=. 2024 , eprint=

    work page 2024

  6. [7]

    Proceedings of the 41st International Conference on Machine Learning , pages =

    Optimizing Watermarks for Large Language Models , author =. Proceedings of the 41st International Conference on Machine Learning , pages =. 2024 , editor =

    work page 2024

  7. [8]

    2022 , eprint=

    OPT: Open Pre-trained Transformer Language Models , author=. 2022 , eprint=

    work page 2022

  8. [9]

    2023 , eprint=

    Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling , author=. 2023 , eprint=

    work page 2023

  9. [10]

    OpenAI blog , volume=

    Language models are unsupervised multitask learners , author=. OpenAI blog , volume=

    work page

  10. [11]

    2023 , eprint=

    Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer , author=. 2023 , eprint=

    work page 2023

  11. [12]

    2023 , eprint=

    A Critical Evaluation of Evaluations for Long-form Question Answering , author=. 2023 , eprint=

    work page 2023

  12. [13]

    Wikimedia Downloads

    Wikimedia Foundation. Wikimedia Downloads

    work page

  13. [14]

    and Ng, Andrew Y

    Blei, David M. and Ng, Andrew Y. and Jordan, Michael I. , title =. J. Mach. Learn. Res. , month = mar, pages =. 2003 , issue_date =

    work page 2003

  14. [19]

    ACM Computing Surveys , volume=

    A survey of text watermarking in the era of large language models , author=. ACM Computing Surveys , volume=. 2024 , publisher=

    work page 2024

  15. [20]

    International conference on machine learning , pages=

    Detectgpt: Zero-shot machine-generated text detection using probability curvature , author=. International conference on machine learning , pages=. 2023 , organization=

    work page 2023

  16. [23]

    Unbiased Watermark for Large Language Models

    Unbiased watermark for large language models , author=. arXiv preprint arXiv:2310.10669 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  17. [24]

    ROUGE : A Package for Automatic Evaluation of Summaries

    Lin, Chin-Yew. ROUGE : A Package for Automatic Evaluation of Summaries. Text Summarization Branches Out. 2004

    work page 2004

  18. [26]

    Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages=

    Bleu: a method for automatic evaluation of machine translation , author=. Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages=. 2002 , url=

    work page 2002

  19. [27]

    2020 , eprint=

    BERTScore: Evaluating Text Generation with BERT , author=. 2020 , eprint=

    work page 2020

  20. [28]

    2019 , eprint=

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , author=. 2019 , eprint=

    work page 2019

  21. [29]

    Mixing: Properties and Examples , pages=

    Mixing , author=. Mixing: Properties and Examples , pages=. 1995 , publisher=

    work page 1995

  22. [30]

    A survey and some open questions , author=

    Basic properties of strong mixing conditions. A survey and some open questions , author=. 2005 , url=

    work page 2005

  23. [31]

    2019 , eprint=

    Dirichlet Variational Autoencoder , author=. 2019 , eprint=

    work page 2019

  24. [32]

    2023 , eprint=

    Three Bricks to Consolidate Watermarks for Large Language Models , author=. 2023 , eprint=

    work page 2023

  25. [34]

    International Workshop on Information Hiding , pages=

    Natural language watermarking: Design, analysis, and a proof-of-concept implementation , author=. International Workshop on Information Hiding , pages=. 2001 , organization=. doi:10.1007/3-540-45496-9_14 , url=

    work page doi:10.1007/3-540-45496-9_14 2001

  26. [35]

    Proceedings of the 8th workshop on Multimedia and security , pages=

    The hiding virtues of ambiguity: quantifiably resilient watermarking of natural language text through synonym substitutions , author=. Proceedings of the 8th workshop on Multimedia and security , pages=

    work page

  27. [36]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Who wrote this code? watermarking for code generation , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=. 2024 , doi=

    work page 2024

  28. [37]

    Proceedings of the 8th workshop on Multimedia and security , pages=

    The hiding virtues of ambiguity: quantifiably resilient watermarking of natural language text through synonym substitutions , author=. Proceedings of the 8th workshop on Multimedia and security , pages=. 2006 , doi=

    work page 2006

  29. [39]

    2024 , eprint=

    A Resilient and Accessible Distribution-Preserving Watermark for Large Language Models , author=. 2024 , eprint=

    work page 2024

  30. [40]

    GPT-4 Technical Report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. URL https://arxiv.org/abs/2303.08774

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  31. [41]

    Random oracles are practical: a paradigm for designing efficient protocols

    Mihir Bellare and Phillip Rogaway. Random oracles are practical: a paradigm for designing efficient protocols. In Proceedings of the 1st ACM Conference on Computer and Communications Security, CCS '93, page 62–73, New York, NY, USA, 1993. Association for Computing Machinery. ISBN 0897916298. doi:10.1145/168588.168596. URL https://doi.org/10.1145/168588.168596

    work page doi:10.1145/168588.168596 1993

  32. [42]

    Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

    Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, and Oskar van der Wal. Pythia: A suite for analyzing large language models across training and scaling, 2023. URL https://arxiv.org/abs/2304.01373

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  33. [43]

    Blei, Andrew Y

    David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent dirichlet allocation. J. Mach. Learn. Res., 3: 0 993–1022, March 2003. ISSN 1532-4435. URL https://www.jmlr.org/papers/v3/blei03a.html

    work page 2003

  34. [44]

    Basic Properties of Strong Mixing Conditions. A Survey and Some Open Questions

    Richard C Bradley. Basic properties of strong mixing conditions. a survey and some open questions. 2005. URL https://arxiv.org/abs/math/0511078

    work page internal anchor Pith review Pith/arXiv arXiv 2005

  35. [45]

    Towards Better Statistical Understanding of Watermarking LLMs

    Zhongze Cai, Shang Liu, Hanzhao Wang, Huaiyang Zhong, and Xiaocheng Li. Towards better statistical understanding of watermarking llms, 2024. URL https://arxiv.org/abs/2403.13027

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  36. [46]

    Beyond the Black Box: A Statistical Model for LLM Reasoning and Inference

    Siddhartha Dalal and Vishal Misra. Beyond the black box: A statistical model for llm reasoning and inference, 2024. URL https://arxiv.org/abs/2402.03175

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  37. [47]

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019. URL https://arxiv.org/abs/1810.04805

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  38. [48]

    Paul Doukhan. Mixing. In Mixing: Properties and Examples, pages 15--23. Springer, 1995. URL https://link.springer.com/book/10.1007/978-1-4612-2642-0

    work page doi:10.1007/978-1-4612-2642-0 1995

  39. [49]

    Detecting hallucinations in large language models using semantic entropy

    Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630 0 (8017): 0 625--630, 2024. doi:10.1038/s41586-024-07421-0. URL https://doi.org/10.1038/s41586-024-07421-0

    work page doi:10.1038/s41586-024-07421-0 2024

  40. [50]

    Three Bricks to Consolidate Watermarks for Large Language Models

    Pierre Fernandez, Antoine Chaffin, Karim Tit, Vivien Chappelier, and Teddy Furon. Three bricks to consolidate watermarks for large language models, 2023. URL https://arxiv.org/abs/2308.00113

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  41. [51]

    Wikimedia downloads

    Wikimedia Foundation. Wikimedia downloads. URL https://dumps.wikimedia.org

    work page

  42. [52]

    A Watermark for Large Language Models

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models, 2024 a . URL https://arxiv.org/abs/2301.10226

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  43. [53]

    On the Reliability of Watermarks for Large Language Models

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. On the reliability of watermarks for large language models, 2024 b . URL https://arxiv.org/abs/2306.04634

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  44. [54]

    Robust Distortion-free Watermarks for Language Models

    Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortion-free watermarks for language models. arXiv preprint arXiv:2307.15593, 2023. URL https://arxiv.org/abs/2307.15593

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  45. [55]

    Xiang Li, Feng Ruan, Huiyuan Wang, Qi Long, and Weijie J. Su. A statistical framework of watermarks for large language models: Pivot, detection efficiency and optimal rules. The Annals of Statistics, 53 0 (1), February 2025. ISSN 0090-5364. doi:10.1214/24-aos2468. URL http://dx.doi.org/10.1214/24-AOS2468

    work page doi:10.1214/24-aos2468 2025

  46. [56]

    ROUGE : A package for automatic evaluation of summaries

    Chin-Yew Lin. ROUGE : A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74--81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https://aclanthology.org/W04-1013/

    work page 2004

  47. [57]

    DeepSeek-V3 Technical Report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. URL https://arxiv.org/abs/2412.19437

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  48. [58]

    Detectgpt: Zero-shot machine-generated text detection using probability curvature

    Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In International conference on machine learning, pages 24950--24962. PMLR, 2023. URL https://proceedings.mlr.press/v202/mitchell23a.html

    work page 2023

  49. [59]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311--318, 2002. URL https://aclanthology.org/P02-1040/

    work page 2002

  50. [60]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1 0 (8): 0 9, 2019

    work page 2019

  51. [61]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer, 2023. URL https://arxiv.org/abs/1910.10683

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  52. [62]

    Can AI-Generated Text be Reliably Detected?

    Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. Can ai-generated text be reliably detected? arXiv preprint arXiv:2303.11156, 2023. URL https://arxiv.org/abs/2303.11156

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  53. [63]

    The Curse of Recursion: Training on Generated Data Makes Models Forget

    Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. The curse of recursion: Training on generated data makes models forget. arXiv preprint arXiv:2305.17493, 2023. URL https://arxiv.org/abs/2305.17493

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  54. [64]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024. URL https://arxiv.org/abs/2403.05530

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  55. [65]

    Optimizing watermarks for large language models

    Bram Wouters. Optimizing watermarks for large language models. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 53251--53269. PMLR, 21--27 ...

    work page 2024

  56. [66]

    A Critical Evaluation of Evaluations for Long-form Question Answering

    Fangyuan Xu, Yixiao Song, Mohit Iyyer, and Eunsol Choi. A critical evaluation of evaluations for long-form question answering, 2023. URL https://arxiv.org/abs/2305.18201

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  57. [67]

    OPT: Open Pre-trained Transformer Language Models

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. Opt: Open pre-trained transformer language models, 2022. URL https://arxiv.org/ab...

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  58. [68]

    BERTScore: Evaluating Text Generation with BERT

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert, 2020. URL https://arxiv.org/abs/1904.09675

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  59. [69]

    Provable Robust Watermarking for AI-Generated Text

    Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu-Xiang Wang. Provable robust watermarking for ai-generated text. arXiv preprint arXiv:2306.17439, 2023. URL https://arxiv.org/abs/2306.17439

    work page internal anchor Pith review Pith/arXiv arXiv 2023