Pith. sign in

REVIEW 3 major objections 5 minor 48 references

Optimal Estimation of Watermark Proportions in Hybrid AI-Human Texts

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read For watermarks with continuous pivotal statistics, the paper proves the AI-written proportion in mixed human-AI text is identifiable, constructs estimators that reach the minimax lower bound, and shows the binary green-red list watermark…

desk verdict New problem, clean identifiability dichotomy, strong empirics; the minimax optimality claim overreaches because the estimators must estimate the alternative CDF that the lower bound treats as known. read the letter →

arxiv 2506.22343 v1 pith:JEFFEZBJ submitted 2025-06-27 stat.ML cs.CLcs.LGstat.ME

Xiang Li , Garrett Wen , Weiqing He , Jiayuan Wu , Qi Long , Weijie J. Su This is my paper
classification stat.MLcs.CLcs.LGstat.ME MSC 62F1262C2062G0568T50
keywords watermarkproportionestimationpivotalstatisticsmixturemodelidentifiabilityminimaxGumbel-maxinversetransformgreen-redlisthybridAI-humantext
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks a practical question: when a document mixes human-written and watermarked LLM text, how much of it is machine-generated? It shows the answer depends on the watermarking scheme. For watermarks whose detection scores are continuous, such as the Gumbel-max and inverse-transform schemes, the proportion is identifiable and can be estimated at the minimax-optimal rate. For the green-red list watermark, whose detection score is binary, the proportion is not identifiable, and any estimator inherits an unavoidable bias. The paper proposes three estimators, the best of which solves a fixed-point equation with an optimal weight function, and verifies their accuracy on synthetic data and open-source LLM outputs.

What carries the argument

The central object is the pivotal statistic, a scalar score $Y_t = Y(w_t, \zeta_t)$ that follows a known null distribution when the token is not watermarked and an alternative distribution when it is. The load-bearing identity is the mixture equation $\bar F(x) = (1-\varepsilon)F_0(x) + \varepsilon\bar F_P(x)$, which expresses $\varepsilon$ as a ratio of weighted expectations; Assumption 3.1 makes the denominator tractable by forcing the alternative CDF to be far smaller than the null CDF near zero. The optimal weight function $v_{\mathrm{opt}}(x) = (1-g(x))/((1-\varepsilon)+\varepsilon g(x))$, where $g(x) = d\bar F_P(x)/dF_0(x)$, minimizes an upper bound on the estimator variance, and the optimal estimator is defined as the fixed point of a projected operator $\hat T(\varepsilon)$; Lemma 3.5 shows this operator is a contraction with high probability, so the fixed point is well posed. The minimax lower bounds are established through a two-point reduction on chi-square divergence, and the data-processing inequality gives the ordering $\sigma^\star_n \geq \tau^\star_n$ between the indicator-based and general-estimator bounds.

What would settle it

On a real LLM with a fixed vocabulary of about 50,000 tokens, generate watermarked tokens with the inverse-transform scheme and compute $F_P(x)/F_0(x)$ for small $x$; if the ratio does not approach zero, the paper's identifiability limit $\varepsilon = 1-\lim_{x\to 0}\bar F(x)/F_0(x)$ fails and the proposed estimators will carry bias. Equally decisive: generate mixed texts with known $\varepsilon$ under the green-red list watermark and check whether any estimator recovers $\varepsilon$ as $n$ grows; the theory predicts it cannot.

Watch

Extended reading notes

Core claim

The paper's central claim is that estimating the watermark proportion reduces to estimating $\varepsilon$ in a time-varying mixture model of pivotal statistics, $Y_t \sim (1-\varepsilon)\mu_0 + \varepsilon\mu_{1,P_t}$, where each pivotal statistic summarizes the evidence that a token is watermarked. When the null and alternative CDFs satisfy Assumption 3.1, notably the tail condition $F_P(x)/F_0(x) \to 0$ as $x \to 0$, the proportion is identifiable as $\varepsilon = 1 - \lim_{x\to 0} \bar F(x)/F_0(x)$. The paper constructs estimators from the moment identity $\varepsilon = (E_{F_0}[v] - E_{\bar F}[v]) / (E_{F_0}[v] - E_{\bar F_P}[v])$, derives the variance-minimizing weight function $v_{\mathrm{opt}}(x) = (1-g(x))/((1-\varepsilon)+\varepsilon g(x))$ with $g = d\bar F_P/dF_0$, and proves that the resulting fixed-point estimator attains the minimax lower bound $\tau^\star_n/\sqrt{n}$ over all measurable estimators based on pivotal statistics. The binary green-red list watermark is shown to be a separate, non-identifiable regime where the bias cannot be removed.

Load-bearing premise

The load-bearing premise is that, near the detection threshold, watermarked scores are vanishingly rare compared with human scores, and that a comparable fully-watermarked sample is available to estimate the averaged alternative distribution.

Editorial extensions

If this is right

  • For Gumbel-max and inverse-transform watermarks, a verifier can output a single number for the AI-written share with error that shrinks at the fastest rate any estimator based on pivotal statistics could achieve.
  • For green-red list deployments, proportion claims are statistically unsupported; only all-or-nothing detection is honest, because the binary scores cannot separate the proportion from the unknown alternative distributions.
  • Post-processing edits such as substitution, insertion, or deletion do not break the estimator, so it applies to edited hybrid documents as well as clean mixtures.
  • The refined estimator's accuracy depends on a good fully-watermarked calibration sample, so practical systems should collect such calibration data from the target model family before deployment.
  • The initial estimator, which ignores the alternative distribution, cannot reach the minimax rate; the correction term $\bar F_P$ is not optional but essential for optimal estimation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A practical corollary the authors leave implicit is that regulation thresholds built on green-red list watermarks would be unenforceable; switching to continuous-pivot watermarks is the natural remedy for any policy that requires a numerical AI-contribution share.
  • If the inverse-transform watermark's identifiability condition holds only asymptotically in vocabulary size, real LLMs with finite vocabularies may show small but nonzero bias; a deployment workflow should validate the tail ratio on the target model's next-token distributions before trusting point estimates.
  • Because the estimator needs a comparable fully-watermarked sample, the closeness of the calibration model to the target model is likely the dominant practical error source; one testable extension is to measure how estimator error grows as the calibration model diverges from the generating model.
  • The success of global proportion estimation relative to segment localization suggests a cheap two-stage workflow: run the optimal proportion estimator as a filter, and invoke expensive segment search only when the estimated proportion crosses a policy threshold.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies estimation of the proportion ε of watermarked tokens in mixed human-AI texts, framed as the mixture proportion in a model Y_t | P_t ∼ (1−ε)µ_0 + εµ_{1,P_t} based on continuous pivotal statistics. The authors prove identifiability under a tail-ratio condition (Assumption 3.1), propose three estimators: an initial threshold-based estimator, a refined estimator using an auxiliary watermarked sample to estimate the averaged alternative CDF, and an optimal fixed-point estimator, and derive MAE bounds and Le Cam–style minimax lower bounds. They claim the estimators achieve the minimax bounds, and they support this with synthetic simulations and open-source LLM experiments for Gumbel-max and inverse-transform watermarks. The green-red list watermark is shown to be non-identifiable, with a regularized-MLE analysis demonstrating unavoidable bias.

Significance. If the core claims hold, this would be a useful contribution: it gives the first systematic treatment of watermark-proportion estimation, identifies a clean sufficient condition for identifiability, and provides simple estimators with n^{-1/2}-type error bounds. The appendix contains self-contained proofs, the Le Cam lower-bound construction is standard and mostly correct, and the experiments are reasonably broad, including three LLMs and post-processing edits. The code is publicly linked. The main reservation is that the central 'minimax optimality' claim is not actually established in the full model with unknown NTP distributions, because the lower bound and the estimator analysis are conducted in different worlds, and the inverse-transform identifiability example is only asymptotic in vocabulary size. These issues are fixable by re-stating the claims and adding the missing analysis or explicit limitations.

major comments (3)
  1. [Section 4.3, Theorem 4.4 vs. Section 4.2, Theorem A.1/A.2] The minimax lower bound and the achievability result address different problems, so the 'achieve these lower bounds' claim in the abstract and Section 4.3 is not justified as stated. Theorem 4.4 draws P_t uniformly from a fixed set and fixes the averaged alternative CDF Fbar_P, so the lower bound is for an oracle problem in which the nuisance is effectively known. In contrast, the refined and optimal estimators require an auxiliary fully watermarked sample to construct bF_P and bg, and the bias terms (explicitly ε(Fbar_P − bF_P)/(F0 − bF_P) in Theorem A.1 and B in Theorem A.2) vanish only when supx |bg(x)−g(x)|/min{1,bg(x)} → 0. No theorem establishes this convergence from a finite auxiliary sample, nor is there a bound on how the auxiliary sample size must grow. Consequently, in the realistic setting where Fbar_P is unknown and must be estimated, the present theorems do not show that the estimators achieve the minimax lower bound; the lower bound class and the estimator class differ, and no full-nuisance minimax analysis is provided. The statement that OPT 'achieves' the bound should be either proven for the auxiliary-estimation problem or weakened to a conditional oracle-quantity statement.
  2. [Appendix A.8, proof of Theorem 4.4] The constant in the stated lower bound does not follow from the proof. With Δ = σ⋆_n/√(2n), the displayed total-variation bound becomes DTV(ρ1,ρ2) ≤ √(n/2)·Δ/σ⋆_n = 1/2, and Le Cam's inequality (26) gives a lower bound of (Δ/2)(1 − 1/2) = σ⋆_n/(4√(2n)), not σ⋆_n/√(2n) as stated. The same factor-of-four issue applies to part (b). Unless a different Δ is intended or the final display is meant only up to universal constants, the theorem statement and the proof are inconsistent. This matters because the paper's 'matching minimax lower bounds' claim relies on these constants being correct; the achievability bounds in Theorem 4.2 are of the form (σ⋆_n + o(1))/√n, so a factor-four discrepancy is harmless for rates but the theorem should be corrected or explicitly described as rate-optimal up to constants.
  3. [Remark 3.3 and Lemma 3.3] The identifiability guarantee for the inverse-transform watermark is only asymptotic in the vocabulary size |W|, and the paper explicitly concedes in Remark 3.3 that the required condition 'may not be satisfied in practice.' The proof of Lemma 3.3 verifies the tail ratio only on the interval x ∈ (0, 1 − P(1)), and the finite-vocabulary case with P(2)·log|W| = Ω(1) is not covered. Since the inverse-transform watermark is one of the two flagship examples in the abstract and in the experimental section, the claim 'ε is identifiable for these two watermarks' should be qualified so that the main text cannot be read as asserting identifiability for realistic finite vocabularies. A concrete statement of the regime in which the estimators' bias is controlled would make the scope of the contribution clear.
minor comments (5)
  1. [Section 3.3, sentence before Eq. (3)] There is a typo: 'it must satify' should be 'it must satisfy'.
  2. [Theorem 4.1] The displayed bound is written as '≤ σ_n √n', but with σ_n^2 defined as the variance term under the square root, the correct form should be σ_n/√n. Please fix the typo, as the same misformatting appears around Corollary 2.
  3. [Section 5.1, 'How we obtain bg'] The number of histogram bins is denoted N, which conflicts with the auxiliary sample size N used in Section 3.3 for the watermarked sample Y^wm_1, ..., Y^wm_N. Please use distinct notation, e.g., B for the number of bins.
  4. [Appendix B.1, 'Optimization for the refined estimator'] This heading refers to the optimal estimator bε_opt, while 'refined estimator' elsewhere denotes bε_rfn. Rename the subsection to 'Optimization for the optimal estimator' to avoid confusion.
  5. [Figure 1d and Appendix B.1] The figure caption does not mention that the plotted INI and IND results for the green-red list watermark are projected onto [0,1]. Please state this explicitly in the caption, since Theorem 4.1 and Theorem 4.2 are stated for unprojected estimators.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the estimator construction inverts the stated mixture model, the fixed-point estimator is a standard Z-estimator, and the minimax lower bound is derived independently via Le Cam's method.

full rationale

The paper's central quantity is the proportion epsilon in the mixture model (1), and equation (2) is just the model equation expressed in CDF form. The proposed estimators are method-of-moments plug-ins: bF replaces Fbar and bFP replaces Fbar_P in the exact identity epsilon = (EF0[v]-EFbar[v])/(EF0[v]-EFbar_P[v]). This is inversion of the model, not definitional circularity: epsilon is not defined in terms of the estimator, and bFP is built from purely watermarked auxiliary data, not from the target mixture or from epsilon itself. The fixed-point estimator bT is a nonlinear estimating equation whose solution is the unknown parameter; the contraction argument in Lemma 3.5 is a standard device for solving such equations, and the true parameter is shown to satisfy the population version, which is the usual identification condition rather than a self-referential renaming. The minimax lower bound in Theorem 4.4 is obtained by Le Cam's method using chi-square divergence between two mixture distributions with different epsilon values; the bound does not assume the proposed estimator and is not equal to it by construction. The matching of tau*_n between the variance-minimization calculation and the lower bound is a genuine efficiency result, not an identity: the upper bound follows from the variance of the optimal weight function, and the lower bound follows from the chi-square distance between the least-favorable pair. The paper does rely on the authors' prior work [19, 20] for the pivotal-statistics framework and for the inverse-transform CDF asymptotics, but these are published, stated-assumption results with proofs, and they are not the target result of this paper. Remark 3.3 honestly flags that the inverse-transform condition may not hold in practice, and Section 4.3 explicitly qualifies the achievability claim as holding 'up to the estimation error og(1) in estimating Fbar_P'. The lower bound problem fixes Fbar_P while the estimator must estimate it, which is a real scope gap but not a circular reduction: the estimator is not defined in terms of the lower bound, and the paper does not conceal the og(1) term. No fitted parameter is relabeled as a prediction, no uniqueness theorem is imported to forbid alternatives, and no known result is merely renamed. On the evidence quoted, the derivation chain is self-contained and not circular.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the surrogate independence model, the tail-ratio condition on the watermark's pivotal statistics, and the availability of a representative fully-watermarked calibration sample. The first two are statistical modeling assumptions; the third is a data-availability assumption that is not emphasized as a limitation in the paper. No new physical entities are introduced.

free parameters (3)
  • delta (tail threshold) = selected from {10^-1, 10^-2, 10^-3}
    Used by INI and IND estimators in Eq. (4) and (5); chosen by grid search in experiments, not by theory.
  • epsilon_min (projection constant) = 10^-3
    Projection interval for the fixed-point estimator Eq. (8); avoids numerical issues but biases estimates when true epsilon is below epsilon_min or above 1-epsilon_min.
  • Number of histogram bins for density ratio = 500
    Used to estimate g(x) = dFbar_P/dF0 in experiments; a hand-chosen hyperparameter affecting the accuracy of bF_P.
assumptions (4)
  • domain assumption Surrogate Model 2: NTP distributions P1,...,Pn are fixed and Y1,...,Yn are independent (Section 3.1).
    The theory is developed under this surrogate; the authors argue Model 1 and Model 2 are indistinguishable to the verifier, but the validity of the independence/dependence structure is an assumption.
  • domain assumption Assumption 3.1: F0 and F_P differentiable, lim_{x to 0} F_P(x)/F0(x) = 0 for non-singular P, and all NTP distributions are non-singular.
    The identifiability result Lemma 3.2 relies on the tail ratio vanishing; for the inverse-transform watermark this holds only asymptotically in vocabulary size (Remark 3.3).
  • domain assumption An accurate fully-watermarked sample from a comparable LLM is available to estimate Fbar_P (Section 3.3, Eq. (5), (8)).
    Refined and optimal estimators replace Fbar_P by bF_P; if the auxiliary sample is not representative, the bias term in Theorem A.1 does not vanish.
  • domain assumption Perfect pseudorandomness: zeta_t are i.i.d. from a known distribution (Section 2).
    Standard in the watermarking literature; the verifier can reconstruct zeta_t given the key and text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimal Estimation of Watermark Proportions in Hybrid AI-Human Texts." pith.science (2026). https://pith.science/paper/JEFFEZBJ

@misc{pith2026250622343,
  author       = {Pith},
  title        = {Pith review of: Optimal Estimation of Watermark Proportions in Hybrid AI-Human Texts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JEFFEZBJ}},
  note         = {Machine review of arXiv:2506.22343}
}
read the original abstract

Text watermarks in large language models (LLMs) are an increasingly important tool for detecting synthetic text and distinguishing human-written content from LLM-generated text. While most existing studies focus on determining whether entire texts are watermarked, many real-world scenarios involve mixed-source texts, which blend human-written and watermarked content. In this paper, we address the problem of optimally estimating the watermark proportion in mixed-source texts. We cast this problem as estimating the proportion parameter in a mixture model based on \emph{pivotal statistics}. First, we show that this parameter is not even identifiable in certain watermarking schemes, let alone consistently estimable. In stark contrast, for watermarking methods that employ continuous pivotal statistics for detection, we demonstrate that the proportion parameter is identifiable under mild conditions. We propose efficient estimators for this class of methods, which include several popular unbiased watermarks as examples, and derive minimax lower bounds for any measurable estimator based on pivotal statistics, showing that our estimators achieve these lower bounds. Through evaluations on both synthetic data and mixed-source text generated by open-source models, we demonstrate that our proposed estimators consistently achieve high estimation accuracy.

Figures

Figures reproduced from arXiv: 2506.22343 by the authors.

Figure 3
Figure 3. CDF plots of mixture distributions for the Gumbel-max watermark. Here, [PITH_FULL_IMAGE:figures/full_fig_p047_3.png] view at source ↗
Figure 4
Figure 4. CDF plots of mixture distributions for the inverse transform watermark. Here, [PITH_FULL_IMAGE:figures/full_fig_p048_4.png] view at source ↗
Figure 5
Figure 5. Mean absolute errors (MAEs) for the Gumbel-max watermark. Here [PITH_FULL_IMAGE:figures/full_fig_p049_5.png] view at source ↗
Figures from the paper (5 more)
Figure 6
Figure 6. Figure 6: Mean absolute errors (MAEs) for the inverse transform watermark. Here [PITH_FULL_IMAGE:figures/full_fig_p050_6.png]
Figure 7
Figure 7. Figure 7: More results illustrating the inherent bias caused by non-identifiability. We vary the values [PITH_FULL_IMAGE:figures/full_fig_p051_7.png]
Figure 8
Figure 8. Figure 8: Mean absolute errors of estimators on the C4 dataset, evaluated across various true [PITH_FULL_IMAGE:figures/full_fig_p053_8.png]
Figure 9
Figure 9. Figure 9: Mean absolute errors of estimators on the C4 dataset, evaluated across various true [PITH_FULL_IMAGE:figures/full_fig_p054_9.png]
Figure 10
Figure 10. Figure 10: Mean absolute errors of estimators on the arXiv dataset, evaluated across various true [PITH_FULL_IMAGE:figures/full_fig_p055_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 35 canonical work pages

  1. [1]

    Aaronson

    S. Aaronson. Watermarking of large language models.https://simons.berkeley.edu/talks/ scott-aaronson-ut-austin-openai-2023-08-17 , August 2023

    work page 2023

  2. [2]

    W. R. Blischke. Moment estimators for the parameters of a mixture of two binomial distributions. The Annals of Mathematical Statistics, 33(2):444–454, 1962

    work page 1962

  3. [3]

    Bommasani, D

    R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021

    arXiv 2021

  4. [4]

    Brown, B

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners. InAdvances in neural information processing systems, volume 33, pages 1877–1901, 2020

    work page 1901

  5. [5]

    T. T. Cai and W. Sun. Optimal screening and discovery of sparse signals with applications to multistage high throughput studies.Journal of the Royal Statistical Society Series B: Statistical Methodology, 79(1):197–223, 2017

    work page 2017

  6. [6]

    Cohan, F

    A. Cohan, F. Dernoncourt, D. S. Kim, T. Bui, S. Kim, W. Chang, and N. Goharian. A discourse- aware attention model for abstractive summarization of long documents. InConference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, volume 2, pages 615–621, 2018. 18

    work page 2018

  7. [7]

    D. Das, K. De Langis, A. Martin, J. Kim, M. Lee, Z. M. Kim, S. Hayati, R. Owan, B. Hu, R. Parkar, et al. Under the surface: Tracking the artifactuality of LLM-generated data.arXiv preprint arXiv:2401.14698, 2024

    arXiv 2024

  8. [8]

    Dathathri, A

    S. Dathathri, A. See, S. Ghaisas, P.-S. Huang, R. McAdam, J. Welbl, V. Bachani, A. Kaskasoli, R. Stanforth, T. Matejovicova, et al. Scalable watermarking for identifying large language model outputs. Nature, 634(8035):818–823, 2024

    work page 2024

Show all 48 references
  1. [9]

    Y. Dodge. The Oxford dictionary of statistical terms. Oxford University Press, USA, 2003

    2003

  2. [10]

    Donoho and J

    D. Donoho and J. Jin. Higher criticism for detecting sparse heterogeneous mixtures.The Annals of Statistics, 32(3):962–994, 2004

    2004

  3. [11]

    Dubey, A

    A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan, et al. The Llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  4. [12]

    Dugan, D

    L. Dugan, D. Ippolito, A. Kirubarajan, S. Shi, and C. Callison-Burch. Real or fake text?: Investigating human ability to detect boundaries between human-written and machine-generated text. In AAAI Conference on Artificial Intelligence, volume 37, pages 12763–12771, 2023

    2023

  5. [13]

    E. J. Gumbel.Statistical theory of extreme values and some practical applications: A series of lectures, volume 33. US Government Printing Office, 1948

    1948

  6. [14]

    Z. Hu, L. Chen, X. Wu, Y. Wu, H. Zhang, and H. Huang. Unbiased watermark for large language models. In International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=uWVC5FVidc

    2024

  7. [15]

    Kirchenbauer, J

    J. Kirchenbauer, J. Geiping, Y. Wen, J. Katz, I. Miers, and T. Goldstein. A watermark for large language models. InInternational Conference on Machine Learning, volume 202, pages 17061–17084, 2023

    2023

  8. [16]

    Kirchenbauer, J

    J. Kirchenbauer, J. Geiping, Y. Wen, M. Shu, K. Saifullah, K. Kong, K. Fernando, A. Saha, M. Goldblum, and T. Goldstein. On the reliability of watermarks for large language models. In International Conference on Learning Representations, 2024. URL https://openreview.net/ forum...

    2024

  9. [17]

    Kuditipudi, J

    R. Kuditipudi, J. Thickstun, T. Hashimoto, and P. Liang. Robust distortion-free watermarks for language models.Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=FpaCL1MO2C

    2024

  10. [18]

    X. Li, G. Li, and X. Zhang. Segmenting watermarked texts from language models. InNeural Information Processing Systems, 2024

    2024

  11. [19]

    X. Li, F. Ruan, H. Wang, Q. Long, and W. J. Su. Robust detection of watermarks in large language models under human edits.arXiv preprint, 2024

    2024

  12. [20]

    X. Li, F. Ruan, H. Wang, Q. Long, and W. J. Su. A statistical framework of watermarks for large language models: Pivot, detection efficiency and optimal rules.The Annals of Statistics, 53(1):322–351, 2025. 19

    2025

  13. [21]

    Liang, Z

    W. Liang, Z. Izzo, Y. Zhang, H. Lepp, H. Cao, X. Zhao, L. Chen, H. Ye, S. Liu, Z. Huang, et al. Monitoring AI-modified content at scale: A case study on the impact of ChatGPT on AI conference peer reviews. InInternational Conference on Machine Learning, 2024

    2024

  14. [22]

    Milano, J

    S. Milano, J. A. McGrane, and S. Leonelli. Large language models challenge the future of higher education. Nature Machine Intelligence, 5(4):333–334, 2023

    2023

  15. [23]

    ChatGPT: Optimizing language models for dialogue.http://web.archive.org/web/ 20230109000707/https://openai.com/blog/chatgpt/, Jan 2023

    OpenAI. ChatGPT: Optimizing language models for dialogue.http://web.archive.org/web/ 20230109000707/https://openai.com/blog/chatgpt/, Jan 2023

    2023

  16. [24]

    J. Piet, C. Sitawarin, V. Fang, N. Mu, and D. Wagner. Mark my words: Analyzing and evaluating language model watermarks.arXiv preprint arXiv:2312.00273, 2023

    2023 arXiv

  17. [25]

    Radford, J

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al. Language models are unsupervised multitask learners.OpenAI blog, 1(8):9, 2019

    2019

  18. [26]

    Radford, J

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever. Robust speech recognition via large-scale weak supervision. InInternational Conference on Machine Learning, pages 28492–28518. PMLR, 2023

    2023

  19. [27]

    Raffel, N

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of Machine Learning Research, 21(1):5485–5551, 2020

    2020

  20. [28]

    J. Shao. Mathematical statistics. Springer Science & Business Media, 2003

    2003

  21. [29]

    Starbird

    K. Starbird. Disinformation’s spread: Bots, trolls and all of us.Nature, 571(7766):449–450, 2019

    2019

  22. [30]

    Thomas and A

    M. Thomas and A. T. Joy.Elements of information theory. Wiley-Interscience, 2006

    2006

  23. [31]

    A. W. Van der Vaart.Asymptotic statistics, volume 3. Cambridge university press, 2000

    2000

  24. [32]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. InAdvances in neural information processing systems, volume 30, 2017

    2017

  25. [33]

    Virtanen, R

    P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, et al. SciPy 1.0: Fundamental algorithms for scientific computing in Python.Nature methods, 17(3):261–272, 2020

    2020

  26. [34]

    M. J. Wainwright.High-dimensional statistics: A non-asymptotic viewpoint, volume 48. Cam- bridge university press, 2019

    2019

  27. [35]

    B. Wouters. Optimizing watermarks for large language models. InInternational Conference on Machine Learning, 2024

    2024

  28. [36]

    Y. Wu, Z. Hu, H. Zhang, and H. Huang. DiPmark: A stealthy, efficient and resilient watermark for large language models.arXiv preprint arXiv:2310.07710, 2023. 20

    2023 arXiv

  29. [37]

    Z. Wu, D. Ji, K. Yu, X. Zeng, D. Wu, and M. Shidujaman. AI creativity and the human-AI co-creation model. InHuman-Computer Interaction International Conference, pages 171–190. Springer, 2021

    2021

  30. [38]

    Y. Xie, X. Li, T. Mallick, W. J. Su, and R. Zhang. Debiasing watermarks for large language models via maximal coupling.arXiv preprint arXiv:2411.11203, 2024

    2024 arXiv

  31. [39]

    Y. Xie, T. Qi, J. Yi, R. Whalen, J. Huang, Q. Ding, Y. Xie, X. Xie, and F. Wu. Measuring human contribution in AI-assisted content generation.arXiv preprint arXiv:2408.14792, 2024

    2024 arXiv

  32. [40]

    Zhang, C

    Q. Zhang, C. Gao, D. Chen, Y. Huang, Y. Huang, Z. Sun, S. Zhang, W. Li, Z. Fu, Y. Wan, et al. LLM-as-a-Coauthor: Can mixed human-written and machine-generated text be detected? In Findings of the Association for Computational Linguistics: NAACL 2024, pages 409–436, 2024

    2024

  33. [41]

    OPT: Open pre-trained transformer language models.arXiv preprint arXiv:2205.01068, 2022

    S.Zhang, S.Roller, N.Goyal, M.Artetxe, M.Chen, S.Chen, C.Dewan, M.Diab, X.Li, X.V.Lin, et al. OPT: Open pre-trained transformer language models.arXiv preprint arXiv:2205.01068, 2022

    2022 arXiv

  34. [42]

    X. Zhao, P. V. Ananth, L. Li, and Y.-X. Wang. Provable robust watermarking for AI-generated text. InInternational Conference on Learning Representations, 2024. URLhttps://openreview. net/forum?id=SsmT8aO45L

    2024

  35. [43]

    X. Zhao, L. Li, and Y.-X. Wang. Permute-and-Flip: An optimally robust and watermarkable decoder for LLMs.arXiv preprint arXiv:2402.05864, 2024

    2024 arXiv

  36. [44]

    X. Zhao, C. Liao, Y.-X. Wang, and L. Li. Efficiently identifying watermarked segments in mixed-source texts. arXiv preprint arXiv:2410.03600, 2024

    2024 arXiv

  37. [45]

    C. Zhu, J. Galjaard, P.-Y. Chen, and L. Y. Chen. Duwak: Dual watermarks in large language models. In Findings of the Association for Computational Linguistics, 2024

    2024

  38. [46]

    1 n nX t=1 pt(ε) # 1 −

    G. K. Zipf.Human behavior and the principle of least effort: An introduction to human ecology. Ravenio books, 2016. 21 Appendices A Proofs of Theoretical Analysis A.1 Proof of Results in Section 3.2 We provide the proofs of Lemmas 3.1, 3.2, and 3.3 in this subsection. Proof of...

    2016

  39. [47]

    For ε ∈ [εmin, 1 2 ]: Var ¯F 1 − bg [1 − ε) + εbg ≤ E ¯F [1 − bg]2 (1 − ε) + εbg]2 ≤ E ¯F [1 − bg]2 (1 − ε)2 ≤ 4 E ¯F [1 − bg]2 < ∞

  40. [48]

    scipy.optimize.minimize

    For ε ∈ [ 1 2 , 1 − εmin]: Var ¯F 1 − bg (1 − ε) + εbg ≤ E ¯F [1 − bg(x)]2 [εbg(x)]2 ≤ 4 E ¯F [1 − bg(x)]2 [bg(x)]2 < ∞. Thus, the variance is bounded, and we define 4 E ¯F [1 − bg]2 + E ¯F [1 − bg(x)]2 [bg(x)]2 ≤ 4 E ¯F [bg − 1/bg]2 =: M 2 . Similarly, for the variance of fun...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.