Methodology · arXiv:2608.14906 v2 · submitted 14 Aug 2026
Optimal Watermark Localization in Mixed-Source Large Language Model Texts
The paper says
“Watermarking provides a principled way to authenticate text generated by large language models (LLMs). In practice, however, the final text may be mixed-source, with watermark evidence surviving at only a subset of token positions after rewriting, insertion, deletion, or paraphrasing.”
more of the abstract
Although prior work has studied global detection of watermark signals, when such signals can be localized remains unclear. We formulate watermark localization as a token-level multiple-testing problem based on pivotal statistics, with a latent indicator recording whether watermark dependence survives at each position. Under an asymptotic regime indexed by exponents for signal sparsity, next-token concentration, and effective-vocabulary growth, we derive a sharp boundary for global detection and phase transitions for discovery and classification within the class of coordinatewise pivot-based localization rules. We show that discovery is strictly harder than detection and that consistent classification is impossible across the parameter regime within this class. We then develop an adaptive thresholding method that does not require knowledge of the exponents or time-varying next-token distributions, but uses a data-driven estimate of the surviving watermark fraction. The method attains the optimal discovery boundary and near-optimal discovery power relative to homogeneous pivot-based rules. Simulations support the theoretical phase transitions, while experiments on model-generated texts demonstrate practical localization performance under common edit mechanisms.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 27 Aug 2026 · of version 1 of the paper; version 2 is newer · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. Solid phase-transition theory for watermark localization, with a real scoping overstatement in the discovery-versus-detection claim and a plug-in accuracy condition that is neither proven nor met at low temperatures. This line is wrong
3 major objections · 5 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the references say.
The paper says, at Abstract; Section 1.1; Theorem 4.2; Section 7
The review did not quote this passage.
Pith says: Objection 1 of 3 · major
The headline claim that discovery is strictly harder than detection is only proved inside the class D_n of coordinatewise local rules. The impossibility half of Theorem 4.2 uses Lemma B.8 to reduce every rule in D_n to a coordinatewise right-tail threshold rule; it does not constrain procedures that combine evidence across positions.
more of objection 1
Global detection in Theorem 4.1, by contrast, is unrestricted. The comparison is therefore asymmetric, and Section 7 explicitly leaves open the possibility that nonlocal localization procedures could discover in the detectable-but-undiscoverable region. The abstract and Section 1.1 nevertheless state the gap without this qualifier. Please either add a general lower bound holding outside D_n or consistently qualify the claim as holding within the class D_n.
The paper says, at Theorem 4.5, Eq. (9); Remark 3.2; Section 6, Table 2
The review did not quote this passage.
Pith says: Objection 2 of 3 · major
The adaptive optimality of SPOT is conditional on the accuracy condition P(|epsilon_hat_n - epsilon_n| > c*eta_n) = o(1/n) with eta_n = C2/(log n)^2. The paper does not prove this condition for the Li et al. (2026) estimator used in the implementation, and Table 2 shows relative RMSE values between 1.37 and 4.12 at T=0.5 and T=0.3.
more of objection 2
That is, the estimator is inaccurate precisely in the low-temperature regime where the plug-in version of SPOT is meant to operate. The claim that SPOT-plugin attains the optimal discovery boundary is therefore not established for the implemented estimator. The theorem should be stated as conditional on Eq. (9), or a verification of Eq. (9) under the paper's assumptions should be provided.
The paper says, at Theorem 4.5; Remark 4.3; Algorithm 1
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
The grid coverage condition 0 < u_min < p+q < u_max < 1 is not truly adaptive unless the endpoints are allowed to depend on n. For any fixed u_max < 1, there exist admissible pairs (p,q) with p+q in (u_max, 1), and then the practical choices [0.05, 0.95] or [0.005, 0.98] cannot cover the needed threshold scale.
more of objection 3
To support the claim that SPOT does not require knowledge of p and q, the paper should specify a sequence u_max(n) that approaches 1 fast enough to cover every p+q < 1, or else state the boundary-optimality result with that additional data-dependent choice of the grid.
5 minor objections
The paper says, at Definition 4.1 and Assumption 4.1(c)
Pith says: minor objection 1 of 5
Definition 4.1 states the alternative H1 with a fixed mixing proportion epsilon_n, whereas Assumption 4.1(c) allows position-dependent conditional probabilities pi_t that may range between c*epsilon_n and C*epsilon_n. The detection proof uses only pi_t approximately epsilon_n, but the definition should be aligned with the assumption to avoid ambiguity about which model is covered by Theorem 4.1.
The paper says, at Section 5, Eq. (11)
Pith says: minor objection 2 of 5
The notation p_miss(delta) for the probability of an empty discovery set is confusing because p is already used as the sparsity exponent throughout the paper. Consider renaming it, for example to r_miss(delta) or pi_0(delta).
show 3 more
The paper says, at Section 4.3 and Figure 7
Pith says: minor objection 3 of 5
The phrase 'up to boundary cases' is used for the detection and discovery phase transitions, but the boundary behavior is never characterized. If the boundary values are not needed for the claims, please say so explicitly; if they are, state the boundary results.
The paper says, at Figure 7 and Appendix C
Pith says: minor objection 4 of 5
The lower endpoint of the q-grid, log_n(|W_n|/(|W_n|-1)), is unexplained in the caption and appears only in the simulation appendix. A one-sentence explanation in the main text would help readers understand why the simulation grid is clipped away from q=0.
The paper says, at Section 6, Table 2
Pith says: minor objection 5 of 5
The relative RMSE is defined in Eq. (12) two paragraphs after Table 2 is first mentioned; move the definition to the first occurrence of the table so the metric is clear when the table is read.
What would settle it
Derive the discovery lower bound for arbitrary decision rules (not just D_n) in the independent-mixture case satisfying Assumptions 4.1–4.2 with fixed vocabulary (α = 0) and p ∈ (0, 1), p + q < 1. Compute the Bayes-optimal mFDR for a rule with full access to all pivotal statistics; if even this oracle cannot achieve vanishing mFDR because the per-position likelihood ratio is bounded, then the local-rule restriction is not hiding a gap, and the paper should state the general theorem. If a nonlocal rule does achieve vanishing mFDR in that region, the abstract's 'strictly harder' claim is false and must be requalified as 'harder for coordinatewise pivot-based local rules.'
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 27 Aug 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 3 major objections, 5 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
The core of this paper is the first sharp phase-transition analysis for token-level watermark localization: it cleanly separates detection, discovery, and classification, proves the discoverable region p < alpha and p + q < 1 for coordinatewise pivot-based local rules, and shows classification is impossible in that class. The proofs are detailed, the simulations track the predicted boundaries, and SPOT is a sensible adaptive thresholding method that beats the AOL baseline at high temperature. That is real progress on a question prior work left open, and the paper deserves a serious referee.
The main soft spot is the headline claim that discovery is strictly harder than detection. That is proved only inside Dn, the class of coordinatewise local rules. The detection boundary is unrestricted, so the comparison is asymmetric. Section 7 explicitly concedes that procedures exploiting multiple token positions, like change-point or block methods, could in principle discover in the detectable-but-undiscoverable region. The abstract and introduction drop that qualifier. That is a scoping overstatement, not a mathematical error, and it should be fixed in revision.
The second issue is the accuracy condition on the fraction estimator in Equation (9). Theorem 4.5 assumes P(|eps_hat - eps| > c eta_n) = o(n−1), but the paper neither proves nor verifies this for the Li et al. plug-in estimator. Their own Table 2 shows relative RMSE above 1 at T=0.5 and T=0.3, so the plug-in version is operating precisely where the condition fails. The oracle version is fine, and the authors do acknowledge the plug-in bottleneck in the discussion. Still, the adaptive optimality claim for the practical method is conditional on an unverified input.
Minor concerns: the empirical tables have no error bars, and the calibration constants are tuned over grids, which makes the reported gains look slightly optimistic. The citation pattern leans heavily on self-authored prior work, but those papers supply the pivotal statistic, the detection boundary, and the fraction estimator, so the dependence is legitimate rather than gratuitous.
Bottom line: the phase transitions are the real contribution, they appear correct within their stated class, and the local-rule caveat is not a fatal flaw. The paper needs a revision that qualifies the discovery-versus-detection claim and either proves condition (9) or states the adaptive optimality only for the oracle version. I would engage with it seriously, and I would send it to peer review.
The strongest objection
Theorem 4.2 establishes the discovery boundary only inside D_n, the class of coordinatewise pivot-based local rules defined in Section 3.1; Lemma B.8 reduces every rule in D_n to a right-tail threshold rule. The impossibility half—no local rule can discover when p ≥ α or p + q > 1—does not extend to rules that combine evidence across positions, such as the change-point segmentation or online localization methods cited in Section 1.2, nor to any procedure exploiting block or segment structure. Nevertheless, the abstract states 'we show that discovery is strictly harder than detection' without the qualifier, and the introduction calls the gap 'fundamental'. Since global detection is unrestricted while discovery is analyzed only within D_n, the comparison is asymmetric: a nonlocal rule could in principle discover in the detectable-but-undiscoverable region. Section 7 explicitly concedes this possibility for 'localization procedures that exploit information across multiple token positions.' Thus the central conceptual claim is a scoping overstatement unless a general lower bound is added. The SPOT-oracle achievability direction and the conditional impossibility theorem for D_n remain mathematically intact; the unverified accuracy condition in Equation (9) is a separate, implementation-level caveat rather than the main conceptual gap.
The reader's weakest_assumption lists two concerns: the unverified Equation (9) accuracy condition for the plug-in fraction estimator, and the restriction of the discovery impossibility to D_n. I regard the D_n restriction as the more load-bearing issue for the paper's central claim, because it affects the unqualified headline statement that 'discovery is strictly harder than detection' and the interpretation of the discoverable region as a fundamental statistical limit. The Equation (9) issue is real but conditional: Theorem 4.5 explicitly assumes it, and the paper's own Discussion acknowledges the plug-in estimator as a practical bottleneck. The local-rule scope, by contrast, is embedded in the main phase-transition theorem and is presented as a fundamental gap in the abstract. A general information-theoretic lower bound for discovery, or an explicit counterexample showing a nonlocal rule can discover in the detectable-but-undiscoverable region, would settle whether the restriction is benign. Since the theorems as stated appear internally consistent and the reader's CONDITIONAL verdict already flags the need for revision, this stress-test does not move the verdict.
The referee's report
If taken as results restricted to the class of coordinatewise pivot-based rules, the paper makes a substantial contribution: it identifies a sharp discovery boundary p < alpha and p + q < 1, shows a nontrivial separation between discovery and detection within that class, proves an impossibility of classification, and proposes a simple adaptive thresholding method whose oracle version attains the boundary. The appendix proofs are detailed, the simulation heatmaps align with the theoretical boundaries, and the empirical comparison is carefully calibrated under false-positive constraints. The main caveats are that the 'discovery is strictly harder than detection' claim is established only for local rules while detection is unrestricted, and that the adaptive optimality theorem for SPOT depends on an estimator accuracy condition that is not proved and is contradicted by the paper's own low-temperature experiments.
Pith's reading of the paper (a machine's summary, not the authors')
Watermarking an LLM's output embeds a hidden statistical signal in each token, but human edits erase that signal at some positions. This paper asks when, and where, the surviving evidence can still be found. It characterizes the answer by three exponents: $p$, the sparsity of surviving watermark positions; $q$, the concentration of the model's next-token probabilities; and $\alpha$, the growth rate of the low-probability vocabulary tail. The paper proves that global detection of any surviving signal is possible iff $\max\{p+q,\,2p+q-\alpha\}<1$, while locating (discovering) at least one watermarked position is possible only in the strictly smaller region $p<\alpha$ and $p+q<1$; consistent per-token classification is impossible throughout. A verifier should care because the boundary says when token-level attribution, credit assignment, or targeted review of mixed-source text is statistically feasible at all.
What carries the argument. The machinery is built on a scalar pivotal statistic $Y_t$ per position — for the Gumbel-max watermark, $Y_t=U_{t,w_t}$ is exactly uniform on $(0,1)$ whenever the token is independent of the pseudorandomness, regardless of the token distribution. This distribution-free null property turns localization into a multiple-testing problem with a known null law. Around it, the paper defines the dominant–core–light vocabulary decomposition governed by the exponents $(p,q,\alpha)$, which yields explicit tail asymptotics for the pivot under the alternative; a monotone likelihood-ratio argument (Lemma B.8) then reduces every coordinatewise local rule to a right-tail threshold rule $\delta_t=\mathbf{1}\{Y_t>1-n^{-u}\}$, making the phase boundaries provable by tail counting. SPOT scans thresholds of the form $\tau(u)=1-n^{-u}$, estimates a surrogate false-discovery level from empirical tail counts, and selects the largest discovery set whose estimated level is controlled.
Core claim. The paper's central claim is that watermark localization in mixed-source text undergoes sharp phase transitions indexed by $(p,q,\alpha)$. Under its generation-editing model (Assumptions 4.1–4.3), global detection is achievable exactly when $\max\{p+q,\,2p+q-\alpha\}<1$ (Theorem 4.1); discovery — selecting at least one watermark-preserving position with vanishing marginal false discovery rate — is achievable exactly when $p<\alpha$ and $p+q<1$ (Theorem 4.2); and consistent classification, requiring both the false discovery rate and the missed discovery rate to vanish, is impossible across the whole regime for coordinatewise pivot-based local rules (Theorem 4.3). The discoverable region is therefore strictly smaller than the detectable region, and when the effective vocabulary size is fixed ($\alpha=0$), discovery is impossible even where detection succeeds. The paper further claims that its adaptive method, SPOT, attains the discovery boundary without knowing the exponents, and recovers a constant fraction of the discoveries of the best homogeneous pivot-based rule.
The premise it rests on. The theory that the plug-in SPOT achieves the discovery boundary assumes the surviving-fraction estimate is within $O(1/(\log n)^2)$ of the truth with failure probability $o(n^{-1})$ — a condition the paper neither proves nor verifies for the estimator it actually uses, and which its own low-temperature experiments appear to violate; separately, the impossibility results cover only coordinatewise local rules, not all localization procedures.
If the paper is right (Pith's inferences, not the authors')
Flagging a document as watermarked can be statistically feasible in regimes where naming even one surviving token position is impossible, so a positive document-level detection is not a license to attribute specific passages.
For fixed effective vocabularies ($\alpha=0$), token-level discovery of surviving watermark evidence is impossible in theory, which explains why localization in the current setting requires the growing low-probability tail.
No coordinatewise pivot-based rule can both control false discoveries and avoid missing weak watermark-preserving positions, so exact token-level classification of mixed-source text is out of reach without additional structure.
The discovery boundary is achieved by SPOT without knowledge of the exponents, and its discovery power is within a constant factor of the best homogeneous pivot-based rule, making the theoretical region practically attainable.
Tr-GoF, an existing global detector, attains the enlarged growing-vocabulary detection boundary adaptively, so the extended detection limit is not only theoretically sharp but algorithmically accessible.
Because the impossibility results are proven only for the class of coordinatewise local decision rules, methods that borrow strength across neighboring tokens — for instance, change-point or segment-level procedures — could in principle discover signals in regimes where per-token rules provably fail; testing SPOT against such methods near the boundary $p=\alpha$ would be a direct probe.
The assumed accuracy condition on the surviving-fraction estimator (Equation 9) is the load-bearing part of the adaptive optimality theorem, yet it is neither proved nor checked for the plug-in estimator; the low-temperature gap between SPOT-oracle and SPOT-plugin seen in the experiments is consistent with exactly this condition being violated, so a fraction-estimation-free threshold rule would be
The phase-transition framework is tied to the Gumbel-max pivot family but should transfer to any watermark with a valid distribution-free pivot; a parallel boundary for semantic or sentence-level watermarks would require redefining the null-versus-signal unit and is left open by the paper.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Watermarking provides a principled way to authenticate text generated by large language models (LLMs). In practice, however, the final text may be mixed-source, with watermark evidence surviving at only a subset of token positions after rewriting, insertion, deletion, or paraphrasing. Although prior work has studied global detection of watermark signals, when such signals can be localized remains unclear. We formulate watermark localization as a token-level multiple-testing problem based on pivotal statistics, with a latent indicator recording whether watermark dependence survives at each position. Under an asymptotic regime indexed by exponents for signal sparsity, next-token concentration, and effective-vocabulary growth, we derive a sharp boundary for global detection and phase transitions for discovery and classification within the class of coordinatewise pivot-based localization rules. We show that discovery is strictly harder than detection and that consistent classification is impossible across the parameter regime within this class. We then develop an adaptive thresholding method that does not require knowledge of the exponents or time-varying next-token distributions, but uses a data-driven estimate of the surviving watermark fraction. The method attains the optimal discovery boundary and near-optimal discovery power relative to homogeneous pivot-based rules. Simulations support the theoretical phase transitions, while experiments on model-generated texts demonstrate practical localization performance under common edit mechanisms.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-09-23
The authors' main claim, as the classifier picked it out 0.81
“The method attains the optimal discovery boundary and near-optimal discovery power relative to homogeneous pivot-based rules.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.08 · states limitation 0.73 · human participants 0.03 · controlled comparison 0.17 · randomized assignment 0.01 · states quantitative result 0.56 · compares with prior results 0.82 · mentions code or data availability 0.02
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Watermarking provides a principled way to authenticate text generated by large language models (LLMs). | background 1.00 | 0.06 | 0.02 | asserted 1.00 |
| In practice, however, the final text may be mixed-source, with watermark evidence surviving at only a subset of token positions after rewriting, insertion, deletion, or paraphrasing. | background 0.98 | 0.09 | 0.02 | hedged possibility 0.89 |
| Although prior work has studied global detection of watermark signals, when such signals can be localized remains unclear. | gap 1.00 | 0.05 | 0.03 | asserted 0.74 |
| We formulate watermark localization as a token-level multiple-testing problem based on pivotal statistics, with a latent indicator recording whether watermark dependence survives at each position. | method 0.69 | 0.93 | 0.06 | asserted 0.99 |
| Under an asymptotic regime indexed by exponents for signal sparsity, next-token concentration, and effective-vocabulary growth, we derive a sharp boundary for global detection and phase transitions for discovery and classification within the class of coordinatewise pivot-based localization rules. | result 0.71 | 0.95 | 0.16 | asserted 1.00 |
| We show that discovery is strictly harder than detection and that consistent classification is impossible across the parameter regime within this class. | result 0.99 | 0.94 | 0.28 | asserted 1.00 |
| We then develop an adaptive thresholding method that does not require knowledge of the exponents or time-varying next-token distributions, but uses a data-driven estimate of the surviving watermark fraction. | method 0.87 | 0.96 | 0.10 | asserted 1.00 |
| The method attains the optimal discovery boundary and near-optimal discovery power relative to homogeneous pivot-based rules. | result 0.99 | 0.96 | 0.29 | asserted 1.00 |
| Simulations support the theoretical phase transitions, while experiments on model-generated texts demonstrate practical localization performance under common edit mechanisms. | result 0.92 | 0.95 | 0.13 | asserted 1.00 |
What kind of paper the classifier reports this is: theoretical result 0.61 · evidence from proof or derivation 0.98
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 6,916 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (13)
Figures are the authors', reproduced from the paper.
Cited by 5 Pith papers
Reviewed papers on Pith that cite this one. The full citation record
- Prevalence Bounds for Edited Watermarked Text: Power Envelopes and Calibration Sensitivity stat.ML · Nov 2026
- CountScan: Simultaneous Lower Bounds on Historical Watermark Matches over All Text Intervals stat.ME · Nov 2026
- Token-Level Attribution of Multi-Mode LLM Watermarks: Optimal Rule and Belief Weighting cs.CR · Oct 2026
- textGrain watermarking is harder to detect: can sum-tests still do it optimally? stat.ML · Oct 2026
- FissionGuard: Testing Local Watermark Proportions with FDR Control stat.ME · Oct 2026
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 6 sentences without a confident label
Builds on 1 paper
-
To model the mixed-source text induced by human edits, we refine the mixture model of [ref] to the token level.
Optimal Estimation of Watermark Proportions in Hybrid AI-Human Texts · Our Contributions · 0.99
Uses the method or tool of 2 papers
-
To better understand this gap, we evaluate the accuracy of the fraction estimator $ε_n$ from [ref] .
Optimal Estimation of Watermark Proportions in Hybrid AI-Human Texts · Localization Performance · 0.82
-
For each document, we use the last $50$ tokens as the prompt and ask OPT-1.3B [ref] to generate an additional $n=400$ tokens as the continuation.
OPT: Open Pre-trained Transformer Language Models · Experiment Setup · 0.92
Cites as background 5 papers
-
[ref] studies global detection under a mixture model for edited text, and [ref] estimates the remaining watermark proportion in hybrid AI–human text.
Optimal Estimation of Watermark Proportions in Hybrid AI-Human Texts · Related Work · 0.90
-
More broadly, LLM watermarking has also been studied through biased [ref] or distribution-preserving watermark design [ref] , robustness-oriented design [ref] , and statistical detection [ref] .
2602.17608 · Related Work · 1.00
-
Large language models (LLMs) have recently become a powerful technology for generating human-like text and other media [ref] .
OpenAI GPT-5 System Card · Introduction · 1.00
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment