Pith. sign in

REVIEW 6 major objections 6 minor 22 references

Let Watermarks Speak: A Robust and Unforgeable Watermark for Language Models

T0 review · 6 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read This paper claims the first undetectable, robust, single-bit watermark for language models, and chains hash-linked blocks so that prompt forgery and tampering are verifiable.

desk verdict A neat bit-level watermarking idea, but the formal write-up is broken at the sampler definition and detector equations, and the experiments are absent. read the letter →

arxiv 2412.19603 v1 pith:3ECOUWTT submitted 2024-12-27 cs.CR

classification cs.CR
keywords languagemodelwatermarkingsingle-bitwatermarkunforgeabilitycomputationalindistinguishabilitydualinversetransformsamplinghashchainrobustnessforgeryattack
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims to be the first to build an undetectable, robust, single-bit watermark for language models, and then a multi-bit unforgeable chain on top of it. The single-bit trick is a sampling rule that hides a 0 or 1 payload without changing the model's output distribution, so a detector can read the bit back from any sufficiently long block while an adversary cannot tell watermarked from unwatermarked text. The multi-bit construction turns the hash of the prompt and of previously generated blocks into the watermark payloads, so changing the prompt or editing the text breaks a verifiable hash chain. If the construction works as claimed, model providers could answer both “is this our output?” and “under which prompt?” at the same time, closing both prompt-misattribution and robustness-exploiting forgery attacks. The cost the paper accepts is that correctness relies on the bit-level predictor keeping a non-negligible minimum probability on enough positions.

What carries the argument

The central object is the Dual Inverse Transform Sampling (DITS) rule, which replaces the model's sampler. For a next-bit distribution $(p(0), p(1))$ and a shared pseudorandom $r \in [0,1]$, DITS outputs a bit by checking which side of the predictor's threshold $r$ falls on, taking the complementary arrangement when the watermark bit is $1$; the detector reads each bit as $\hat{m}' = \mathbf{1}(r < \frac{1}{2}) \oplus b$ and aggregates the votes over a block. DITS does the work of making the watermark invisible: the marginal distribution of the output bit equals the predictor's distribution regardless of $m$, so the watermarked model is computationally indistinguishable from the original. Aggregation does the work of making the watermark reliable and robust: the vote mean concentrates around $\frac{1}{2} \pm \mathrm{gap}$, and a majority of flipped votes can be tolerated up to a Hamming radius. The multi-bit scheme adds a hash chain called a watermark link, where the payload of each block is one bit of the hash of the prompt or of the preceding content, and Verify recomputes the chain; this is the mechanism that turns robustness into unforgeability.

What would settle it

Measure $\min\{p(0),p(1)\}$ from a real token-level language model on ordinary prompts, compute the block length $N$ needed to make $\exp\!\left(-2N\,\mathrm{gap}^2\right)$ negligible where $\mathrm{gap}=\frac{1}{N}\sum_i \min\{p_i(0),p_i(1)\}$, and check whether that $N$ fits inside a realistic model response; if the measured gap is so small that $N$ exceeds practical response lengths, the central claim of practical effectiveness is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that a one-bit watermark can be both statistically undetectable and robust, and that chaining such bits makes the watermark prefix-unforgeable. The key identity is the pair of complementary sampling and detection rules: a bit is generated by comparing a shared pseudorandom number $r$ against the predictor's thresholds under one arrangement for watermark bit $m=0$ and the mirror arrangement for $m=1$, and it is recovered by testing whether $r$ falls below $\frac{1}{2}$ and XORing the result with the observed bit. Because the sample mean of these per-bit votes sits at $\frac{1}{2} \pm \frac{1}{N}\sum_i \min\{p_i(0), p_i(1)\}$, the three states $0$, $1$, and $\bot$ (no watermark) are separated by a gap that Hoeffding's inequality makes overwhelming once enough bits are gathered. Repeating this within a block yields one watermark signal, and using the bits of $\mathrm{Hash}(\text{prompt} \| \text{previous block})$ as the signals for successive blocks gives a chain whose first link binds the text to its actual prompt and whose later links expose modification. The paper proves correctness, computational undetectability, and substitution robustness for the single-bit scheme, and prefix-unforgeability for the multi-bit chain under collision resistance of the hash.

Load-bearing premise

The whole construction stands on the assumption that the model's bit-level next-bit predictor keeps both bit values reasonably likely over enough generated positions; if real token-level models are often near-deterministic at the bit level, the statistical gap between watermarked and unwatermarked output shrinks and the block length needed becomes impractical.

Editorial extensions

If this is right

  • A detector can now distinguish three states from a watermarked text: signal 0, signal 1, and no signal, so a single generated response can carry a meaningful payload rather than only a yes/no flag.
  • Because the first watermark link carries the hash of the actual prompt, claiming a watermarked output was produced under a different prompt fails verification with overwhelming probability.
  • Because later links carry hashes of the preceding content, any substitution that flips enough bits to change the content is exposed when Verify recomputes the chain, even though the watermark signal itself is still detectable.
  • Watermarked text remains computationally indistinguishable from ordinary model output, so the scheme does not degrade text quality or change the sampling distribution.
  • The scheme's substitution robustness is proportional to the square root of the block length, so longer responses tolerate more edits before detection fails.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same DITS construction could be applied per token rather than per bit by encoding token IDs into bit lists, but the effective minimum probability would then be governed by the least likely prefix across the vocabulary, which may be far smaller than the bit-level values the paper assumes; measuring this on real models is the natural next check.
  • Beyond the paper, the robustness guarantee is stated only against substitution flips; deletions and insertions would desynchronize the shared randomness between embedder and detector, so an edit-robust extension would need an alignment or reset mechanism the paper does not provide.
  • Beyond the paper, the hash-chain verification is keyed and private; turning it into a publicly verifiable scheme would require a signature or zero-knowledge component, which the paper explicitly avoids because bit payloads are expensive, so a public-verifiability version may need a different trade-off.
  • Beyond the paper, prefix-unforgeability leaves the final open-ended link unchecked; a truncation attack that cuts the chain at a complete link boundary would still pass Verify, so the scheme detects modification only where a successor link exists.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 6 minor

Summary. The paper proposes a single-bit watermarking scheme for language models, claiming undetectability, robustness, and extensibility to a multi-bit unforgeable scheme that hashes the prompt and previous content into watermark links. The core construction is a 'Wat-Sampler' (Eq. 5) and a detector (Eq. 7), together with aggregation via Hoeffding bounds and a multi-bit chain built on hash values. The abstract also promises 'sufficient experiments' on popular language models. As written, however, the central sampler definition is self-referential and not computable, the detection formula is inconsistent with the paper's own figures and analysis, the decision rule in the algorithm is reversed relative to the derived expectations, the unforgeability proof is only a one-sentence appeal to collision resistance, and Section 5 contains no actual experiments or results.

Significance. The intended idea, a dual inverse-transform sampling that embeds a bit without changing the marginal output distribution, is a plausible and potentially useful building block for language-model watermarking. If the construction were correctly specified and the gaps in the analysis filled, the paper could be a valuable contribution to the undetectable- and unforgeable-watermark literature. However, the manuscript as submitted does not contain a well-defined central algorithm: Eq. (5) is not a sampling rule, and the detection rule is inverted with respect to the analysis. Because the formal claims are about an object different from the one actually defined, the current version does not establish its main results. The absence of any experimental content further weakens the paper's practical claims.

major comments (6)
  1. [§4.1, Eq. (5) and Algorithm 1] The definition b = 1(r < p(b)) ⊕ m is a fixed-point equation, not a computable sampling rule, because b appears on both sides. For p(0)=0.7, p(1)=0.3, m=0, and r=0.5, neither b=0 nor b=1 satisfies the equation, so the sampler is undefined on a positive-measure set of random inputs. The subsequent proofs (Claim 1, Figure 1, and the games in Figure 3) use a different, well-defined dual inverse-transform rule. The manuscript must replace Eq. (5) and Algorithm 1 with the explicitly sequential rule used in the proofs, or the central construction is absent.
  2. [§4.1, Eq. (7), Algorithm 2, and Figure 2] The detection formula m' = 1(r < 1/2) ⊕ b is the inverse of the rule that the paper's own analysis uses. With the construction in Figure 1, for p(0)=p(1)=1/2 and m=0, r<1/2 gives b=0, and Eq. (7) yields m'=1; the paper's text immediately after Eq. (7) claims the opposite. Figure 2 and Eqs. (8)-(13) are consistent with the rule m' = 1(r ≥ 1/2) ⊕ b, not with Eq. (7). This is not a typographical issue: the detection algorithm as written would output the wrong watermark bit.
  3. [§4.1, Eqs. (11)-(12) and Algorithm 4] Even if Eq. (7) were corrected to match Figure 2, the decision rule in Algorithm 4 and the bullet list in Section 4.1 is reversed. Equations (11) and (12) put E[X | m=0] below 1/2 and E[X | m=1] above 1/2, yet the algorithm outputs m=0 when X > 1/2 and m=1 when X < 1/2. The detector as specified would systematically misclassify the embedded bit.
  4. [§4.1 and §4.2 (Claim 2)] The stopping condition in Algorithm 3 relies on the empirical mean X deviating from 1/2 enough to make exp(-i(X-1/2)^2/2) negligible. The required block length depends on the gap δ = (1/N) Σ min{p_i(0), p_i(1)}. The paper only assumes δ > 0; it does not show that δ is non-negligible, nor that Embed terminates in expected polynomial time. For real token-level models converted to bits, min{p(0), p(1)} can be extremely small, and the necessary block length could be impractically large. The paper provides no measurement of this quantity, so the claimed practical effectiveness is unsupported.
  5. [§4.4, Claim 6] The proof of prefix-unforgeability consists of a single sentence citing collision resistance of the hash. The game in Definition 11 allows the adversary to modify the transcript arbitrarily, and the reduction is not established: an adversary need not find a hash collision, but could truncate, insert, or overwrite bits so that the decoded watermark bits pass Verify without any hash collision. The proof must explicitly model the decoding procedure, the robustness threshold, and the exact role of the hash, and show how any winning strategy leads to a collision or a second-preimage.
  6. [§5] The paper claims in the abstract to provide 'sufficient experiments' and that the results show the method is 'practically effective and robust,' but Section 5.1 ends after a single sentence ('We set the negl(λ) as e^{-λ}.') and no experiments, datasets, models, baselines, or result tables are included. The empirical claims are thus entirely unverified in the manuscript.
minor comments (6)
  1. [§2.2] 'gumble-softmax' should be 'Gumbel-softmax'.
  2. [§4.1, Lemma 1 title] 'Ineqality' should be 'Inequality'.
  3. [Algorithm 6, line 4] The condition 'if m_i ≠ ⊥' should be 'if m' ≠ ⊥', since the variable m_i is not defined at that point.
  4. [Definition 5 title] 'Single W atermark Output' should be 'Single Watermark Output'.
  5. [Figure 3] The pseudocode for G2 and G3 outputs 1 when r ≤ p(0), which is the inverse of the inverse-transform sampling described in the text; although the distribution is preserved up to relabeling, the code should match the intended mapping.
  6. [§3.2] The conversion from token-level language models to the bit-level Model is described only informally; Section 5.1 offers no additional implementation details despite its title.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the main derivation is not an input-output tautology, although the central sampler equation (5) is self-referential and ill-posed as written.

full rationale

The derivation chain is not circular in the sense of the rubric. The detector threshold 1/2 comes from the fixed symmetry of Detect-1bit (Eq. 7), not from a fitted parameter; block length is chosen adaptively by the embedder until a Hoeffding bound is negligible; and the entropy assumption on the Predictor is stated as an explicit assumption. Unforgeability of the multi-bit construction is reduced to collision resistance of Hash, an external standard assumption, and correctness and undetectability are inherited from the single-bit scheme rather than assumed. There are no load-bearing self-citations. What the manuscript does contain is a formal defect: Eq. (5), b = 1(r < p(b)) XOR m, defines the output bit in terms of p(b), so the right-hand side depends on the value being defined; for example, with p(0)=0.7, p(1)=0.3, m=0 and r=0.5, neither candidate bit satisfies the equation. The proof of Claim 1 instead uses the case split of Fig. 1, so the formal object supporting the central theorems is not well-defined as written. That is a correctness and well-definedness problem, not a circular derivation: the theorems are not made true by definition, and no result is rebuilt from its own conclusion. The experimental section is also missing its actual results, but that is a completeness issue.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The construction introduces no fitted parameters or new physical entities. It relies on standard cryptographic assumptions (PRG/PRF, CRHF), a modelling conversion from tokens to bits, and an unverified entropy assumption on the model's next-bit distribution. The absence of experiments means the practical validity of the entropy assumption is untested.

assumptions (5)
  • domain assumption PRG/PRF output is computationally indistinguishable from uniform random bits.
    Used in Claim 3 to show watermarked output is indistinguishable from model output; standard cryptographic assumption.
  • domain assumption Collision-resistant hash functions exist.
    Used in Claim 6 for prefix-unforgeability; standard cryptographic assumption.
  • domain assumption The model's next-bit distribution has non-negligible entropy, i.e., the Predictor does not always output deterministic distributions.
    Section 4.1: the gap in conditional expectations depends on min{p(0),p(1)}; if this is 0, embedding is impossible. This is load-bearing and unverified for real LMs.
  • domain assumption Token-level LMs can be converted to a bit-level Model by recursive token-bit sampling.
    Section 3.2: this conversion assumes each token ID can be expressed as a bit string and that the conditional distribution over bit prefixes is available.
  • standard math Hoeffding's inequality applies to the aggregated detector outputs X_i, which are treated as independent random variables.
    Lemma 1 used for correctness and robustness bounds; the independence of X_i across positions is assumed, and the paper does not address dependencies introduced by the autoregressive context.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Let Watermarks Speak: A Robust and Unforgeable Watermark for Language Models." pith.science (2026). https://pith.science/paper/3ECOUWTT

@misc{pith2026241219603,
  author       = {Pith},
  title        = {Pith review of: Let Watermarks Speak: A Robust and Unforgeable Watermark for Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3ECOUWTT}},
  note         = {Machine review of arXiv:2412.19603}
}
read the original abstract

Watermarking is an effective way to trace model-generated content. Current watermark methods cannot resist forgery attacks, such as a deceptive claim that the model-generated content is a response to a fabricated prompt. None of them can be made unforgeable without degrading robustness. Unforgeability demands that the watermarked output is not only detectable but also verifiable for integrity, indicating whether it has been modified. This underscores the necessity and significance of a multi-bit watermarking scheme. Recent works try to build multi-bit scheme based on existing zero-bit watermarking scheme, but they either degrades the robustness or brings a significant computational burden. We aim to design a novel single-bit watermark scheme, which provides the ability to embed 2 different watermark signals. This paper's main contribution is that we are the first to propose an undetectable, robust, single-bit watermarking scheme. It has a comparable robustness to the most advanced zero-bit watermarking schemes. Then we construct a multi-bit watermarking scheme to use the hash value of prompt or the newest generated content as the watermark signals, and embed them into the following content, which guarantees the unforgeability. Additionally, we provide sufficient experiments on some popular language models, while the other advanced methods with provable guarantees do not often provide. The results show that our method is practically effective and robust.

Figures

Figures reproduced from arXiv: 2412.19603 by the authors.

Figure 1
Figure 1. An illustration of Wat-Sampler and DITS. Sampling (DITS), and details are shown in [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. An illustration of Detect-1bit. correct watermark signals. Thus, aggregating the output of Detect￾1bit on multiple bits and making a decision bound are necessary. Assuming that Embed generates a watermarked bit string 𝑏, the distribution of each bit is (𝑝𝑖(0), 𝑝𝑖(1)),𝑖 ∈ {1, 2, · · · , |𝑏|} and the randomness used to sample each bit is 𝑟𝑖 . For the 𝑖-th bit 𝑏𝑖 , the probability that Detect-1bit outputs a wrong water… view at source ↗
Figure 3
Figure 3. Games used in the proof of computational indistin [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 20 canonical work pages

  1. [1]

    Simons institute talk on watermarking of large language models, 2023

    Scott Aaronson. Simons institute talk on watermarking of large language models, 2023

  2. [2]

    Fast- detectGPT: Efficient zero-shot detection of machine-generated text via condi- tional probability curvature

    Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. Fast- detectGPT: Efficient zero-shot detection of machine-generated text via condi- tional probability curvature. In The Twelfth International Conference on Learning Representations, 2024

  3. [3]

    Pseudorandom error-correcting codes

    Miranda Christ and Sam Gunn. Pseudorandom error-correcting codes. In Annual International Cryptology Conference, pages 325–347. Springer, 2024

  4. [4]

    Undetectable watermarks for language models

    Miranda Christ, Sam Gunn, and Or Zamir. Undetectable watermarks for language models. In The Thirty Seventh Annual Conference on Learning Theory, pages 1125–

  5. [5]

    Watermarking language models for many adaptive users

    Aloni Cohen, Alexander Hoover, and Gabe Schoenbach. Watermarking language models for many adaptive users. In 2025 IEEE Symposium on Security and Privacy (SP), pages 84–84. IEEE Computer Society, 2024

  6. [6]

    Publicly-detectable watermarking for language models, 2024

    Jaiden Fairoze, Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mah- moody, and Mingyuan Wang. Publicly-detectable watermarking for language models, 2024

  7. [7]

    Edit distance robust watermarks for language models, 2024

    Noah Golowich and Ankur Moitra. Edit distance robust watermarks for language models, 2024

  8. [8]

    Spotting LLMs with binoculars: Zero-shot detection of machine-generated text

    Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Spotting LLMs with binoculars: Zero-shot detection of machine-generated text. In Forty- first International Conference on Machine Learning , 2024

Show all 22 references
  1. [9]

    Carnegie Mellon University, 2004

    Nicholas J Hopper.Toward a theory of Steganography. Carnegie Mellon University, 2004

  2. [10]

    Radar: Robust ai-text detection via adversarial learning

    Xiaomeng Hu, Pin-Yu Chen, and Tsung-Yi Ho. Radar: Robust ai-text detection via adversarial learning. Advances in Neural Information Processing Systems , 36:15077–15095, 2023

  3. [11]

    Categorical reparametrization with gumble-softmax

    Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparametrization with gumble-softmax. In International Conference on Learning Representations (ICLR 2017). OpenReview. net, 2017

  4. [12]

    Jois, Matthew Green, and Aviel D

    Gabriel Kaptchuk, Tushar M. Jois, Matthew Green, and Aviel D. Rubin. Meteor: Cryptographically secure steganography for realistic distributions. InProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security , CCS ’21, page 1529–1548, New York, NY, USA...

  5. [13]

    A watermark for large language models

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In International Conference on Machine Learning, pages 17061–17084. PMLR, 2023

  6. [14]

    On the reliability of watermarks for large language models, 2024

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. On the reliability of watermarks for large language models, 2024

  7. [15]

    Ro- bust distortion-free watermarks for language models

    Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Ro- bust distortion-free watermarks for language models. Transactions on Machine Learning Research, 2024

  8. [16]

    Aiwei Liu, Leyi Pan, Xuming Hu, Shuang Li, Lijie Wen, Irwin King, and Philip S. Yu. An unforgeable publicly verifiable watermark for large language models. In The Twelfth International Conference on Learning Representations , 2024

  9. [17]

    Detectgpt: Zero-shot machine-generated text detection using probability curvature

    Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In International Conference on Machine Learning , pages 24950–24962. PMLR, 2023

  10. [18]

    Provably robust multi-bit watermarking for ai-generated text, 2024

    Wenjie Qu, Wengrui Zheng, Tianyang Tao, Dong Yin, Yanze Jiang, Zhihua Tian, Wei Zou, Jinyuan Jia, and Jiaheng Zhang. Provably robust multi-bit watermarking for ai-generated text, 2024

  11. [19]

    SeqXGPT: Sentence-level AI-generated text detection

    Pengyu Wang, Linyang Li, Ke Ren, Botian Jiang, Dong Zhang, and Xipeng Qiu. SeqXGPT: Sentence-level AI-generated text detection. In The 2023 Conference on Empirical Methods in Natural Language Processing , 2023

  12. [20]

    Zero-shot detection of machine-generated codes, 2023

    Xianjun Yang, Kexun Zhang, Haifeng Chen, Linda Petzold, William Yang Wang, and Wei Cheng. Zero-shot detection of machine-generated codes, 2023

  13. [21]

    Excuse me, sir? your language model is leaking (information)

    Or Zamir. Excuse me, sir? your language model is leaking (information). arXiv preprint arXiv:2401.10360, 2024

  14. [22]

    Prov- able robust watermarking for AI-generated text

    Xuandong Zhao, Prabhanjan Vijendra Ananth, Lei Li, and Yu-Xiang Wang. Prov- able robust watermarking for AI-generated text. In The Twelfth International Conference on Learning Representations , 2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.