Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Enhancing Watermarking Quality for LLMs via Contextual Generation States Awareness

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A plug-and-play watermarking framework that adapts watermark strength to each token's contextual importance can improve both detection rate and output quality.

desk verdict Useful plug-and-play idea for watermark quality preservation, but the evaluator's training story is thin and the gains may partly reflect a learned softness schedule. read the letter →

arxiv 2506.07403 v1 pith:G7WKSFEX submitted 2025-06-09 cs.CR

Peiru Yang , Xintian Li , Wanchun Ni , Jinhua Yin , Huili Wang , Guoshun Nan , Shangguang Wang , Yongfeng Huang
show 1 more author
Tao Qi
This is my paper
classification cs.CR
keywords LLMwatermarkingcontextualgenerationstateswatermarkcapacityevaluatoradaptivestrengthtreeattentionqualityAIGTdetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that LLM watermarking can degrade generated text because existing methods decide where and how strongly to watermark using heuristics such as token confidence or entropy, which do not capture true semantic importance. It introduces a contextual generation states-aware watermarking framework, CAW, that first predicts each token's watermark capacity from the probability distributions of the token and its immediate context, then leaves important tokens untouched and scales watermark strength for the rest. A tree-attention pre-generation mechanism makes this adaptive strategy fast enough for practical use. The result, evaluated on reasoning and knowledge benchmarks across four model variants, is a better trade-off between detectability and task accuracy than hard, soft, and entropy-based watermarking baselines.

What carries the argument

The key mechanism is the watermark capacity evaluator, a three-layer fully connected network that takes the ranked top-100 probability values of $p(t_{i-1})$, $p(t_i)$, and $p(t_{i+1}\,|\,t_i')$ as input and outputs a capacity score $\hat{C}_i \in (0,1)$: $\hat{C}_i = E([p(t_{i+k})]_{k=-N^-}^{N^+}; \Theta^*)$ with $N^-=N^+=1$. The network is trained with cross-entropy on segments marked by a strong external judge model as quality-critical. Tokens with $\hat{C}_i > \theta$ are protected from watermarking; for other tokens the watermark strength is scaled as $K' = \beta(\theta-\hat{C}_i)/\theta\,K$ for sampling-based methods and $\delta' = \beta(\theta-\hat{C}_i)/\theta\,\delta$ for reweighting methods. To avoid the latency of looking ahead at future tokens, the framework uses tree attention to pre-generate all candidate next-token distributions in a single forward pass, so the adaptive decision comes at only a small overhead.

What would settle it

Take a task where the token that determines correctness is low-probability and isolated, such as a single digit in a long arithmetic chain, and compare CAW's accuracy-detectability frontier against the hard baseline at matched detection rate; if CAW's accuracy advantage over the entropy-based plugin disappears or reverses when the evaluator is trained on a different token-importance distribution, the claim of general quality preservation would be falsified.

Watch

Extended reading notes

Core claim

The central claim is that watermark-induced quality loss can be minimized by making the watermark injection itself aware of contextual generation states. The paper defines watermark capacity as a token's semantic importance and tolerance to perturbation, learns an evaluator that predicts this capacity from the probability distributions of the previous, current, and next token positions, and then adaptively watermarks: tokens predicted to be critical are generated without watermarking, while tokens with spare capacity receive watermark strength scaled to that capacity. Integrated with three base watermarking methods (KGW, Unigram, and EXP), CAW achieves higher task accuracy at comparable detection rates on GSM8K and MMLU across Llama2 and Vicuna models at 7B and 13B scales. The paper also argues that perplexity is an inadequate quality metric because watermarking can change a critical digit or fact while leaving fluency intact, and therefore uses task-level accuracy as the quality measure.

Load-bearing premise

The load-bearing premise is that the training labels produced by the external judge model on the training prompts correctly identify the tokens that must stay untouched for the answer to remain correct, and that this mapping carries over to unseen prompts and models.

Editorial extensions

If this is right

  • Existing single-bit watermarking methods can be wrapped by CAW without changing their detection procedure, gaining accuracy at matched detectability; the paper demonstrates this for KGW, Unigram, and EXP.
  • Perplexity-based claims of quality preservation become suspect, because task-level accuracy is what reveals watermark harm in reasoning and knowledge tasks.
  • The multi-branch pre-generation mechanism keeps added latency below about 15% and memory overhead within roughly 5-10%, so adaptive watermarking remains practical.
  • On GSM8K, CAW reaches accuracy close to the soft baseline while holding detection rates near the hard baseline, indicating users do not have to choose between detectability and answer correctness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves the capacity evaluator tied to its two evaluation tasks; a natural extension is to test whether the same evaluator transfers to code, dialogue, or other domains whose critical tokens differ sharply.
  • If the learned capacity scores track human-judged informational importance, the evaluator could serve as a reusable token-importance signal beyond watermarking, though this is not examined.
  • A testable extension is training the evaluator on multi-bit or sentence-level watermarks, where the strength variable is richer than a single scalar and the capacity prediction may need to be aggregated over longer spans.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript proposes CAW, a plug-and-play watermarking-quality-preservation framework for LLMs. CAW trains a three-layer neural 'watermark capacity evaluator' that takes the probability distributions of the current token and its immediate left and right contexts as input and predicts whether the token is quality-critical; the prediction is then used to decide whether to watermark the position and how strongly, using a linear strength mapping with threshold θ and scaling coefficient β. For sampling-based watermarks the top-K pool is narrowed; for reweighting watermarks the logit bias δ is scaled. To avoid the latency of the one-step lookahead, CAW pre-generates all candidate branches in a single tree-attention forward pass. Experiments integrate CAW with KGW, Unigram, and EXP on GSM8K and MMLU sociology for Llama2/Vicuna-7B/13B, comparing against hard/soft baselines and an entropy-based plugin on accuracy, AUROC, and F1, plus robustness and latency analyses.

Significance. If the mechanism works as described, CAW is a practically useful contribution: it is model-agnostic, applies to both reweighting- and sampling-based watermark families, and explicitly targets task accuracy rather than perplexity when measuring quality degradation. The tree-attention pre-generation mechanism is a sensible way to bound the latency cost of contextual lookahead, and the paper makes an honest case that perplexity is insufficient for watermark-quality evaluation. However, the evidence as presented does not yet establish that the contextual capacity evaluator is the source of the gains: the GPT-4o labels are not validated against the counterfactual quantity of interest, and the experimental comparisons lack variance information despite repeated runs.

major comments (4)
  1. [§3.2, Eq. (1), and Appendix C] The ground truth used to train the capacity evaluator is not shown to measure the quantity the method actually needs. The evaluator is trained on GPT-4o annotations of 'quality-critical segments' in human-written reference answers, while the central mechanism requires these labels to predict the counterfactual task-accuracy damage that watermarking causes at each token of model-generated chain-of-thought responses. Figure 3 validates the evaluator only against the GPT-4o labels; there is no experiment linking those labels, or the learned evaluator, to answer flips on GSM8K or MMLU. If the labels are noisy or task-specific, the improvements in Table 1 could come from the θ/β strength schedule rather than from contextual state awareness. Please add a direct validation (e.g., oracle capacity scores obtained by perturbing individual tokens and measuring answer changes, or an ablation with random capacity scores), and report the training-data provenance: source datasets, split, number of examples, the GPT-4o annotation prompt, and agreement statistics.
  2. [Table 1 and Figure 6] The central claim that CAW 'consistently outperforms' baselines is made without variance information, even though Appendix F states that every setup was run five times. Several reported differences are small relative to the plausible run-to-run noise (e.g., GSM8K Llama2-7b: KGW+ours accuracy 0.1105 vs KGW-Soft 0.1140; EXP+ours 0.1084 vs EXP+entropy 0.1060), and AUC/F1 differences between CAW and hard baselines are often under 0.03. Please report standard deviations or confidence intervals and, where the same generated outputs are compared, paired significance tests; without these, 'consistently' and the Pareto-dominance statements are not supported.
  3. [§4.3 and Figure 6] The claim of 'full Pareto optimality' is based on only three manually selected strength settings for the baselines, with no error bars and no specification of how the soft/mid/hard tiers were calibrated across methods. If the baseline points are placed at unfavorable strengths, CAW can appear Pareto-dominant without providing a better algorithmic trade-off. Please provide denser trade-off curves, a transparent strength-calibration rule, and variance bands, or restrict the claim to the specific configurations tested.
  4. [§4.2/§4.3 and Table 1] The comparison does not isolate the contribution of the contextual capacity evaluator from the effect of simply applying a softer watermark. CAW is free to choose a per-token strength schedule, while the baselines are fixed hard/soft strengths; an ablation that matches the average watermark strength, green-token rate, or detection F1 of the baselines is needed to show that the per-token contextual selection, not just a lower effective embedding rate, drives the accuracy gains. The entropy-based plugin is the right kind of control, but it is also a heuristic threshold; a randomized-capacity control with the same θ/β schedule would strengthen the causal claim.
minor comments (6)
  1. [Eq. (5)] The training objective as written is not a valid cross-entropy loss: it uses \(\hat C_i\) both as the ground-truth label and inside \(\log(E(p(t_i); \Theta))\), so the expression collapses to \(\log(E(\cdot))\) regardless of the label. Please rename the label (e.g., \(y_i\)) and write the standard binary cross-entropy form \(y_i \log(E) + (1-y_i) \log(1-E)\).
  2. [§3.2] The term 'watermark capacity' is used oppositely to its usual meaning: tokens with higher predicted capacity are left unmodified, while tokens with lower capacity receive the watermark. This will confuse readers; consider renaming the quantity to something like 'importance' or 'sensitivity', or explicitly define the inverse relationship more carefully.
  3. [§3.1, Step 4, and Eq. (1)] It should be stated explicitly that the \(p(t_{i+1})\) appearing in the evaluator input is \(p(t_{i+1} \mid t_i'^{(ori)})\), the distribution conditioned on the original candidate token, not on the eventual selected token. As written, the notation \([p(t_{i+k})]\) in Eq. (1) is ambiguous about which candidate branch is used.
  4. [Appendix D] The definitions \(K' = \beta (\theta - \hat C_i)/\theta \cdot K\) and \(\delta' = \beta (\theta - \hat C_i)/\theta \cdot \delta\) introduce \(\beta\) and \(\theta\) as empirical parameters, but no values or tuning ranges are reported anywhere in the paper; please provide these values or state how they were selected.
  5. [Figure 8] The text states that memory overhead remains within 5% for KGW and UNI and around 10% for EXP, but these numbers are not directly readable from the figure; please add explicit labels or a small table with the normalized values and their standard deviations.
  6. [§4.4] The robustness section says each data point is the mean of five repeated trials with variance, but only means are plotted; please add error bars or explicitly state that the variance is omitted for visual clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CAW's capacity evaluator is a learned component trained on GPT-4o-derived labels, but the reported accuracy and detection metrics are empirically measured downstream and are not entailed by the training objective.

full rationale

The paper's central claim is that CAW improves the watermarking detection/quality trade-off. The capacity evaluator in Section 3.2, Eq. (1), is trained in Eq. (5) to match GPT-4o annotations of quality-critical segments from human reference responses (Appendix C). This is a fitted classifier, but it is not used to predict the reported evaluation quantities: task accuracy on GSM8K/MMLU and AUROC/F1 are measured after watermark generation on Llama2 and Vicuna outputs. No equation or definition identifies the GPT-4o labels with the final accuracy metric, so the quality gains are not forced by construction. There is no load-bearing self-citation chain: tree attention is cited to an external system (SpecInfer, Ref. [14]) and all base watermarking methods are external. The main weakness is construct validity, namely whether GPT-4o labels of human-written references approximate counterfactual watermark damage, and the paper's Appendix G limitation about generalizing beyond the tested benchmarks is an empirical scope concern, not a circular derivation. The derivation chain from evaluator to adaptive strength to measured accuracy is therefore self-contained and not circular in the sense defined here.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central result rests on a supervised capacity model rather than a derivation. The evaluator weights, threshold theta, scaling beta, and per-benchmark strength settings are fitted or tuned, and four domain assumptions carry the transfer from GPT-4o labels and contextual probabilities to quality preservation. No new physical entities are introduced.

free parameters (5)
  • Evaluator weights Theta* = Trained on GPT-4o-labeled critical tokens, values not released
    The capacity prediction is the output of a three-layer network fitted to a supervised objective in Equation (5), so the central quality-preservation mechanism depends on these fitted weights.
  • Threshold theta = Not reported
    Appendix D says theta is empirically determined; it controls which tokens are protected from watermarking, and the quality versus detection trade-off depends directly on its value.
  • Scaling coefficient beta = Not reported
    Beta scales the reduced top-K or delta in Appendix D, but no value or selection procedure is given, leaving the adaptive strength mapping underspecified.
  • Watermark strength hyperparameters (delta, top-K) = Tuned per model and dataset
    Baselines and CAW adjust these hyperparameters to build the trade-off curves in Figure 6; tuning on the test benchmarks is not excluded.
  • Context window sizes N- = 1, N+ = 1 = 1 and 1
    The window size is chosen from preliminary experiments as the best latency and accuracy trade-off, so it is a design choice fitted to the evaluated task family.
assumptions (4)
  • domain assumption GPT-4o annotations of quality-critical segments in human-written answers are a valid ground truth for token-level watermark capacity.
    The evaluator's supervision and the paper's notion of critical tokens rest on this proxy, and it is not validated against human raters or a direct measure of task loss.
  • domain assumption The contextual probability distributions of the previous, current, and next token contain enough signal to predict where watermark perturbation hurts quality.
    The whole capacity evaluator is built on this input representation; preliminary experiments support it only within the tested models, with no transfer or robustness guarantee.
  • domain assumption Skipping or weakening watermarks on high-capacity tokens leaves the original detector statistically valid.
    CAW does not modify the detector or its null distribution, and the reported AUC and F1 scores assume the original z-statistic remains appropriate after non-watermarked tokens are interleaved.
  • standard math Tree attention with the sibling-suppression mask yields the same next-token distributions as sequential decoding for every candidate branch.
    This is inherited from SpecInfer and is needed so the pre-generated next distribution matches what sequential decoding would compute.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Watermarking Quality for LLMs via Contextual Generation States Awareness." pith.science (2026). https://pith.science/paper/G7WKSFEX

@misc{pith2026250607403,
  author       = {Pith},
  title        = {Pith review of: Enhancing Watermarking Quality for LLMs via Contextual Generation States Awareness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G7WKSFEX}},
  note         = {Machine review of arXiv:2506.07403}
}
read the original abstract

Recent advancements in watermarking techniques have enabled the embedding of secret messages into AI-generated text (AIGT), serving as an important mechanism for AIGT detection. Existing methods typically interfere with the generation processes of large language models (LLMs) to embed signals within the generated text. However, these methods often rely on heuristic rules, which can result in suboptimal token selection and a subsequent decline in the quality of the generated content. In this paper, we introduce a plug-and-play contextual generation states-aware watermarking framework (CAW) that dynamically adjusts the embedding process. It can be seamlessly integrated with various existing watermarking methods to enhance generation quality. First, CAW incorporates a watermarking capacity evaluator, which can assess the impact of embedding messages at different token positions by analyzing the contextual generation states. Furthermore, we introduce a multi-branch pre-generation mechanism to avoid the latency caused by the proposed watermarking strategy. Building on this, CAW can dynamically adjust the watermarking process based on the evaluated watermark capacity of each token, thereby minimizing potential degradation in content quality. Extensive experiments conducted on datasets across multiple domains have verified the effectiveness of our method, demonstrating superior performance compared to various baselines in terms of both detection rate and generation quality.

Figures

Figures reproduced from arXiv: 2506.07403 by the authors.

Figure 1
Figure 1. Comparison between watermarked and unwatermarked outputs on Llama2-13b with [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Architectural overview of CAW, highlighting its plug-and-play integration capability. CAW [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison of watermark capacity modeling methods (Left: Llama2-13B; Right: Vicuna [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Details of tree attention decoding. (a) Sequential attention for each t3 in t ′ 3 requires P + 1 computations. (b) Tree attention mask enables a single computation for all candidates by zeroing others’ mask values. We use Llama2 and Vicuna-v1.5 models (7B and 13B) to e…
Figure 5
Figure 5. Figure 5: Box plots of perplexity and accuracy for non-watermarked text, baselines (KGW, UNI, EXP), [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Pareto front comparisons of CAW and baselines across Llama2 and Vicuna-v1.5 (7B/13B), [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Detection rate loss under three attacks on watermarked Llama2-13B and Vicuna-v1.5-13B, [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Time and memory overhead of different CAW implementations (tree attention, sequential, batch), normalized to the non-watermarked baseline. Tree attention shows clear efficiency advantages. As watermarking algorithms are embedded in LLM inference, minimizing their impac…
Figure 9
Figure 9. Figure 9: The structure of the contextual generation states-aware watermark capacity evaluator, [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Marking the Wrong Symptoms: Evaluating LLM Watermarks in Medical Texts

    cs.AI 2026-05 conditional novelty 6.0 of 10

    Watermarking medical AI outputs can degrade reasoning, terminology, and image interpretation even when benchmark accuracy stays stable, so accuracy-only evaluations hide clinically important damage.

Reference graph

Works this paper leans on

36 extracted references · 19 canonical work pages · cited by 1 Pith paper

  1. [1]

    Academic integrity considerations of ai large language models in the post- pandemic era: Chatgpt and beyond.Journal of University Teaching and Learning Practice, 20 (2), 2023

    Mike Perkins. Academic integrity considerations of ai large language models in the post- pandemic era: Chatgpt and beyond.Journal of University Teaching and Learning Practice, 20 (2), 2023

    work page 2023

  2. [2]

    Copyright protection in generative ai: A technical perspective.arXiv:2402.02333, 2024

    Jie Ren, Han Xu, Pengfei He, Yingqian Cui, Shenglai Zeng, Jiankun Zhang, Hongzhi Wen, Jiayuan Ding, Hui Liu, Yi Chang, et al. Copyright protection in generative ai: A technical perspective.arXiv:2402.02333, 2024

    arXiv 2024

  3. [3]

    The ethical need for watermarks in machine- generated language.arXiv:2209.03118, 2022

    Alexei Grinbaum and Laurynas Adomaitis. The ethical need for watermarks in machine- generated language.arXiv:2209.03118, 2022

    arXiv 2022

  4. [4]

    Building Intelligence Identification System via Large Language Model Watermarking: A Survey and Beyond

    Xuhong Wang, Haoyu Jiang, Yi Yu, Jingru Yu, Yilun Lin, Ping Yi, Yingchun Wang, Qiao Yu, Li Li, and Fei-Yue Wang. Building intelligence identification system via large language model watermarking: A survey and beyond.arXiv:2407.11100, 2024

    work page Pith review arXiv 2024

  5. [5]

    A Watermark for Large Language Models

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A Watermark for Large Language Models. InICML, pages 17061–17084, 2023

    work page 2023

  6. [6]

    Watermarking gpt outputs, 2022

    Scott Aaronson and Hendrik Kirchner. Watermarking gpt outputs, 2022. https://www. scottaaronson.com/talks/watermark.ppt

    work page 2022

  7. [7]

    Provable robust watermarking for ai-generated text.arXiv:2306.17439, 2023

    Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu-Xiang Wang. Provable robust watermarking for ai-generated text.arXiv:2306.17439, 2023

    arXiv 2023

  8. [8]

    Who wrote this code? watermarking for code generation.arXiv:2305.15060, 2023

    Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim. Who wrote this code? watermarking for code generation.arXiv:2305.15060, 2023

    arXiv 2023

Show all 36 references
  1. [9]

    An entropy-based text water- marking detection method.arXiv:2403.13485, 2024

    Yijian Lu, Aiwei Liu, Dianzhi Yu, Jingjing Li, and Irwin King. An entropy-based text water- marking detection method.arXiv:2403.13485, 2024

    2024 arXiv

  2. [10]

    Optimizing Watermarks for Large Language Models

    Bram Wouters. Optimizing Watermarks for Large Language Models. InICML, pages 53251– 53269, 2024

    2024

  3. [11]

    Adaptive text watermark for large language models

    Yepeng Liu and Yuheng Bu. Adaptive text watermark for large language models. arXiv:2401.13927, 2024

    2024 arXiv

  4. [12]

    Unbiased watermark for large language models.arXiv:2310.10669, 2023

    Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. Unbiased watermark for large language models.arXiv:2310.10669, 2023

    2023 arXiv

  5. [13]

    A Resilient and Accessible Distribution-Preserving Watermark for Large Language Models

    Yihan Wu, Zhengmian Hu, Junfeng Guo, Hongyang Zhang, and Heng Huang. A Resilient and Accessible Distribution-Preserving Watermark for Large Language Models. InICML, pages 53443–53470, 2024

    2024

  6. [14]

    SpecInfer: Accelerating Large Language Model Serving with Tree-Based Speculative Inference and Verification

    Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Zhengxin Zhang, Rae Ying Yee Wong, Alan Zhu, Lijie Yang, Xiaoxiang Shi, et al. SpecInfer: Accelerating Large Language Model Serving with Tree-Based Speculative Inference and Verification. InASPLOS, pages 932–...

    2024

  7. [15]

    An Unforgeable Publicly Verifiable Watermark for Large Language Models

    Aiwei Liu, Leyi Pan, Xuming Hu, Shuang Li, Lijie Wen, Irwin King, and S Yu Philip. An Unforgeable Publicly Verifiable Watermark for Large Language Models. InICLR, 2023

    2023

  8. [16]

    Towards Codable Watermarking for Injecting Multi-Bits Information to LLMs

    Lean Wang, Wenkai Yang, Deli Chen, Hao Zhou, Yankai Lin, Fandong Meng, Jie Zhou, and Xu Sun. Towards Codable Watermarking for Injecting Multi-Bits Information to LLMs. In ICLR, 2024

    2024

  9. [17]

    Advancing Beyond Identification: Multi-bit Watermark for Large Language Models

    KiYoon Yoo, Wonhyuk Ahn, and Nojun Kwak. Advancing Beyond Identification: Multi-bit Watermark for Large Language Models. InNAACL, pages 4031–4055, 2024. 10

    2024

  10. [18]

    Codeip: A grammar-guided multi-bit watermark for large language models of code

    Batu Guan, Yao Wan, Zhangqian Bi, Zheng Wang, Hongyu Zhang, Pan Zhou, and Lichao Sun. Codeip: A grammar-guided multi-bit watermark for large language models of code. arXiv:2404.15639, 2024

    2024 arXiv

  11. [19]

    A robust semantics-based watermark for large language model against paraphrasing

    Jie Ren, Han Xu, Yiding Liu, Yingqian Cui, Shuaiqiang Wang, Dawei Yin, and Jiliang Tang. A robust semantics-based watermark for large language model against paraphrasing. arXiv:2311.08721, 2023

    2023 arXiv

  12. [20]

    REMARK- LLM: A Robust and Efficient Watermarking Framework for Generative Large Language Models

    Ruisi Zhang, Shehzeen Samarah Hussain, Paarth Neekhara, and Farinaz Koushanfar. REMARK- LLM: A Robust and Efficient Watermarking Framework for Generative Large Language Models. InUSENIX Security, pages 1813–1830, 2024

    2024

  13. [21]

    Robust distortion- free watermarks for language models.arXiv:2307.15593, 2023

    Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortion- free watermarks for language models.arXiv:2307.15593, 2023

    2023 arXiv

  14. [22]

    Undetectable Watermarks for Language Models

    Miranda Christ, Sam Gunn, and Or Zamir. Undetectable Watermarks for Language Models. In COLT, pages 1125–1139, 2024

    2024

  15. [23]

    GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick

    Jiayi Fu, Xuandong Zhao, Ruihan Yang, Yuansen Zhang, Jiangjie Chen, and Yanghua Xiao. GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick. InACL, pages 5791–5808, 2024

    2024

  16. [24]

    Scalable watermarking for identifying large language model outputs.Nature, 634(8035):818–823, 2024

    Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, et al. Scalable watermarking for identifying large language model outputs.Nature, 634(8035):818–823, 2024

    2024

  17. [25]

    Duwak: Dual Watermarks in Large Language Models

    Chaoyi Zhu, Jeroen Galjaard, Pin-Yu Chen, and Lydia Chen. Duwak: Dual Watermarks in Large Language Models. InACL Findings, pages 11416–11436, 2024

    2024

  18. [26]

    Watermarking Conditional Text Generation for AI Detection: Unveiling Challenges and a Semantic-Aware Watermark Remedy

    Yu Fu, Deyi Xiong, and Yue Dong. Watermarking Conditional Text Generation for AI Detection: Unveiling Challenges and a Semantic-Aware Watermark Remedy. InAAAI, pages 18003–18011, 2024

    2024

  19. [27]

    WatME: Towards Lossless Watermarking Through Lexical Redundancy

    Liang Chen, Yatao Bian, Yang Deng, Deng Cai, Shuaiyi Li, Peilin Zhao, and Kam-Fai Wong. WatME: Towards Lossless Watermarking Through Lexical Redundancy. InACL, pages 9166– 9180, 2024

    2024

  20. [28]

    Markllm: An open-source toolkit for llm watermarking

    Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, et al. Markllm: An open-source toolkit for llm watermarking. arXiv:2405.10051, 2024

    2024 arXiv

  21. [29]

    Attention is all you need.Advances in Neural Information Processing Systems, 2017

    A Vaswani. Attention is all you need.Advances in Neural Information Processing Systems, 2017

    2017

  22. [30]

    What Language Model Architecture and Pretraining Objective Works Best for Zero-Shot Generalization? InICML, pages 22964–22984, 2022

    Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay, and Colin Raffel. What Language Model Architecture and Pretraining Objective Works Best for Zero-Shot Generalization? InICML, pages 22964–22984, 2022

    2022

  23. [31]

    Llama: Open and efficient foundation language models.arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv:2302.13971, 2023

    2023 arXiv

  24. [32]

    Vicuna: An open-source chatbot impressing gpt-4 with 90% chatgpt quality, 2023.https://vicuna.lmsys.org

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90% chatgpt quality, 2023.https://vicuna.lmsys.org

    2023

  25. [33]

    Measuring massive multitask language understanding.arXiv:2009.03300, 2020

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding.arXiv:2009.03300, 2020

    2009 arXiv

  26. [34]

    Training verifiers to solve math word problems.arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv:2110.14168, 2021. 11

    arXiv

  27. [35]

    Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research, 21(140):1–67, 2020

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research, 21(140):1–67, 2020

    2020

  28. [36]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022. 12 A Further Discussion on Rel...

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.