Pith. sign in

REVIEW 3 major objections 5 minor 61 references

From Trade-off to Synergy: A Versatile Symbiotic Watermarking Framework for Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a per-token entropy gate lets logits-based and sampling-based watermarks reinforce each other, overcoming the usual trade-off between detectability and text quality.

desk verdict Entropy-driven hybrid watermarking is a real idea, but Algorithm 1 does not implement the claimed hybrid—a fixable but load-bearing inconsistency. read the letter →

arxiv 2505.09924 v2 pith:FW6US3G7 submitted 2025-05-15 cs.CL cs.CR

Yidan Wang , Yubing Ren , Yanan Cao , Binxing Fang This is my paper · ORCID
classification cs.CLcs.CR
keywords LLMwatermarkingsymbioticwatermarklogits-basedsampling-basedtokenentropysemanticsecurityrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the standard trade-off in LLM watermarking—stronger marks degrade text quality and invite attack—is not inevitable. It proposes SymMark, a framework that runs a logits-based watermark and a sampling-based watermark at the same time, and in its main 'Hybrid' configuration decides per token which of the two (or neither) to apply, using two entropy measures as the decision rule. The claim is that this adaptive combination reaches the best reported balance of detectability, resilience, text quality, and security across several models and datasets. If the claim holds, watermarking can be far less intrusive than the current best single-method baselines while staying more resistant to watermark-stealing and text-altering attacks.

What carries the argument

The central object is the entropy-gated hybrid watermarking rule. At each step $t$, the model's logits give a distribution $p_t$; token entropy $H^{TE} = -\sum_i p_i^t \log p_i^t$ gates the logits-based watermark, and semantic entropy $H^{SE}$ (computed as the entropy of the distribution merged across $n$ K-means clusters of the top-$k$ token embeddings) gates the sampling-based watermark. The rule 'apply logits watermark if $H^{TE} > \alpha$; apply sampling watermark if $H^{SE} < \beta$' is what carries the argument: it places the stronger, fluency-affecting mark only where the model is already uncertain, and the sampling mark only where swapping among semantically similar candidates will not change meaning. Detection is a simple OR of the two component detectors, applied to all tokens, so detection stays cheap and does not require knowing which tokens were watermarked.

What would settle it

Take a model not in the paper's set (e.g., a multilingual or code-specialised model), generate with the hybrid method using the paper's fixed thresholds, and measure detection F1 and perplexity against the same baselines; if the hybrid's F1 or AUC falls below the best single-method baseline, or its perplexity rises above Unigram's, the generalisation claim is falsified.

Watch

Extended reading notes

Core claim

SymMark's core discovery is that two watermark families need not compete for the same token. In the serial strategy both watermarks are embedded in every token, maximizing signal but hurting fluency; in the parallel strategy they alternate by token position, preserving quality but losing resilience to edits. The hybrid strategy, the paper's main contribution, computes for each position the token entropy (Shannon entropy of the next-token distribution) and the semantic entropy (entropy over clusters of semantically similar top candidates), and applies the logits watermark only when token entropy is high, the sampling watermark only when semantic entropy is low, and both or neither otherwise. The authors report that this per-token selection outperforms each family alone and every tested single-method baseline, with the serial variant hitting perfect detection under their settings and the hybrid holding an average AUC of 0.984 under attacks against 0.951 for Unigram. The message is that the trade-off is a design choice, not a law.

Load-bearing premise

The fixed thresholds $\alpha=1.0$ and $\beta=0.5$, tuned on a 50-sample C4 subset with one model, are assumed to transfer across all models, datasets, and attacks, and the K-means clustering with $k=64,n=10$ is assumed to capture semantically similar candidates.

Editorial extensions

If this is right

  • On the paper's own results, the hybrid variant widens the operating window: it reaches near-perfect detection with the lowest perplexity among all tested watermarks, and on the four downstream tasks its quality drop is under 1% in two of them.
  • Because detection is an OR of two independent detectors, the paper's dual-signal design means that damaging the text enough to remove the logits signal will likely leave the sampling signal measurable, and vice versa.
  • The framework is modular: the paper's ablations swap Unigram for KGW and AAR for GumbelSoft, and the hybrid combination still outperforms its components, indicating that the entropy gating transfers across watermark families.
  • The serial and parallel variants bound the design space; the hybrid sits between them and inherits the serial's robustness and the parallel's fluency, which the paper demonstrates in the perplexity and attack experiments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the fixed thresholds $\alpha=1.0$ and $\beta=0.5$ were chosen on 50 C4 samples with OPT-6.7B; a domain-adaptive threshold, estimated online from the generation context, could push the quality-detectability frontier further on short generations or code tasks.
  • Editorial extension: the semantic-entropy gate is doing duty as a proxy for 'will the reader notice a substitution'; a direct test of that proxy would be a human or NLI study comparing swapped-token sentences under the gating rule.
  • Editorial extension: because the semantic-entropy computation depends on a tokenizer-aligned embedding space, the method's portability to multilingual or code-tokenized models is an open question that the paper does not address.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SymMark, a framework that integrates logits-based and sampling-based LLM watermarking through three strategies: Serial, Parallel, and Hybrid. In the Hybrid strategy, token entropy and semantic entropy are used per token to decide whether to apply a logits watermark, a sampling watermark, both, or neither. Detection is performed by OR-ing the outputs of the two corresponding watermark detectors. The authors report experiments on OPT, LLaMA, and GPT-J models over C4, OpenGen, and WaterBench-style tasks, claiming state-of-the-art performance in detectability, robustness, text quality, and security, with code released at a public repository.

Significance. If the Hybrid mechanism works as described, the paper makes a useful contribution to LLM watermarking by directly attacking the known trade-offs among robustness, text quality, and security, and by systematically comparing three integration strategies. The experimental scope is broad: multiple models, datasets, downstream tasks, attacks, and a watermark-stealing security evaluation, and the authors also test several combinations of base watermarking schemes. The release of code is a concrete strength that supports reproducibility. However, the central algorithmic description of Hybrid is internally inconsistent, and the reported Hybrid results therefore cannot be unambiguously attributed to the method described in the abstract and introduction.

major comments (3)
  1. [Section 4.3, Algorithm 1, Appendix H] Algorithm 1's if/else-if structure prevents any token from receiving both watermarks. In Algorithm 1, line 5 checks H_TE > alpha and applies the logits watermark; because line 9 is an else-if branch, a token satisfying H_TE > alpha never reaches the sampling-watermark branch, even if it also has H_SE < beta. This directly contradicts the text of Section 4.3, which states that Hybrid may apply both watermarks, and Appendix H, which says the authors adopt Scheme 1 and 'embed two watermark signals when token entropy is high and semantic entropy is low.' Appendix I's Algorithm 3 uses independent if statements and is consistent with the intended behavior, which makes the discrepancy clear. The authors must state exactly which generation procedure was used in the experiments, correct Algorithm 1 if the independent-if logic was actually used, and re-derive any claims about dual-signal robustness from the procedure actually implemented.
  2. [Section 6.3] The robustness explanation attributed to 'Dual-signal Injection' is not supported under the literal Algorithm 1. The text claims Hybrid remains detectable because 'even if one watermarking signal is partially disrupted, the other remains intact,' but if no token receives both watermarks, then the Hybrid text does not contain two independent signals on the same tokens, and the stated mechanism cannot explain the reported robustness. The authors should either correct the algorithm to the independent-if version and confirm that the experiments use it, or provide a different, evidence-based explanation for the observed AUC values.
  3. [Appendix G and Tables 1-2] The fixed thresholds alpha=1.0 and beta=0.5 are tuned on 50 samples from C4 using OPT-6.7B, yet they are then applied to all models, datasets, and attack settings without reporting sensitivity to these choices on the other evaluation settings. This is not circular, since the main evaluations appear to be separate experiments, but it is a generalization concern for the SOTA claim. The authors should report the entropy-threshold sensitivity for at least one additional model/dataset or otherwise justify that the OPT-6.7B/C4 tuning transfers.
minor comments (5)
  1. [Equation (2)] Equation (2) assigns the logits watermark to even positions and the sampling watermark to odd positions, whereas the text of Section 4.2 states the opposite assignment; the equation and text should be aligned.
  2. [Algorithm 1] Algorithm 1 as printed is syntactically confusing: the softmax step appears between the if block and the else-if branch. The indentation and control flow should be reformatted so that the intended branch structure is unambiguous.
  3. [Section 6.1] The sentence 'Series scheme exhibits a perfect TPR of 1.000, signifying no false positives' conflates TPR with TNR; a TPR of 1.000 means no false negatives, while TNR is the metric for false positives.
  4. [Appendix J] The watermark-stealing experiment does not state the gamma and delta used for the attacked Unigram baseline; the appendix specifies gamma=0.25 and delta=0.4 for the Unigram component inside Hybrid, but not for the standalone Unigram comparator, which is needed for a fair comparison.
  5. [General] Tables 1 and 2 report values to three decimals without confidence intervals or multiple-seed statistics. Given that generation is stochastic, at least a note on fixed seeds or repeated trials would be helpful.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims rest on independent benchmarks, and the only self-citation is non-load-bearing.

full rationale

The paper's framework is defined constructively (Algorithm 1 and Equations 1-5) rather than derived from fitted values, and the reported detectability, robustness, text quality, and security numbers come from running those algorithms on C4, OpenGen, and Waterbench tasks. The entropy thresholds alpha=1.0 and beta=0.5 are tuned on 50 C4 samples with OPT-6.7B in Appendix G and then fixed; this is hyperparameter selection, not a fitted parameter renamed as a prediction, because the paper does not claim to predict the benchmark metrics from the threshold values. The only self-citation, Ren et al. (2024b), appears in related work as one of several prior logits-based methods ("Ren et al. (2024b) incorporate the vocabulary's prior distribution") and is not load-bearing for the framework's central premise. The skeptical observation that Algorithm 1 uses else-if while Appendix H describes embedding two watermark signals is an internal-consistency or correctness issue, not a circularity reduction, so it does not affect the circularity score.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the heuristic that high token entropy and low semantic entropy indicate safe positions for watermarking, plus the transferability of two tuned thresholds. The clustering and entropy computations are standard but the semantic grouping via K-means on token embeddings is an arbitrary modeling choice.

free parameters (4)
  • token entropy threshold alpha = 1.0
    Tuned on 50 C4 samples with OPT-6.7B to balance detectability and text quality (Appendix G).
  • semantic entropy threshold beta = 0.5
    Tuned on the same subset; controls when the sampling watermark is applied.
  • top-k token count k = 64
    Chosen for clustering efficiency; sensitivity analysis shows minimal impact on results.
  • cluster count n = 10
    Chosen for clustering efficiency; minimal impact on semantic entropy.
assumptions (4)
  • standard math Shannon entropy and K-means clustering are valid for measuring token uncertainty and semantic grouping.
    Used in Equations 3-5 and Algorithm 1.
  • domain assumption High token entropy implies low text-quality cost for logits watermarking; low semantic entropy implies low quality cost for sampling watermarking.
    Heuristic stated in Section 4.3 and supported only by experiments, not proven.
  • domain assumption The OR fusion of the two detectors maintains an acceptably low false positive rate.
    Algorithm 2; the paper does not analyze the combined FPR.
  • domain assumption Attacker model in watermark stealing follows Jovanović et al. (2024); sampling watermarks are assumed immune.
    Appendix J; the security claim relies on this attack model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Trade-off to Synergy: A Versatile Symbiotic Watermarking Framework for Large Language Models." pith.science (2026). https://pith.science/paper/FW6US3G7

@misc{pith2026250509924,
  author       = {Pith},
  title        = {Pith review of: From Trade-off to Synergy: A Versatile Symbiotic Watermarking Framework for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FW6US3G7}},
  note         = {Machine review of arXiv:2505.09924}
}
read the original abstract

The rise of Large Language Models (LLMs) has heightened concerns about the misuse of AI-generated text, making watermarking a promising solution. Mainstream watermarking schemes for LLMs fall into two categories: logits-based and sampling-based. However, current schemes entail trade-offs among robustness, text quality, and security. To mitigate this, we integrate logits-based and sampling-based schemes, harnessing their respective strengths to achieve synergy. In this paper, we propose a versatile symbiotic watermarking framework with three strategies: serial, parallel, and hybrid. The hybrid framework adaptively embeds watermarks using token entropy and semantic entropy, optimizing the balance between detectability, robustness, text quality, and security. Furthermore, we validate our approach through comprehensive experiments on various datasets and models. Experimental results indicate that our method outperforms existing baselines and achieves state-of-the-art (SOTA) performance. We believe this framework provides novel insights into diverse watermarking paradigms. Our code is available at https://github.com/redwyd/SymMark.

Figures

Figures reproduced from arXiv: 2505.09924 by the authors.

Figure 1
Figure 1. Paradigm comparison between our symbiotic [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. A Versatile Symbiotic Watermark Framework for LLMs. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. High Token Entropy with High Semantic Entropy (Left) and Low Semantic Entropy (Right). Token Entropy Derived from Shannon entropy (Shannon, 1948), quantifies the uncertainty in the logits distribution of a token at the current time step t. Given the model’s logits output, we apply softmax normalization to obtain the probability p i t for each token i ∈ V, and compute token entropy as follows: HT E = − X i p i t log … view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: A comparison of PPL across three symbiotic [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The AUROC curve of watermarked text generated by OPT-6.7B under various attacks on C4 dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: The ASR of watermark stealing for varying [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Comparing AIGT detection methods and ours [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: The AUROC curve of watermarked text generated by OPT-6.7B under various attacks on C4 dataset. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: A comparison of PPL across three symbiotic [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Hyperparameter Analysis of Top-k Selection, Number of Clusters n, TE threshold α and SE threshold β. • Translation Translate the text from English to Chinese and then back to English using the fine￾tuned T5 translation model 2 . • Rephrase (GPT-3.5-turbo) Call GPT-3.5…
Figure 11
Figure 11. Figure 11: Comparison of two watermarking schemes: high versus low token and semantic entropy. “Symbiotic” [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Scheme 1 (Left), Scheme 2 (Right) two extreme cases of hybrid watermarks, corre￾sponding to series and parallel configurations, con￾strain the impact of entropy thresholds on the de￾tectability F1 score. However, an increased propor￾tion of symbiotic watermarks more s…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 16 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Scott Aaronson. 2023. https://www.youtube.com/watch?v=2Kx9jbSMZqA Watermarking of large language models . In Large Language Models and Transformers Workshop at Simons Institute for the Theory of Computing, 2023

    work page 2023

  4. [4]

    Liang Chen, Yatao Bian, Yang Deng, Deng Cai, Shuaiyi Li, Peilin Zhao, and Kam-Fai Wong. 2024. https://doi.org/10.18653/v1/2024.acl-long.496 W at ME : Towards lossless watermarking through lexical redundancy . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9166--9180, Bangkok, Thail...

    doi:10.18653/v1/2024.acl-long.496 2024

  5. [5]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374

    arXiv 2021

  6. [6]

    Miranda Christ, Sam Gunn, and Or Zamir. 2024. Undetectable watermarks for language models. In The Thirty Seventh Annual Conference on Learning Theory, pages 1125--1139. PMLR

    2024

  7. [7]

    Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free dolly: Introducing the world’s first truly open instruction-tuned llm. Company Blog of Databricks

    2023

  8. [8]

    Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, et al. 2024. Scalable watermarking for identifying large language model outputs. Nature, 634(8035):818--823

    work page 2024

Show all 61 references
  1. [9]

    Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    2018 arXiv

  2. [10]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  3. [11]

    Alexander Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir Radev. 2019. https://doi.org/10.18653/v1/P19-1102 Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model . In Proceedings of the 57th Annual Meeting of the Association for...

    2019 doi

  4. [12]

    Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. 2019. https://doi.org/10.18653/v1/P19-1346 ELI 5: Long form question answering . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3558--356...

    2019 doi

  5. [13]

    Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. 2024. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017):625--630

    2024

  6. [14]

    Pierre Fernandez, Antoine Chaffin, Karim Tit, Vivien Chappelier, and Teddy Furon. 2023. https://arxiv.org/abs/2308.00113 Three bricks to consolidate watermarks for large language models . Preprint, arXiv:2308.00113

    2023 arXiv

  7. [15]

    Jiayi Fu, Xuandong Zhao, Ruihan Yang, Yuansen Zhang, Jiangjie Chen, and Yanghua Xiao. 2024 a . https://doi.org/10.18653/v1/2024.acl-long.315 G umbel S oft: Diversified language model watermarking via the G umbel M ax-trick . In Proceedings of the 62nd Annual Meeting of the Ass...

    2024 doi

  8. [16]

    Yu Fu, Deyi Xiong, and Yue Dong. 2024 b . Watermarking conditional text generation for ai detection: Unveiling challenges and a semantic-aware watermark remedy. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18003--18011

    2024

  9. [17]

    Chenchen Gu, Xiang Lisa Li, Percy Liang, and Tatsunori Hashimoto. 2024. https://openreview.net/forum?id=9k0krNzvlV On the learnability of watermarks for language models . In The Twelfth International Conference on Learning Representations

    2024

  10. [18]

    Batu Guan, Yao Wan, Zhangqian Bi, Zheng Wang, Hongyu Zhang, Pan Zhou, and Lichao Sun. 2024. https://aclanthology.org/2024.findings-emnlp.541 C ode IP : A grammar-guided multi-bit watermark for large language models of code . In Findings of the Association for Computational Lin...

    2024

  11. [19]

    Zhiwei He, Binglin Zhou, Hongkun Hao, Aiwei Liu, Xing Wang, Zhaopeng Tu, Zhuosheng Zhang, and Rui Wang. 2024. https://doi.org/10.18653/v1/2024.acl-long.226 Can watermarks survive translation? on the cross-lingual consistency of text watermark for large language models . In Pro...

    2024 doi

  12. [20]

    Abe Hou, Jingyu Zhang, Tianxing He, Yichen Wang, Yung-Sung Chuang, Hongwei Wang, Lingfeng Shen, Benjamin Van Durme, Daniel Khashabi, and Yulia Tsvetkov. 2024 a . https://doi.org/10.18653/v1/2024.naacl-long.226 S em S tamp: A semantic watermark with paraphrastic robustness for ...

    2024 doi

  13. [21]

    Abe Hou, Jingyu Zhang, Yichen Wang, Daniel Khashabi, and Tianxing He. 2024 b . https://doi.org/10.18653/v1/2024.findings-acl.98 k- S em S tamp: A clustering-based semantic watermark for detection of machine-generated text . In Findings of the Association for Computational Ling...

    2024 doi

  14. [22]

    Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. 2024. https://openreview.net/forum?id=uWVC5FVidc Unbiased watermark for large language models . In The Twelfth International Conference on Learning Representations

    2024

  15. [23]

    Mingjia Huo, Sai Ashish Somayajula, Youwei Liang, Ruisi Zhang, Farinaz Koushanfar, and Pengtao Xie. 2024. https://openreview.net/forum?id=AqBz54aFyj Token-specific watermarking with enhanced detectability and semantic coherence for large language models . In Forty-first Intern...

    2024

  16. [24]

    Nikola Jovanovi \'c , Robin Staab, and Martin Vechev. 2024. https://openreview.net/forum?id=3dlVsf6yfz Watermark stealing in large language models . In ICLR 2024 Workshop on Reliable and Responsible Foundation Models

    2024

  17. [25]

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. https://proceedings.mlr.press/v202/kirchenbauer23a.html A watermark for large language models . In Proceedings of the 40th International Conference on Machine Learning, volume 202 o...

    2023

  18. [26]

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. 2024. https://openreview.net/forum?id=DEJIDCmWOz On the reliability of watermarks for large language models . In The Twelfth...

    2024

  19. [27]

    Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Frederick Wieting, and Mohit Iyyer. 2023. https://openreview.net/forum?id=WbFhFvjjKj Paraphrasing evades detectors of AI -generated text, but retrieval is an effective defense . In Thirty-seventh Conference on Neural Inform...

    2023

  20. [28]

    Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. 2023. Robust distortion-free watermarks for language models. arXiv preprint arXiv:2307.15593

    2023 arXiv

  21. [29]

    Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. 2024. https://openreview.net/forum?id=FpaCL1MO2C Robust distortion-free watermarks for language models . Transactions on Machine Learning Research

    2024

  22. [30]

    Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim. 2024. https://doi.org/10.18653/v1/2024.acl-long.268 Who wrote this code? watermarking for code generation . In Proceedings of the 62nd Annual Meeting of the Association for ...

    2024 doi

  23. [31]

    Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, and James Zou. 2023. https://openreview.net/forum?id=SPuX8tKKIQ GPT detectors are biased against non-native english writers . In ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models

    2023

  24. [32]

    Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng, and Lijie Wen. 2024 a . https://openreview.net/forum?id=6p8lpe4MNf A semantic invariant robust watermark for large language models . In The Twelfth International Conference on Learning Representations

    2024

  25. [33]

    Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Xi Zhang, Lijie Wen, Irwin King, Hui Xiong, and Philip Yu. 2024 b . A survey of text watermarking in the era of large language models. ACM Computing Surveys, 57(2):1--36

    2024

  26. [34]

    Yepeng Liu and Yuheng Bu. 2024. https://arxiv.org/abs/2401.13927 Adaptive text watermark for large language models . Preprint, arXiv:2401.13927

    2024 arXiv

  27. [35]

    Yijian Lu, Aiwei Liu, Dianzhi Yu, Jingjing Li, and Irwin King. 2024. https://doi.org/10.18653/v1/2024.acl-long.630 An entropy-based text watermarking detection method . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long P...

    2024 doi

  28. [36]

    Yiyang Luo, Ke Lin, and Chao Gu. 2024. https://arxiv.org/abs/2403.10020 Lost in overlap: Exploring watermark collision in llms . Preprint, arXiv:2403.10020

    2024 arXiv

  29. [37]

    James MacQueen et al. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, pages 281--297. Oakland, CA, USA

    1967

  30. [38]

    George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39--41

    1995

  31. [39]

    R OpenAI et al. 2023. Gpt-4 technical report. ArXiv, 2303:08774

    2023

  32. [40]

    Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, Irwin King, and Philip S. Yu. 2024. https://aclanthology.org/2024.emnlp-demo.7 M ark LLM : An open-source toolkit for LLM watermarking . In Proceedings of th...

    2024

  33. [41]

    Qi Pang, Shengyuan Hu, Wenting Zheng, and Virginia Smith. 2024. https://openreview.net/forum?id=rIOl7KbSkv No free lunch in LLM watermarking: Trade-offs in watermarking design choices . In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  34. [42]

    Hao Peng, Xiaozhi Wang, Shengding Hu, Hailong Jin, Lei Hou, Juanzi Li, Zhiyuan Liu, and Qun Liu. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.335 COPEN : Probing conceptual knowledge in pre-trained language models . In Proceedings of the 2022 Conference on Empirical Metho...

    2022 doi

  35. [43]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  36. [44]

    Jie Ren, Han Xu, Yiding Liu, Yingqian Cui, Shuaiqiang Wang, Dawei Yin, and Jiliang Tang. 2024 a . https://doi.org/10.18653/v1/2024.findings-naacl.40 A robust semantics-based watermark for large language model against paraphrasing . In Findings of the Association for Computatio...

    2024 doi

  37. [45]

    Yubing Ren, Ping Guo, Yanan Cao, and Wei Ma. 2024 b . https://doi.org/10.18653/v1/2024.findings-acl.327 Subtle signatures, strong shields: Advancing robust and imperceptible watermarking in large language models . In Findings of the Association for Computational Linguistics: A...

    2024 doi

  38. [46]

    Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. 2024. https://openreview.net/forum?id=NvSwR4IvLO Can AI -generated text be reliably detected?

    2024

  39. [47]

    C. E. Shannon. 1948. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x A mathematical theory of communication . The Bell System Technical Journal, 27(3):379--423

    1948

  40. [48]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  41. [49]

    Shangqing Tu, Yuliang Sun, Yushi Bai, Jifan Yu, Lei Hou, and Juanzi Li. 2024. https://doi.org/10.18653/v1/2024.acl-long.83 W ater B ench: Towards holistic evaluation of watermarks for large language models . In Proceedings of the 62nd Annual Meeting of the Association for Comp...

    2024 doi

  42. [50]

    A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems

    2017

  43. [51]

    Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model . https://github.com/kingoflolz/mesh-transformer-jax

    2021

  44. [52]

    Lean Wang, Wenkai Yang, Deli Chen, Hao Zhou, Yankai Lin, Fandong Meng, Jie Zhou, and Xu Sun. 2024. https://openreview.net/forum?id=JYu5Flqm9D Towards codable watermarking for injecting multi-bits information to LLM s . In The Twelfth International Conference on Learning Repres...

    2024

  45. [53]

    Bram Wouters. 2023. Optimizing watermarks for large language models. arXiv preprint arXiv:2312.17295

    2023 arXiv

  46. [54]

    Qilong Wu and Varun Chandrasekaran. 2024. https://doi.org/10.18653/v1/2024.acl-long.464 Bypassing LLM watermarks with color-aware substitutions . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8549--858...

    2024 doi

  47. [55]

    Yihan Wu, Zhengmian Hu, Junfeng Guo, Hongyang Zhang, and Heng Huang. 2024. https://openreview.net/forum?id=c8qWiNiqRY A resilient and accessible distribution-preserving watermark for large language models . In Forty-first International Conference on Machine Learning

    2024

  48. [56]

    Yihan Wu, Zhengmian Hu, Hongyang Zhang, and Heng Huang. 2023. Dipmark: A stealthy, efficient and resilient watermark for large language models. arXiv preprint arXiv:2310.07710

    2023 arXiv

  49. [57]

    KiYoon Yoo, Wonhyuk Ahn, and Nojun Kwak. 2024. https://doi.org/10.18653/v1/2024.naacl-long.224 Advancing beyond identification: Multi-bit watermark for large language models . In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computatio...

    2024 doi

  50. [58]

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068

    2022 arXiv

  51. [59]

    Zhaoxi Zhang, Xiaomei Zhang, Yanjun Zhang, Leo Yu Zhang, Chao Chen, Shengshan Hu, Asif Gill, and Shirui Pan. 2024. Large language model watermark stealing with mixed integer programming. arXiv preprint arXiv:2405.19677

    2024 arXiv

  52. [60]

    Xuandong Zhao, Prabhanjan Vijendra Ananth, Lei Li, and Yu-Xiang Wang. 2024. https://openreview.net/forum?id=SsmT8aO45L Provable robust watermarking for AI -generated text . In The Twelfth International Conference on Learning Representations

    2024

  53. [61]

    Chaoyi Zhu, Jeroen Galjaard, Pin-Yu Chen, and Lydia Chen. 2024. https://doi.org/10.18653/v1/2024.findings-acl.678 Duwak: Dual watermarks in large language models . In Findings of the Association for Computational Linguistics: ACL 2024, pages 11416--11436, Bangkok, Thailand. As...

    2024 doi

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.