Pith. sign in

REVIEW 1 major objections 4 minor 46 references

StealthInk: A Multi-bit and Stealthy Watermark for Large Language Models

T0 review · 1 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read StealthInk claims to embed multi-bit provenance messages into LLM-generated text while provably preserving the original token distribution, carrying 24-bit payloads with about 92% bit accuracy in 300 tokens and resisting watermark-forgery…

desk verdict Genuinely new multi-bit extension of stealthy watermarking, but the K-shot stealthiness guarantee is conditional on independence the algorithm doesn't provide. read the letter →

arxiv 2506.05502 v1 pith:YEFWVZHO submitted 2025-06-05 cs.CR cs.AI

Ya Jiang , Chuxiong Wu , Massieh Kordi Boroujeny , Brian Mark , Kai Zeng This is my paper
classification cs.CRcs.AI
keywords LLMwatermarkingmulti-bitwatermarkstealthyorunbiasedtextprovenancedistribution-preservingreweightingspoofingAI-generateddetectionred-listtoken
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

StealthInk proposes a way to stamp AI-generated text with a multi-bit message — user ID, timestamp, model ID — without measurably changing the distribution of what the model writes. The paper's central claim is that a token-reweighting rule can delete some candidate tokens and double others at each position while keeping the expected output distribution identical to the unwatermarked model's. If this holds, a provider can trace text back to a specific user and moment, and an attacker who studies the output statistics cannot tell watermarked from clean text, which blocks the known watermark-forgery attack. The paper proves the expected-preservation property in Theorem 4.4, derives a token-count lower bound for detection at a fixed equal error rate, and reports an AUC near 0.98 with bit accuracy near 0.92 for 24-bit messages in 300 tokens, at text quality close to the unwatermarked baseline. The motivation is that zero-bit watermarks only answer "was this AI-written?", while provenance data answers who wrote it, with which model, and when.

What carries the argument

The load-bearing object is the reweighting function $F_k(\theta, M, P_O)$ in Eq. (5): with the vocabulary permuted by $\theta$, the message $M$ fixes a red-list interval of length $\gamma = 2^{-m}$ whose token probabilities are zeroed, and a complementary interval whose probabilities are doubled, with four cases depending on where the cumulative probabilities $\alpha$ and $\beta$ sit relative to 0.5. The construction is chosen so that for every token the expected probability over the random permutation equals the original probability, which is exactly the paper's definition of a stealthy multi-bit watermark (Definition 4.1). This expected-preservation identity is what makes the decoder's red-list counting statistically valid and what denies an attacker any distributional signal to forge. Around this core, StealthInk adds the position-allocation method of MPAC to spread a message sequence across token chunks, and a per-query history log that keeps successive ciphers independent as Theorem 4.3 requires.

What would settle it

Set the timestamp and user metadata bits to a fixed value, submit the same prompt to the watermarked model tens of thousands of times, and compare the empirical first-token distribution with the unwatermarked model's first-token distribution for that prompt. If the distributions differ by more than sampling error — in particular, a cluster of tokens appearing near zero frequency where the red list saturates the early vocabulary positions — then the unconditional stealthiness claim fails and the guarantee holds only because messages vary in practice.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that multi-bit watermarking need not distort the language model's output distribution. For each token position, StealthInk orders the vocabulary by a pseudorandom permutation $\theta$ seeded with the watermark key and the recent n-gram context, and the message chunk $M$ selects an interval of that permutation of length $\gamma = 2^{-m}$; tokens in that interval are the 'red list' and get probability zero, tokens in a mirror interval get doubled probability, and a four-case rule (Eq. 5) fixes the intervals so that the expected reweighted distribution, averaged over $\theta$, equals the original $P_O$. Theorem 4.4 states this unbiasedness for every i.i.d. permutation, and Theorem 4.3 extends it to $K$ query-response pairs when the ciphers are independent. The decoder maps each token to a message position, counts tokens in each candidate red list, picks the message minimizing that count, and runs a z-test over the recovered message sequence. The paper also derives a lower bound on tokens needed for a target equal error rate, finding that splitting a payload across many single-bit positions is more token-efficient than packing bits into fewer positions, and that repeated texture keys substantially raise the token requirement. Empirically, with 24 bits in 300 tokens it reports bit accuracy around 0.92, AUC around 0.98, near-baseline perplexity, and the lowest forged-text detection rates under a spoofing attack compared with MPAC and the schemes of Qu et al. and Fernandez et al.

Load-bearing premise

K-shot stealthiness (Theorem 4.3) requires the vocabulary permutations across separate queries to be independent, but StealthInk keeps no global history, so identical prompts carrying the same message (same user ID and timestamp) reuse the same permutation; the paper's defense — that timestamps make messages vary and that flooding queries within one millisecond is impractical (Section 4.2, Appendix F) — is an assumption about attacker timing, not a proven property, so a determined attacker who forces identical messages can detectably distort the first-token distribution.

Editorial extensions

If this is right

  • With the watermark key, a verifier can extract userID, timestamp, and modelID from text alone, needing neither the generating model's API nor the original prompt, which makes forensic traceability fast and cheap.
  • Because the watermarked and clean distributions coincide in expectation, the stealing/spoofing attack that exploits n-gram distribution gaps mostly fails: on the evaluated Dolly-CW prompts only about 29% of forged texts are flagged at a 10% false-positive rate, versus about 70% for MPAC and about 98% for the Qu et al. scheme under the FNR = $10^{-3}$ setting.
  • The equal-error-rate lower bound guides capacity: distributing unit capacity across positions (more chunks, fewer bits each) needs fewer tokens for the same detection reliability, and keeping the repetition rate $p$ of texture keys low is what lets StealthInk hold its token requirement down where global-history schemes degrade over time.
  • Text quality stays close to the unwatermarked model on perplexity, BLEU, BERTScore, and ROUGE, whereas the compared distribution-shifting schemes trade quality for detectability; StealthInk instead pays a detectability cost at short lengths, catching up to MPAC as token count grows (e.g., comparable TPR at 400 tokens where MPAC reaches it at 200).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Operationally, the guarantee rides on the message changing between queries: the per-query timestamp bits are what make repeated identical prompts produce different red lists. A deployment that freezes or strips those metadata bits would silently revert to a detectable watermark, so the paper's defense against the same-message attack is a deployment discipline rather than a theorem.
  • The red-list counting decoder only needs the text and the key, so the same machinery should transfer to structured outputs (code, JSON, and other constrained decoding) and to streaming generation, where positions and chunk boundaries are known to both encoder and decoder.
  • The paper's own Appendix M says low-entropy contexts — where $\alpha$ and $\beta$ fall in the overlapping Cases 3 and 4 — weaken detection; a practical calibration would map per-position output entropy to achievable bit accuracy, giving deployers a rule for when 24-bit payloads are feasible.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 4 minor

Summary. StealthInk proposes a multi-bit watermarking scheme for LLMs that reweights token probabilities based on a vocabulary permutation, a message-dependent red list, and a doubling operation on a complementary interval. The central formal claims are (i) per-token unbiasedness: for each random permutation, the expected watermarked token distribution equals the original model's distribution (Theorem 4.4), and (ii) K-shot stealthiness: over multiple prompt-response pairs with independent ciphers, the joint watermarked distribution matches the original (Theorem 4.3). The paper also derives a relation between minimum detectable text length and equal error rate, and reports experiments on text quality, spoofing resistance, detectability, and robustness against copy-paste and paraphrasing attacks. The empirical results show that StealthInk achieves AUC around 0.98 and bit accuracy around 0.92 for 24-bit messages in 300 tokens, while maintaining low perplexity relative to some prior multi-bit schemes.

Significance. If the central claims held in their full generality, StealthInk would be a valuable contribution: it combines multi-bit embedding with a distribution-preserving reweighting rule, and it provides an explicit lower-bound analysis relating capacity to the equal error rate. The per-token unbiasedness proof via the symmetry of the intervals A and B is elegant and appears to be exact rather than dependent on the Gaussian approximation used elsewhere. The empirical study is reasonably broad, covering text quality, detection, spoofing, and editing attacks, and the paper honestly discusses limitations such as low entropy and paraphrasing robustness. However, the load-bearing K-shot stealthiness guarantee is proven only under an independence assumption that the deployed encoder violates, and the paper's practical defense against the resulting attack is an unproven assertion about attacker timing. These gaps currently prevent the paper from substantiating its advertised 'preserves the original text distribution' property over repeated queries.

major comments (1)
  1. [Section 4.2 / Algorithm 1 vs. Theorem 4.3] The security assertion in Section 4.2 and Appendix F that an attacker would need about 100,000 same-prompt, same-timestamp queries to detect the first-token distortion is not supported by any analysis. With m=1, the red list has probability mass roughly 1/2 under P_O; a goodness-of-fit test comparing the empirical first-token distribution to the known P_O over a few hundred repeated queries should already reject, because half the vocabulary is assigned zero probability. The paper provides no sample-complexity bound or experiment for this scenario, and Appendix F itself concedes that if queries share the same timestamp, 'a bunch of tokens' probabilities are 0.' Thus the 'practically impossible' claim is an unsupported quantitative assertion, and the gap between the theorem's assumption and the algorithm's behavior cannot be dismissed as an unrealistic attack.
minor comments (4)
  1. [Table 3] In the H=2 row for the Dolly dataset, the AUC value '09491' appears to be a typo for '0.9491'.
  2. [Definition 4.2] The right-hand side of Eq. (1) is written as a product over P_O(x^k | a^k), but the left-hand side is an expectation of the joint watermarked distribution; the notation would be clearer if the definition explicitly stated that the expectation equals the product of the unwatermarked marginals.
  3. [Section 6.2 / Table 4] The discussion of the lower PPL for StealthInk in Table 4 attributes the difference to filtering of low-probability tokens and to outliers in the non-watermarked texts. The statement that 'some non-watermarked responses exhibit extremely high PPL ... which raises their median' is statistically incorrect, as the median is insensitive to outliers; the authors should provide a more careful finite-sample explanation or show statistical significance for the PPL convergence in Appendix J.
  4. [Section 4.2] The sentence 'the same texture key should not repeat for watermarking' is potentially misleading because hist is per query; the text should clarify that this rule applies only within a single query attempt, which is the cause of the K-shot independence gap discussed above.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: StealthInk's unbiasedness theorem is a symmetry calculation over random permutations, and its Lmin bound is derived from the scheme's own statistics, not from a fitted input.

full rationale

The central derivation is self-contained. Theorem 4.4 proves E_θ[P_W^M] = P_O by a direct calculation: for a uniform random permutation θ, token intervals on the cumulative-probability axis are [x_i, x_i + p_i], the zeroed interval A(θ) and doubled interval B(θ) are defined symmetrically in Eq. (20), and the integrand g(x_i) = |S_i∩B| - |S_i∩A| is shown to be antisymmetric about (1-p_i)/2, so each case integral in Eq. (29) vanishes and E_θ[p_i^w] = p_i. This is a proof of a constructed property, not an assumption of it; the theorem does not presuppose the equality it establishes. The multi-bit capacity claims and the minimum-length/EER analysis in Section 5 and Appendix G are likewise detection-theoretic calculations of the scheme's own binomial statistics, not a fitted parameter relabeled as a prediction. The only self-citation, Kordi Boroujeny et al. 2024, is used as a comparison point for prior distortion-free multi-bit methods and is not load-bearing. The main caveat, correctly flagged in Appendix F, is a robustness gap rather than circularity: Algorithm 1 resets hist per query, so repeated same-prompt, same-message queries reuse the same first-token permutation, and the K-shot theorem's independence assumption is not guaranteed by the deployed system. But an unmet hypothesis is not an equation made true by definition, so no circular step can be exhibited from the paper's own equations.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central scheme introduces no new physical or mathematical entities; it is an algorithmic construction. The free parameters are design choices (m and H) that directly shape the reported capacity and detectability. The axioms are mostly standard cryptographic and probabilistic assumptions, plus one ad hoc approximation in the theoretical length analysis and one domain assumption about timestamp-driven message randomness.

free parameters (2)
  • Unit capacity m (and red-list fraction gamma = 2^-m) = m = 1 chosen; gamma = 0.5
    m is a tunable reliability and capacity knob. The authors test m in {1,2,3,4} and select m=1 for main experiments (Section 6.2, Figure 4), which affects all reported bit accuracy and TPR numbers.
  • Message sequence length H (number of chunks) = 24, 36, 48 depending on payload
    H is chosen as the payload size in experiments; it determines how many positions are embedded and interacts with detection length. The reported AUC and bit accuracy depend on this choice.
assumptions (4)
  • domain assumption The pseudorandom function output is indistinguishable from a truly random permutation to an attacker without the watermark key.
    Used in Section 2 and all experiments (SHA-256). Stealthiness is computational, not information-theoretic; if the PRF is broken, the red lists can be guessed.
  • standard math For a random permutation, the cumulative position x_i of token V_i is uniform on [0, 1-p_i].
    Used in Appendix C in the proof of Theorem 4.4 to establish, by symmetry, that the expected reweighted probability equals the original probability.
  • ad hoc to paper The sums alpha and beta, and the residual block delta, are jointly Gaussian with sigma'^2 approximately sigma^2.
    Appendix C and Appendix G rely on this approximation to compute expected red-list probabilities and the minimum detection length. The equality sigma'^2 approximately sigma^2 is asserted, not proven.
  • domain assumption The message chunks at texture-key positions that repeat across queries are sufficiently random (timestamp bits) so that the mixture over messages restores the original distribution.
    Section 4.2 and Appendix F rely on changing timestamps to maintain K-shot stealthiness when the same prompt is queried repeatedly. This condition is not stated in Theorem 4.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of StealthInk: A Multi-bit and Stealthy Watermark for Large Language Models." pith.science (2026). https://pith.science/paper/YEFWVZHO

@misc{pith2026250605502,
  author       = {Pith},
  title        = {Pith review of: StealthInk: A Multi-bit and Stealthy Watermark for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YEFWVZHO}},
  note         = {Machine review of arXiv:2506.05502}
}
read the original abstract

Watermarking for large language models (LLMs) offers a promising approach to identifying AI-generated text. Existing approaches, however, either compromise the distribution of original generated text by LLMs or are limited to embedding zero-bit information that only allows for watermark detection but ignores identification. We present StealthInk, a stealthy multi-bit watermarking scheme that preserves the original text distribution while enabling the embedding of provenance data, such as userID, TimeStamp, and modelID, within LLM-generated text. This enhances fast traceability without requiring access to the language model's API or prompts. We derive a lower bound on the number of tokens necessary for watermark detection at a fixed equal error rate, which provides insights on how to enhance the capacity. Comprehensive empirical evaluations across diverse tasks highlight the stealthiness, detectability, and resilience of StealthInk, establishing it as an effective solution for LLM watermarking applications.

Figures

Figures reproduced from arXiv: 2506.05502 by the authors.

Figure 1
Figure 1. StealthInk encoder: m = 2, H = 4. 0 α β 0.5 β¯ α¯ 1 ⋆ 0 ⋆ 2⋆ ⋆ Case 1: β ≤ 0.5 0 β¯ α¯ 0.5 α β 1 ⋆ 2⋆ ⋆ 0 ⋆ Case 2: α ≥ 0.5 0 α β¯ 0.5 β α¯ 1 ⋆ 0 ⋆ 2⋆ ⋆ Case 3: α<0.5 and β>0.5 and α + β ≤ 1 0 β¯ α 0.5 α¯ β 1 ⋆ 2⋆ ⋆ 0 ⋆ Case 4: α<0.5 and β>0.5 and α + β>1 [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Multi-bit watermarking reweighting rule. With the original distribution P i O at the ith gener￾ation step, the reweighted probability of token tk is P M W (tk|a, x1:i−1, θi) = G(F(θ, M, Pi O))k = F(θi , M, Pi O)k − F(θi , M, Pi O)k−1. We prove the follow￾ing result in Appendix C. Theorem 4.4. For each θ ∈ i.i.d. θ1:L, the reweighting func￾tion Fk(θ, M, PO) in (5) ensures stealthy watermarking. Algorithm 1 shows the … view at source ↗
Figure 3
Figure 3. Lmin vs. eer for p = 0 and p = 0.2. 6. Experiments We compare StealthInk with SOTA methods (Yoo et al., 2024; Qu et al., 2024; Fernandez et al., 2023) on stealthiness, detectability, and robustness. For text completion, unless noted otherwise, we use LLAMA2-7B (Touvron et al., 2023) and 500 randomly selected texts from the RealNewsLike subset of C4 (Raffel et al., 2020), trimming a fixed number of tokens from the st… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Detection performance comparisons with m ∈ {1, 2, 3, 4} and H = 1. In particular, we compare the TPR of StealthInk and MPAC with 1% FPR and 0.1%FPR, respectively, across various number of tokens in [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: TPR of StealthInk and MPAC with number of tokens [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Text quality of watermarking schemes at 200 tokens. For 36 and 48 bits, we omit (Fernandez et al., 2023) due to its high computational cost (see extraction time in [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Performance of StealthInk across increasing number of tokens. L. Capacity Enhancement through Key Iteration Intuitively, the watermark keys can be used to embed messages. For instance, to embed a 2-bit message, the watermark key can be selected from 4 possible candidat…
Figure 8
Figure 8. Figure 8: Performance when enhancing the capacity of StealthInk through two dimensions, with a total of 24 + m′ bits embedded, where m′ ∈ [0, 1, 2, 3, 4, 5, 6] [PITH_FULL_IMAGE:figures/full_fig_p024_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 30 canonical work pages

  1. [1]

    My AI Safety Lecture for UT Effective Altruism

    Aaronson, S. My AI Safety Lecture for UT Effective Altruism. , Nov. 2023. URL https://scottaaronson.blog/?p=6823. Accessed May 5, 2023

    work page 2023

  2. [2]

    and Fritz, M

    Abdelnabi, S. and Fritz, M. Adversarial watermarking transformer: Towards tracing text provenance with data hiding. In 2021 IEEE Symposium on Security and Privacy (SP). IEEE, 2021

    work page 2021

  3. [3]

    J., Raskin, V., Crogan, M., Hempelmann, C., Kerschbaum, F., Mohamed, D., and Naik, S

    Atallah, M. J., Raskin, V., Crogan, M., Hempelmann, C., Kerschbaum, F., Mohamed, D., and Naik, S. Natural language watermarking: Design, analysis, and a proof-of-concept implementation. In Information Hiding: 4th International Workshop, IH 2001 Pittsburgh, PA, USA, April 25--27, 2001 Proceedings 4. Springer, 2001

    work page 2001

  4. [4]

    J., Raskin, V., Hempelmann, C

    Atallah, M. J., Raskin, V., Hempelmann, C. F., Karahan, M., Sion, R., Topkara, U., and Triezenberg, K. E. Natural language watermarking and tamperproofing. In International workshop on information hiding. Springer, 2002

    work page 2002

  5. [5]

    Program synthesis with large language models

    Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021

    arXiv 2021

  6. [6]

    and Hu., K

    Bartz, D. and Hu., K. OpenAI, Google, others pledge to watermark AI content for safety, White House says . Website, 2023. https://www.reuters.com/technology/ openai-google-others-pledge-watermark-ai-content-safety-white-house-2023-07-21

    work page 2023

  7. [7]

    Bojar, O., Buck, C., Federmann, C., Haddow, B., Koehn, P., Leveling, J., Monz, C., Pecina, P., Post, M., Saint-Amand, H., Soricut, R., Specia, L., and Tamchyna, A. s. Findings of the 2014 workshop on statistical machine translation. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pp.\ 12--58, Baltimore, Maryland, USA, June 2014. A...

    2014

  8. [8]

    Undetectable watermarks for language models

    Christ, M., Gunn, S., and Zamir, O. Undetectable watermarks for language models. In Agrawal, S. and Roth, A. (eds.), Proceedings of Thirty Seventh Conference on Learning Theory, volume 247 of Proceedings of Machine Learning Research, pp.\ 1125--1139. PMLR, 30 Jun--03 Jul 2024. URL https://proceedings.mlr.press/v247/christ24a.html

    work page 2024

Show all 46 references
  1. [9]

    Cohran, W. G. Sampling Techniques. Wiley, 3rd edition, 1977

    1977

  2. [10]

    Free dolly: Introducing the world’s first truly open instruction-tuned llm

    Conover, M., Hayes, M., Mathur, A., Xie, J., Wan, J., Shah, S., Ghodsi, A., Wendell, P., Zaharia, M., and Xin, R. Free dolly: Introducing the world’s first truly open instruction-tuned llm. Company Blog of Databricks, 2023

    2023

  3. [11]

    Three bricks to consolidate watermarks for large language models

    Fernandez, P., Chaffin, A., Tit, K., Chappelier, V., and Furon, T. Three bricks to consolidate watermarks for large language models. In 2023 IEEE International Workshop on Information Forensics and Security (WIFS). IEEE, 2023

    2023

  4. [12]

    How to construct random functions

    Goldreich, O., Goldwasser, S., and Micali, S. How to construct random functions. Journal of the ACM (JACM), 33 0 (4), 1986

    1986

  5. [13]

    M., Kocisky, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., and Blunsom, P

    Hermann, K. M., Kocisky, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., and Blunsom, P. Teaching machines to read and comprehend. Advances in neural information processing systems, 28, 2015

    2015

  6. [14]

    Unbiased watermark for large language models

    Hu, Z., Chen, L., Wu, X., Wu, Y., Zhang, H., and Huang, H. Unbiased watermark for large language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=uWVC5FVidc

    2024

  7. [15]

    Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7b. Technical report, 2023

    2023

  8. [16]

    Watermark stealing in large language models

    Jovanovi\' c , N., Staab, R., and Vechev, M. Watermark stealing in large language models. In Proceedings of the 41st International Conference on Machine Learning, 2024

    2024

  9. [17]

    A watermark for large language models

    Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., and Goldstein, T. A watermark for large language models. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proceedings of the 40th International Conference on Machine Learnin...

    2023

  10. [18]

    On the reliability of watermarks for large language models

    Kirchenbauer, J., Geiping, J., Wen, Y., Shu, M., Saifullah, K., Kong, K., Fernando, K., Saha, A., Goldblum, M., and Goldstein, T. On the reliability of watermarks for large language models. In ICLR, 2024. URL https://openreview.net/forum?id=DEJIDCmWOz

    2024

  11. [19]

    Multi-Bit Distortion-Free Watermarking for Large Language Models

    Kordi Boroujeny, M., Jiang, Y., Zeng, K., and Mark, B. Multi-Bit Distortion-Free Watermarking for Large Language Models . arXiv preprint arXiv:2402.16578, 2024

    2024 arXiv

  12. [20]

    Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense

    Krishna, K., Song, Y., Karpinska, M., Wieting, J., and Iyyer, M. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA...

    2023

  13. [21]

    Robust distortion-free watermarks for language models

    Kuditipudi, R., Thickstun, J., Hashimoto, T., and Liang, P. Robust distortion-free watermarks for language models. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=FpaCL1MO2C

    2024

  14. [22]

    Rouge: A package for automatic evaluation of summaries

    Lin, C.-Y. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp.\ 74--81, 2004

    2004

  15. [23]

    Multilingual denoising pre-training for neural machine translation

    Liu, Y., Gu, J., Goyal, N., Li, X., Edunov, S., Ghazvininejad, M., Lewis, M., and Zettlemoyer, L. Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics, 8, 2020

    2020

  16. [24]

    ChatGPT: Optimizing language models for dialogue

    OpenAI. ChatGPT: Optimizing language models for dialogue . Website, 2022. https://openai.com/blog/chatgpt

    2022

  17. [25]

    Bleu: a method for automatic evaluation of machine translation

    Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp.\ 311--318, 2002

    2002

  18. [26]

    Academic Integrity considerations of AI Large Language Models in the post-pandemic era: ChatGPT and beyond

    Perkins, M. Academic Integrity considerations of AI Large Language Models in the post-pandemic era: ChatGPT and beyond . Journal of university teaching & learning practice, 20 0 (2), 2023

    2023

  19. [27]

    Mark my words: Analyzing and evaluating language model watermarks

    Piet, J., Sitawarin, C., Fang, V., Mu, N., and Wagner, D. Mark my words: Analyzing and evaluating language model watermarks. arXiv preprint arXiv:2312.00273, 2023

    2023 arXiv

  20. [28]

    Provably Robust Multi-bit Watermarking for AI-generated Text via Error Correction Code

    Qu, W., Yin, D., He, Z., Zou, W., Tao, T., Jia, J., and Zhang, J. Provably Robust Multi-bit Watermarking for AI-generated Text via Error Correction Code . arXiv preprint arXiv:2401.16820, 2024

    2024 arXiv

  21. [29]

    Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21 0 (140), 2020

    2020

  22. [30]

    Shih, F. Y. Digital watermarking and steganography: fundamentals and techniques. CRC press, 2017

    2017

  23. [31]

    M., Hauth, A., Millican, K., et al

    Team, G., Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  24. [32]

    Topkara, U., Topkara, M., and Atallah, M. J. The hiding virtues of ambiguity: quantifiably resilient watermarking of natural language text through synonym substitutions. In Proceedings of the 8th workshop on Multimedia and security, 2006

    2006

  25. [33]

    Llama 2: Open foundation and fine-tuned chat models

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  26. [34]

    Towards codable watermarking for injecting multi-bits information to LLM s

    Wang, L., Yang, W., Chen, D., Zhou, H., Lin, Y., Meng, F., Zhou, J., and Sun, X. Towards codable watermarking for injecting multi-bits information to LLM s. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=JYu5Flqm9D

    2024

  27. [35]

    Dipmark: A stealthy, efficient and resilient watermark for large language models

    Wu, Y., Hu, Z., Zhang, H., and Huang, H. Dipmark: A stealthy, efficient and resilient watermark for large language models. In Proceedings of the 41st International Conference on Machine Learning, 2024

    2024

  28. [36]

    Advancing beyond identification: Multi-bit watermark for large language models

    Yoo, K., Ahn, W., and Kwak, N. Advancing beyond identification: Multi-bit watermark for large language models. In Duh, K., Gomez, H., and Bethard, S. (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Huma...

    2024 doi

  29. [37]

    Excuse me, sir? Your language model is leaking (information)

    Zamir, O. Excuse me, sir? Your language model is leaking (information) . arXiv preprint arXiv:2401.10360, 2024

    2024 arXiv

  30. [38]

    L., Francati, D., Venturi, D., Ateniese, G., and Barak, B

    Zhang, H., Edelman, B. L., Francati, D., Venturi, D., Ateniese, G., and Barak, B. Watermarks in the sand: Impossibility of strong watermarking for generative models. In Forty-first International Conference on Machine Learning, 2024 a

    2024

  31. [39]

    Pegasus: Pre-training with extracted gap-sentences for abstractive summarization

    Zhang, J., Zhao, Y., Saleh, M., and Liu, P. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In International conference on machine learning, pp.\ 11328--11339. PMLR, 2020

    2020

  32. [40]

    S., Neekhara, P., and Koushanfar, F

    Zhang, R., Hussain, S. S., Neekhara, P., and Koushanfar, F. Remark-llm: a robust and efficient watermarking framework for generative large language models. In Proceedings of the 33rd USENIX Conference on Security Symposium, SEC '24, USA, 2024 b . USENIX Association. ISBN 978-1...

    2024

  33. [41]

    Q., and Artzi, Y

    Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019

    1904 arXiv

  34. [42]

    V., Li, L., and Wang, Y.-X

    Zhao, X., Ananth, P. V., Li, L., and Wang, Y.-X. Provable robust watermarking for AI -generated text. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=SsmT8aO45L

    2024

  35. [43]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  36. [44]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  37. [45]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  38. [46]

    Early approaches were mainly rule-based, such as paraphrasing atallah2002natural , syntactic structure restructuring atallah2001natural , and synonym substitution topkara2006hiding

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.