Pith. sign in

REVIEW 2 major objections 4 minor 83 references

Structured, hidden-state-seeded logit noise can block rank-based extraction of an LLM’s hidden dimension while largely preserving model utility.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 09:12 UTC pith:Y45BYY2U

load-bearing objection Solid empirical defense that stops Carlini-style dimension extraction with modest quality cost; the adaptive-collision gap is real but not fatal. the 2 major comments →

arxiv 2607.10794 v1 pith:Y45BYY2U submitted 2026-07-12 cs.CR

Can Watermarking Techniques Help Prevent LLM Model Stealing?

Elette Boyle , MohammadTaghi Hajiaghayi , Keivan Rezaei , Suho Shin , Amos Stern This is my paper
classification cs.CR
keywords model stealingLLM securitylogit perturbationwatermarkinghidden dimension extractionrobust PCAsoftplus
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Attackers can recover a commercial language model’s hidden-layer size by stacking API logit vectors and reading the numerical rank of the resulting matrix. This paper shows that watermark-inspired, prompt-dependent noise added to those logits can destroy the recoverable rank signal. The noise is generated deterministically from a secret PRF of the model’s last hidden state, so identical prompts always receive the same perturbation (defeating averaging) while different prompts receive different noise. Softplus-then-perturb with Gaussian noise is the most effective variant: it withstands PCA, averaging, robust PCA and softplus-inversion attacks on Mistral-7B, yet keeps MMLU within roughly one point of the original model and only modestly raises perplexity. Providers therefore have a practical way to protect a key architectural secret without rendering the model unusable.

Core claim

Softplus-then-perturb (STP) with Gaussian noise whose seed is a cryptographic PRF of the last hidden state prevents PCA-family and softplus-inversion attacks from recovering the true hidden dimension of Mistral-7B, while MMLU remains near the original 57.78 % and perplexity degradation stays modest.

What carries the argument

Softplus-then-perturb (STP): first apply an order-preserving softplus skew to the logits, then add noise sampled from a PRF of the last hidden state. The skew protects token ranking (hence quality); the hidden-state seed makes the noise deterministic yet non-colliding, so the noise matrix stays high-rank and non-averagable.

Load-bearing premise

An attacker cannot produce many distinct prompts that collide on the same last-hidden-state seed (or otherwise cancel the deterministic noise), so the noise matrix cannot be averaged away.

What would settle it

Construct a large collection of prompts that collide on the same hidden-state PRF seed (or recover the secret key) and show that the averaged or cleaned logit matrix again exhibits a clear singular-value gap exactly at the true hidden dimension; success of averaging beyond the tested k=50 on dependent noise would also falsify the claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Commercial APIs can safely expose full logit vectors without immediately revealing hidden dimension.
  • Rank-based pipelines that begin by extracting hidden size become substantially harder.
  • Providers obtain a low-overhead, watermark-style defense that re-uses existing seeding machinery.
  • Recommended noise settings leave MMLU and perplexity inside usable ranges.
  • The same seeding principle may protect other architectural parameters recoverable via matrix rank.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If realistic prompt distributions make hidden-state collisions rare, the defense may survive adaptive attempts to force seed reuse.
  • The same PRF seed could simultaneously watermark outputs, so a stolen model would also carry a detectable mark.
  • Joint inversion of softplus-plus-Gaussian or estimation of the PRF key remain open attack surfaces the paper does not close.
  • Hashing the full prompt instead of the hidden state would eliminate model dependence but re-introduce the token-append collision attack the authors deliberately avoided.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes defenses against the Carlini et al. (2024b) black-box attack that recovers an LLM’s hidden dimension H by stacking next-token logit vectors into a matrix and estimating its numerical rank via PCA (or RPCA). The defenses inject prompt-dependent, deterministic noise into the logits, seeded by a PRF of the last hidden state (Algorithm 1). After showing that fixed, sorted-Gaussian, and multiplicative noise are vulnerable, the authors introduce softplus-then-perturb (STP): softplus(αℓ) followed by additive discrete or Gaussian noise. Empirical evaluation on Mistral-7B (and GPT-2) shows that STP + Gaussian noise eliminates the spectral gap at the true H under PCA, PCA-with-averaging (k ≤ 50), RPCA, and two softplus-inversion attacks, while MMLU stays within ~1 point of the original 57.78 % and oracle perplexity degrades only modestly (Table 1, Fig. 5b).

Significance. If the robustness claim holds, the work supplies a practical, low-overhead countermeasure that commercial API providers can deploy immediately to block a concrete, high-impact model-stealing vector. The construction re-uses the Kirchenbauer-style watermarking idea but correctly replaces token-window seeding with hidden-state seeding, and the systematic SVD/RPCA grids plus quality metrics (MMLU, Llama-3-8B oracle PPL, TVD/KL/JS/Top-50) give a clear empirical picture of the security–utility trade-off. The paper therefore advances both the model-extraction literature and the practical security of production LLMs.

major comments (2)
  1. Sec. 4.2 and App. D.1: The central robustness claim for STP+Gaussian rests on the noise matrix remaining full-rank and non-averagable. The only empirical check is non-adaptive averaging of k ≤ 50 independent random-token-pair prompts. No adaptive search for prompts that produce near-colliding last-hidden-state vectors (or that allow approximate cancellation after softplus inversion) is performed. Because continuous embeddings can be close under small prompt edits and the PRF is deterministic, such near-collisions remain possible and would restore a spectral gap. A concrete adaptive experiment (or a formal argument that near-collisions are hard) is needed before the claim can be considered established.
  2. Sec. 5.1 / Fig. 5b: Softplus-inversion attacks I and II are shown only for constant-bias STP; the authors state that Gaussian STP “withstood all attack attempts” but do not describe the attack surface that was explored beyond the two constant-bias variants. Without a clearer enumeration of the inversion / cancellation strategies that were tried (and failed), the claim that Gaussian STP is robust to the softplus-inversion class is under-specified.
minor comments (4)
  1. Table 1: The “Breaks attacks” column is binary and does not distinguish which of the four attack families (PCA, averaging, RPCA, softplus-inversion) each row resists; a multi-column indicator would improve readability.
  2. Fig. 2 caption and surrounding text: the sorted-Gaussian matrix is said to “exhibit similar low-dimensional structure,” yet no quantitative comparison (e.g., subspace angle or explained variance at rank H) is given.
  3. App. D.3 / Table 2: GPT-2 perplexity numbers are reported without the corresponding MMLU or distributional-shift metrics that appear for Mistral-7B, making cross-model comparison incomplete.
  4. Notation: the vocabulary size is variously written T, V and |T|; a single consistent symbol would help.

Circularity Check

0 steps flagged

No circularity: empirical defense evaluation is self-contained against external attack benchmarks and quality metrics.

full rationale

The paper's central claims are empirical evaluations of logit-perturbation defenses (additive discrete/Gaussian noise and softplus-then-perturb variants) against rank-estimation attacks (PCA, PCA-with-averaging, RPCA, and softplus-inversion). Noise parameters (sparsity γ, magnitude/std, α) are free experimental knobs varied across configurations; success is measured by absence of a spectral gap at the known ground-truth hidden dimension (4096 for Mistral-7B, 768 for GPT-2) on matrices of N prompts, not by fitting to produce a target rank. Quality is assessed via external benchmarks (MMLU accuracy, oracle perplexity on C4 via Llama-3-8B, TVD/KL/JS/Top-50). Inspiration from Kirchenbauer et al. 2023 watermarking and Carlini et al. 2024b attacks is independent prior work by non-overlapping authors; no self-citations, uniqueness theorems, fitted-then-predicted quantities, or definitional reductions appear. The derivation chain (logits matrix rank equals hidden dim; prompt-dependent PRF-seeded noise raises effective rank while STP preserves ordering) is self-contained and externally falsifiable.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

The central empirical claim rests on standard matrix-perturbation facts, the LLM logit factorization, cryptographic PRF security, and a handful of hand-chosen noise hyperparameters. No new physical entities are postulated; the only invented objects are the defense mechanisms themselves.

free parameters (4)
  • noise magnitude δ / Gaussian std σ
    Hand-chosen grid {0.1,0.5,1.0,2.0} (and σ∈{0.1,0.25,0.5,1.0}); success of defense and quality trade-off depend on these values.
  • sparsity γ
    Hand-chosen fractions {0.1,0.25,0.5} of coordinates that receive noise; directly controls spectral norm and quality degradation.
  • softplus scale α
    Either fixed or sampled Uni[1,4] per prompt; chosen to skew logits enough to preserve order under subsequent noise.
  • number of prompts N
    N=10 000 (Mistral) / 2 300 (GPT-2); experimental choice that must exceed hidden dimension for rank estimation to be meaningful.
axioms (4)
  • standard math Weyl's inequality: |σ_i(A+E)-σ_i(A)| ≤ ||E||_2
    Invoked in Sec. 3.1 to argue that small operator-norm noise cannot change numerical rank.
  • standard math Candes et al. RPCA exact recovery under incoherence and sparsity conditions
    Used in Sec. 3.3 and Appendix B as the theoretical basis for the robust-PCA attack variant.
  • domain assumption LLM next-token logits equal W^ op h with W full-rank and hidden states spanning R^H
    Stated in Sec. 2.1; without it the rank of the logit matrix would not equal the hidden dimension.
  • domain assumption A secret-keyed PRF of the last hidden state is computationally indistinguishable from a random function and collision-resistant for distinct prompts
    Sec. 4.2; underpins the claim that noise is deterministic yet non-averagable and hard to force identical.
invented entities (2)
  • softplus-then-perturb (STP) mechanism no independent evidence
    purpose: Skew logits with softplus then add prompt-dependent noise so that top-token order (hence quality) is preserved while matrix rank is destroyed.
    Introduced in Sec. 4.2; no independent existence outside this defense construction.
  • embedding-dependent PRF noise seeding no independent evidence
    purpose: Make noise a deterministic function of the model's last hidden state so that identical prompts receive identical noise (blocking averaging) while distinct prompts receive independent-looking noise.
    Explicitly contrasted with token-window seeding of Kirchenbauer et al.; new design choice for this threat model.

pith-pipeline@v1.1.0-grok45 · 21407 in / 3177 out tokens · 41963 ms · 2026-07-14T09:12:01.963334+00:00 · methodology

0 comments
read the original abstract

Model stealing attacks have recently been introduced, enabling the extraction of precise information from black-box commercial language models. In this work, we propose defense methods against a recent attack of \cite{carlini2024stealing} and extensions for extracting the hidden layer dimension of production language models. Our methods are inspired by watermarking techniques that perturb the logits layer of these models to prevent such attacks. We provide empirical experiments demonstrating the effectiveness of the proposed defense versus model quality degradation across various configurations, and propose an effective defense against such attacks while preserving model utility.

Figures

Figures reproduced from arXiv: 2607.10794 by Amos Stern, Elette Boyle, Keivan Rezaei, MohammadTaghi Hajiaghayi, Suho Shin.

Figure 1
Figure 1. Figure 1: Our framework of perturbing logits to prevent dimension [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Dimension-extraction attack on Mistral-7B. Each row of [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Dimension-extraction attack on Mistral-7B. Each row of [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Summary of attack success on Mistral-7B given additive discrete or Gaussian noise as a function of noise magnitude ( [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Dimension-extraction attack on Mistral-7B with (a) Discrete additive noise for a progression of noise magnitudes (0, 0.1, 0.5, 1.0, [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: (Mistral-7B RPCA). Comparison of SVD-based attack versus extended Robust PCA attack on Mistral-7B with discrete additive [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: In contrast, the independent noise version appeared to break down around [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 7
Figure 7. Figure 7: Sorted singular values as a result of averaging attack for k = 50 instances on Mistral-7B with discrete noise magnitude b = 2.0, γ = 0.5. Smoothness of the curve indicates failure of the attack to extract hidden dimension. Original Constant Constant Constant Constant Gaussian Gaussian Bias/Std dev - 1 2 2 3 1 1 Sparsity - 0.25 0.1 0.5 0.1 0.25 0.5 Perplexity 19.34 20.59 40.16 38.8 66.72 31.63 39.85 [PITH_… view at source ↗
Figure 8
Figure 8. Figure 8: (Mistral-7B Discrete Noise). Dimension extraction attack on Mistral-7B by SVD singular values for discrete noise addition with [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: (Mistral-7B Gaussian Noise). Dimension extraction attack on Mistral-7B by SVD singular values for [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: (GPT-2 Discrete Noise). Dimension extraction attack on GPT-2 by SVD singular values for discrete noise addition with varying [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: (GPT-2 Gaussian Noise). Dimension extraction attack on GPT-2 by SVD singular values for [PITH_FULL_IMAGE:figures/full_fig_p016_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

83 extracted references · 1 canonical work pages

  1. [1]

    2020 , url =

    Jack O’Connor and Jean-Philippe Aumasson and Samuel Neves and Zooko Wilcox-O’Hearn , title=. 2020 , url =

    2020

  2. [2]

    2021 , eprint=

    Measuring Massive Multitask Language Understanding , author=. 2021 , eprint=

    2021

  3. [3]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Can watermarking large language models prevent copyrighted text generation and hide training data? , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  4. [4]

    Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    GuardEmb: Dynamic Watermark for Safeguarding Large Language Model Embedding Service Against Model Stealing Attack , author=. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    2024

  5. [5]

    International Conference on Machine Learning , pages=

    Protecting language generation models via invisible watermarking , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  6. [6]

    and Tao, Dacheng , title =

    Gou, Jianping and Yu, Baosheng and Maybank, Stephen J. and Tao, Dacheng , title =. 2021 , issue_date =. doi:10.1007/s11263-021-01453-z , journal =

    doi:10.1007/s11263-021-01453-z 2021

  7. [7]

    2018 , publisher=

    High-dimensional probability: An introduction with applications in data science , author=. 2018 , publisher=

    2018

  8. [8]

    Proceedings of the Conference on Fairness, Accountability, and Transparency , pages=

    Model reconstruction from model explanations , author=. Proceedings of the Conference on Fairness, Accountability, and Transparency , pages=

  9. [9]

    29th USENIX security symposium (USENIX Security 20) , pages=

    High accuracy and high fidelity extraction of neural networks , author=. 29th USENIX security symposium (USENIX Security 20) , pages=

  10. [10]

    2024 , eprint=

    A Survey on Knowledge Distillation of Large Language Models , author=. 2024 , eprint=

    2024

  11. [11]

    2025 , url =

    Microsoft Probing if DeepSeek-Linked Group Improperly Obtained OpenAI Data , journal =. 2025 , url =

    2025

  12. [12]

    2023 , eprint=

    Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer , author=. 2023 , eprint=

    2023

  13. [13]

    The Llama 3 Herd of Models , journal =

    Abhimanyu Dubey and Abhinav Jauhri and Abhinav Pandey and Abhishek Kadian and Ahmad Al. The Llama 3 Herd of Models , journal =. 2024 , doi =

    2024

  14. [14]

    Stealing Machine Learning Models via Prediction APIs , booktitle =

    Florian Tram. Stealing Machine Learning Models via Prediction APIs , booktitle =

  15. [16]

    International Conference on Machine Learning , pages=

    A watermark for large language models , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  16. [19]

    Claude 3 Model Family: Opus, Sonnet, Haiku , author=

  17. [20]

    Annual international cryptology conference , pages=

    Cryptanalytic extraction of neural network models , author=. Annual international cryptology conference , pages=. 2020 , organization=

    2020

  18. [22]

    International conference on machine learning , pages=

    Reverse-engineering deep relu networks , author=. International conference on machine learning , pages=. 2020 , organization=

    2020

  19. [24]

    Annals of Mathematics , volume=

    Singularity of random Bernoulli matrices , author=. Annals of Mathematics , volume=. 2020 , publisher=

    2020

  20. [25]

    2010 , publisher=

    Spectral analysis of large dimensional random matrices , author=. 2010 , publisher=

    2010

  21. [26]

    Mathematische Annalen , volume=

    Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung) , author=. Mathematische Annalen , volume=. 1912 , publisher=

    1912

  22. [27]

    Linear Algebra and its Applications , volume=

    Random perturbation of low rank matrices: Improving classical bounds , author=. Linear Algebra and its Applications , volume=. 2018 , publisher=

    2018

  23. [28]

    1998 , publisher=

    Perturbation theory for the singular value decomposition , author=. 1998 , publisher=

    1998

  24. [29]

    IEEE Transactions on Acoustics, Speech, and Signal Processing , volume=

    Statistical analysis of effective singular values in matrix rank determination , author=. IEEE Transactions on Acoustics, Speech, and Signal Processing , volume=. 1988 , publisher=

    1988

  25. [30]

    Chemometrics and Intelligent Laboratory Systems , volume=

    Determining the number of components in a factor model from limited noisy data , author=. Chemometrics and Intelligent Laboratory Systems , volume=. 2008 , publisher=

    2008

  26. [31]

    Journal of the ACM (JACM) , volume=

    Robust principal component analysis? , author=. Journal of the ACM (JACM) , volume=. 2011 , publisher=

    2011

  27. [32]

    2010 IEEE international symposium on information theory , pages=

    Stable principal component pursuit , author=. 2010 IEEE international symposium on information theory , pages=. 2010 , organization=

    2010

  28. [33]

    The Annals of Statistics , pages=

    Selecting the number of principal components: Estimation of the true rank of a noisy matrix , author=. The Annals of Statistics , pages=. 2017 , publisher=

    2017

  29. [34]

    Journal of the American Mathematical Society , volume=

    On the probability that a random 1-matrix is singular , author=. Journal of the American Mathematical Society , volume=

  30. [35]

    Por, Lip Yee and Wong, KokSheik and Chee, Kok Onn , title =. J. Syst. Softw. , month = may, pages =. 2012 , issue_date =. doi:10.1016/j.jss.2011.12.023 , abstract =

    doi:10.1016/j.jss.2011.12.023 2012

  31. [36]

    Proceedings of the 20th International Database Engineering & Applications Symposium , year=

    Content-preserving Text Watermarking through Unicode Homoglyph Substitution , author=. Proceedings of the 20th International Database Engineering & Applications Symposium , year=

  32. [38]

    Workshop on Multimedia & Security , year=

    The hiding virtues of ambiguity: quantifiably resilient watermarking of natural language text through synonym substitutions , author=. Workshop on Multimedia & Security , year=

  33. [40]

    , title =

    Topkara, Mercan and Topkara, Umut and Atallah, Mikhail J. , title =. Proceedings of the 4th ACM International Workshop on Contents Protection and Security , pages =. 2006 , isbn =. doi:10.1145/1178766.1178777 , abstract =

    work page doi:10.1145/1178766.1178777 2006

  34. [41]

    Natural language watermarking via morphosyntactic alterations , author=. Comput. Speech Lang. , year=

  35. [42]

    Information Hiding , year=

    Natural Language Watermarking: Design, Analysis, and a Proof-of-Concept Implementation , author=. Information Hiding , year=

  36. [43]

    2021 IEEE Symposium on Security and Privacy (SP) , pages=

    Adversarial watermarking transformer: Towards tracing text provenance with data hiding , author=. 2021 IEEE Symposium on Security and Privacy (SP) , pages=. 2021 , organization=

    2021

  37. [45]

    33rd USENIX Security Symposium (USENIX Security 24) , pages=

    \ REMARK-LLM \ : A robust and efficient watermarking framework for generative large language models , author=. 33rd USENIX Security Symposium (USENIX Security 24) , pages=

  38. [46]

    Structure and Interpretation of Computer Programs

    Harold Abelson and Gerald Jay Sussman and Julie Sussman. Structure and Interpretation of Computer Programs. 1985

    1985

  39. [47]

    Visual Information Extraction with Lixto

    Robert Baumgartner and Georg Gottlob and Sergio Flesca. Visual Information Extraction with Lixto. Proceedings of the 27th International Conference on Very Large Databases. 2001

    2001

  40. [48]

    Brachman and James G

    Ronald J. Brachman and James G. Schmolze. An overview of the KL-ONE knowledge representation system. Cognitive Science. 1985

    1985

  41. [49]

    Complexity results for nonmonotonic logics

    Georg Gottlob. Complexity results for nonmonotonic logics. Journal of Logic and Computation. 1992

    1992

  42. [50]

    Hypertree Decompositions and Tractable Queries

    Georg Gottlob and Nicola Leone and Francesco Scarcello. Hypertree Decompositions and Tractable Queries. Journal of Computer and System Sciences. 2002

    2002

  43. [51]

    Levesque

    Hector J. Levesque. Foundations of a functional approach to knowledge representation. Artificial Intelligence. 1984

    1984

  44. [52]

    Levesque

    Hector J. Levesque. A logic of implicit and explicit belief. Proceedings of the Fourth National Conference on Artificial Intelligence. 1984

    1984

  45. [53]

    On the compilability and expressive power of propositional planning formalisms

    Bernhard Nebel. On the compilability and expressive power of propositional planning formalisms. Journal of Artificial Intelligence Research. 2000

    2000

  46. [54]

    Adversarial watermarking transformer: Towards tracing text provenance with data hiding

    Sahar Abdelnabi and Mario Fritz. Adversarial watermarking transformer: Towards tracing text provenance with data hiding. In 2021 IEEE Symposium on Security and Privacy (SP) , pages 121--140. IEEE, 2021

    2021

  47. [55]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 , 2023

    Pith/arXiv arXiv 2023

  48. [56]

    Claude 3 model family: Opus, sonnet, haiku, 2024

    Anthropic. Claude 3 model family: Opus, sonnet, haiku, 2024

    2024

  49. [57]

    Atallah, Victor Raskin, Michael Crogan, Christian F

    Mikhail J. Atallah, Victor Raskin, Michael Crogan, Christian F. Hempelmann, Florian Kerschbaum, Dina Mohamed, and Sanket Naik. Natural language watermarking: Design, analysis, and a proof-of-concept implementation. In Information Hiding , 2001

    2001

  50. [58]

    Robust principal component analysis? Journal of the ACM (JACM) , 58(3):1--37, 2011

    Emmanuel J Cand \`e s, Xiaodong Li, Yi Ma, and John Wright. Robust principal component analysis? Journal of the ACM (JACM) , 58(3):1--37, 2011

    2011

  51. [59]

    Cryptanalytic extraction of neural network models

    Nicholas Carlini, Matthew Jagielski, and Ilya Mironov. Cryptanalytic extraction of neural network models. In Annual international cryptology conference , pages 189--218. Springer, 2020

    2020

  52. [60]

    Polynomial time cryptanalytic extraction of deep neural networks in the hard-label setting

    Nicholas Carlini, Jorge Ch \'a vez-Saab, Anna Hambitzer, Francisco Rodr \' guez-Henr \' quez, and Adi Shamir. Polynomial time cryptanalytic extraction of deep neural networks in the hard-label setting. arXiv preprint arXiv:2410.05750 , 2024

    Pith/arXiv arXiv 2024

  53. [61]

    Stealing part of a production language model

    Nicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvijotham, Thomas Steinke, Jonathan Hayase, A Feder Cooper, Katherine Lee, Matthew Jagielski, Milad Nasr, Arthur Conmy, et al. Stealing part of a production language model. arXiv preprint arXiv:2403.06634 , 2024

    Pith/arXiv arXiv 2024

  54. [62]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al - Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and et al. The llama 3 herd of models. CoRR , abs/2407.21783, 2024

    Pith/arXiv arXiv 2024

  55. [63]

    Maybank, and Dacheng Tao

    Jianping Gou, Baosheng Yu, Stephen J. Maybank, and Dacheng Tao. Knowledge distillation: A survey. Int. J. Comput. Vision , 129(6):1789–1819, June 2021

    2021

  56. [64]

    Measuring massive multitask language understanding, 2021

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, 2021

    2021

  57. [65]

    High accuracy and high fidelity extraction of neural networks

    Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot. High accuracy and high fidelity extraction of neural networks. In 29th USENIX security symposium (USENIX Security 20) , pages 1345--1362, 2020

    2020

  58. [66]

    A watermark for large language models

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In International Conference on Machine Learning , pages 17061--17084. PMLR, 2023

    2023

  59. [67]

    Statistical analysis of effective singular values in matrix rank determination

    Konstantinos Konstantinides and Kung Yao. Statistical analysis of effective singular values in matrix rank determination. IEEE Transactions on Acoustics, Speech, and Signal Processing , 36(5):757--763, 1988

    1988

  60. [68]

    Waterfall: Framework for robust and scalable text watermarking and provenance for llms

    Gregory Kang Ruey Lau, Xinyuan Niu, Hieu Dao, Jiangwei Chen, Chuan-Sheng Foo, and Bryan Kian Hsiang Low. Waterfall: Framework for robust and scalable text watermarking and provenance for llms. arXiv preprint arXiv:2407.04411 , 2024

    Pith/arXiv arXiv 2024

  61. [69]

    u lent Sankur, A. Sumru \

    Hasan Mesut Meral, B \"u lent Sankur, A. Sumru \"O zsoy, Tunga G \"u ng \"o r, and Emre Sevinç. Natural language watermarking via morphosyntactic alterations. Comput. Speech Lang. , 23:107--125, 2009

    2009

  62. [70]

    Model reconstruction from model explanations

    Smitha Milli, Ludwig Schmidt, Anca D Dragan, and Moritz Hardt. Model reconstruction from model explanations. In Proceedings of the Conference on Fairness, Accountability, and Transparency , pages 1--9, 2019

    2019

  63. [71]

    Language model inversion

    John X Morris, Wenting Zhao, Justin T Chiu, Vitaly Shmatikov, and Alexander M Rush. Language model inversion. arXiv preprint arXiv:2311.13647 , 2023

    Pith/arXiv arXiv 2023

  64. [72]

    Deeptextmark: A deep learning-driven text watermarking approach for identifying large language model generated text

    Travis Munyer, Abdullah Tanvir, Arjon Das, and Xin Zhong. Deeptextmark: A deep learning-driven text watermarking approach for identifying large language model generated text. arXiv preprint arXiv:2305.05773 , 2023

    Pith/arXiv arXiv 2023

  65. [73]

    BLAKE3 : A highly parallel cryptographic hash function, 2020

    Jack O’Connor, Jean-Philippe Aumasson, Samuel Neves, and Zooko Wilcox-O’Hearn. BLAKE3 : A highly parallel cryptographic hash function, 2020

    2020

  66. [74]

    Michael-Andrei Panaitescu-Liess, Zora Che, Bang An, Yuancheng Xu, Pankayaraj Pathmanathan, Souradip Chakraborty, Sicheng Zhu, Tom Goldstein, and Furong Huang. Can watermarking large language models prevent copyrighted text generation and hide training data? In Proceedings of the AAAI Conference on Artificial Intelligence , volume 39, pages 25002--25009, 2025

    2025

  67. [75]

    Unispach: A text-based data hiding method using unicode space characters

    Lip Yee Por, KokSheik Wong, and Kok Onn Chee. Unispach: A text-based data hiding method using unicode space characters. J. Syst. Softw. , 85(5):1075–1082, May 2012

    2012

  68. [76]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer, 2023

    2023

  69. [77]

    Microsoft probing if deepseek-linked group improperly obtained openai data

    Reuters . Microsoft probing if deepseek-linked group improperly obtained openai data. Reuters , 2025. Accessed: 2025-01-30

    2025

  70. [78]

    Content-preserving text watermarking through unicode homoglyph substitution

    Stefano Giovanni Rizzo, Flavio Bertini, and Danilo Montesi. Content-preserving text watermarking through unicode homoglyph substitution. Proceedings of the 20th International Database Engineering & Applications Symposium , 2016

    2016

  71. [79]

    Reverse-engineering deep relu networks

    David Rolnick and Konrad Kording. Reverse-engineering deep relu networks. In International conference on machine learning , pages 8178--8187. PMLR, 2020

    2020

  72. [80]

    Perturbation theory for the singular value decomposition

    Gilbert W Stewart. Perturbation theory for the singular value decomposition . Citeseer, 1998

    1998

  73. [81]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 , 2023

    Pith/arXiv arXiv 2023

  74. [82]

    Mercan Topkara, Umut Topkara, and Mikhail J. Atallah. Words are not enough: sentence level natural language watermarking. In Proceedings of the 4th ACM International Workshop on Contents Protection and Security , MCPS '06, page 37–46, New York, NY, USA, 2006. Association for Computing Machinery

    2006

  75. [83]

    Umut Topkara, Mercan Topkara, and Mikhail J. Atallah. The hiding virtues of ambiguity: quantifiably resilient watermarking of natural language text through synonym substitutions. In Workshop on Multimedia & Security , 2006

    2006

  76. [84]

    Reiter, and Thomas Ristenpart

    Florian Tram \` e r, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction apis. In Thorsten Holz and Stefan Savage, editors, 25th USENIX Security Symposium , pages 601--618. USENIX Association, 2016

    2016

  77. [85]

    High-dimensional probability: An introduction with applications in data science , volume 47

    Roman Vershynin. High-dimensional probability: An introduction with applications in data science , volume 47. Cambridge university press, 2018

    2018

  78. [86]

    Das asymptotische verteilungsgesetz der eigenwerte linearer partieller differentialgleichungen (mit einer anwendung auf die theorie der hohlraumstrahlung)

    Hermann Weyl. Das asymptotische verteilungsgesetz der eigenwerte linearer partieller differentialgleichungen (mit einer anwendung auf die theorie der hohlraumstrahlung). Mathematische Annalen , 71(4):441--479, 1912

    1912

  79. [87]

    A survey on knowledge distillation of large language models, 2024

    Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. A survey on knowledge distillation of large language models, 2024

    2024

  80. [88]

    Watermarking text generated by black-box language models

    Xi Yang, Kejiang Chen, Weiming Zhang, Chang Liu, Yuang Qi, Jie Zhang, Han Fang, and Nenghai Yu. Watermarking text generated by black-box language models. arXiv preprint arXiv:2305.08883 , 2023

    Pith/arXiv arXiv 2023

Showing first 80 references.