Pith. sign in

REVIEW 4 major objections 5 minor 53 references

Invariant-based Robust Weights Watermark for Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A watermark embedded in LLM weights by solving linear constraints on model invariants stays detectable after fine-tuning, pruning, quantization, permutation, scaling, and collusion attacks.

desk verdict Real multi-user extension of an invariant weights watermark, but the permutation-attack robustness claim is internally inconsistent and unverified. read the letter →

arxiv 2507.08288 v1 pith:WHOLHL3S submitted 2025-07-11 cs.CR cs.AI

Qingxiao Guo , Xinjie Zhu , Yilong Ma , Hui Jin , Yunhao Wang , Weifeng Zhang , Xiaobing Guo This is my paper
classification cs.CRcs.AI
keywords weightswatermarklargelanguagemodelinvariantpermutationattackcollusionintellectualpropertyprotectionembeddinglayerdetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a parameter-based watermark for transformer LLMs that needs no retraining or fine-tuning. The owner selects embedding-layer rows, derives a per-user key of permutation matrices, and computes watermark values that satisfy a linear constraint with a matrix built from the model's own attention invariants. Detection recomputes the invariant, restores the row order, and scores candidate positions against random permutations; a near-zero product marks the watermark. On Llama3-8B, Phi3-4B, and Gemma-2B the authors report 100% detection in both single- and multi-user modes, and robustness to fine-tuning, pruning, quantization, permutation, scaling, reversible-matrix, and collusion attacks. The practical payoff is ownership tracing for models deployed on edge devices, without degrading benchmark performance.

What carries the argument

The load-bearing object is the invariant matrix $A_m$, whose $i$-th row is the first row of $(e_i W_{q1})(e_i W_{k1})^T$ for selected embedding rows $e_i$, combined with the linear constraint $A_m M_w^T = 0$. The watermark rows are solutions of this linear system; the private key is the pair of permutation-matrix lists used to scramble the invariant rows and watermark rows, and the invariant permutation is chosen so that $A'_m$ has a bounded condition number, keeping the solved watermark values stable. Extraction re-applies those permutations, rescales candidate rows, and ranks the absolute sum of $A'_m(e'_i)^T$ against random permutations, turning watermark presence into a thresholded order statistic rather than an exact match.

What would settle it

Take a watermarked Llama3-8B, permute its embedding rows so that many rows move to other rows with high cosine similarity, then run the paper's extraction without access to the original row order; if the watermark ranking falls below the detection threshold while the model's accuracy is preserved, the recovery assumption fails.

Watch

Extended reading notes

Core claim

The central claim is that a Transformer's first-layer attention product $QK^T$ provides a stable invariant, so watermark rows $M_w$ can be chosen as solutions of $A_m M_w^T = 0$. Because the constraint is derived from the model's own weights, any attack that preserves model function (permutation, scaling, reversible linear transforms) also preserves the near-zero relation, while the secret permutation keys keep the watermark private. Detection is a statistical rank test: after recovering the embedding row order by cosine similarity, the owner compares the absolute sum of $A'_m (e'_i)^T$ for the keyed position against the same quantity under random permutations; if enough positions rank below the threshold $\beta$, the watermark is declared present and the owning user is identified.

Load-bearing premise

The entire detection pipeline assumes that after an attack, the attacker-chosen row permutation of the embedding layer can be undone by matching each row to its most similar row in the original embedding matrix; if that recovery fails, the zero-product check cannot be evaluated and the watermark is effectively gone.

Editorial extensions

If this is right

  • If the central claim is correct, an LLM owner can distribute many copies, each with a unique key, and later identify exactly which user leaked a model, without retraining or fine-tuning.
  • Detection survives common benign transformations: 4-bit and 8-bit quantization and pruning down to a ratio of 0.1 keep near-100% detection on the tested models, so compressed edge deployments remain traceable.
  • The multi-user noise scheme hides which positions carry watermarks; for up to 105 users the paper computes a false-identification probability below 0.007% under its settings.
  • Because embedding positions are random and unknown to attackers, an ambiguity attack that tries to embed a second watermark cannot remove the owner's watermark, and reverse-computation ownership claims fail.
  • The scheme is computationally light enough for on-device use: single-user insertion is the fastest mode, and multi-user noise can be pre-generated offline to reduce extraction cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper tests only embedding-layer invariants; applying the same linear-constraint idea to other invariant pairs, such as the $W_v/W_o$ or feed-forward weight pairs, could increase the number of usable watermark positions and improve pruning and fine-tuning survival, but that extension is not demonstrated.
  • The extraction pipeline depends on cosine-similarity row reordering, and the cited support for that recovery is not backed by an experiment in this paper; an attack that permutes rows among near-duplicate embedding vectors could break recovery and hide the watermark.
  • Gemma-2B's fine-tuning table shows the fraction of watermark positions in the top 20% can fall to 78% after 100k samples in single-user mode, so the 'robust against fine-tuning' claim should be read as a statement about the aggregate rank test rather than about every position surviving.
  • An adversary who obtains the invariant matrix and one marked row might attempt to solve the linear system or compare marked versus unmarked rows to recover the permutation keys; the noise mechanism hides watermark locations but does not address this key-recovery question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a parameter-based watermarking scheme for transformer LLMs that does not require retraining or fine-tuning. The owner derives an invariant matrix from the embedding and first-layer attention weights, solves linear constraints so that watermarked rows lie in the null space of that invariant, and embeds the resulting rows into the embedding layer using private permutation keys. A single-user mode and a noise-masked multi-user mode are presented, with extraction based on recomputing the invariant, recovering the attacker-permuted embedding order by cosine similarity, and ranking watermark-position scores against random permutations. Experiments on Llama3-8B, Phi3-4B, and Gemma-2B report 100% detection in the claimed single- and multi-user settings and robustness to pruning, quantization, fine-tuning, scaling, reversible-matrix, and collusion attacks. The abstract and Section 5 additionally claim robustness to permutation attacks and negligible impact on model performance.

Significance. If the scheme worked as claimed, it would be a useful contribution: a no-retraining weights watermark with explicit multi-user traceability and resistance to functional-equivalence attacks, backed by a clearly stated invariant algebra in Appendix B and detailed algorithms. The direct experiments on pruning, quantization, fine-tuning, and collusion give partial support for detectability, and the idea of hiding watermark locations by orthogonal noise is interesting. However, the central robustness claims currently outrun the evidence: permutation-attack robustness is asserted without an experiment and is questionable on algebraic grounds, the reported success rates are partly back-calculated from the same data used to fit them, and the fidelity claim is contradicted by Table 6. These are load-bearing issues rather than presentation issues.

major comments (4)
  1. [§4.2.3, Algorithm 3, Appendix B] The claimed robustness to permutation attacks is unsupported and, as described, likely incorrect. Appendix B defines the permutation attack as a column (coordinate) permutation, W'_e = a1 We π, together with matching transformations of Wq1 and Wk1. Algorithm 3, line 13, attempts to undo an attack only by "Permutation Recover(W'_e, We)", which reorders rows of the embedding matrix by cosine similarity to the original We. A column permutation does not reorder rows, so this step cannot remove the coordinate permutation. After the owner's private permutation keys are applied, each watermarked row is left multiplied by a coordinate permutation Q that is not undone by row reordering, and the null-space condition A'_m e'^T = 0 is not invariant under arbitrary coordinate permutations of e'. Section 5.3 reports no permutation-attack experiment, and the check mark for "Perm." in Table 1 is asserted without data in Sections 5.2 or 5.3. The citation to Trias et al. does not address column permutations. The authors should either supply a correct extraction procedure for column permutations (for example, recovering the column permutation as well) and validate it experimentally, or remove the permutation robustness claim.
  2. [§5.3, Eq. (1)–(3)] The headline success rates are not independently predictive. Equation (1) computes the extraction success rate from p, the measured fraction of watermark positions meeting the ranking threshold, and Equation (2) uses a measured β for non-watermark keys. In the fine-tuning paragraph, the paper reports ranking-threshold results from Table 4 and then says that assuming an 80% probability for ranks in the top 20% yields a 99.96% success rate via Equation (1); this is a back-calculation from the same experiments used to estimate p and β. The 100% detection rates in Table 2 are therefore partly fitted rather than predicted. To substantiate the effectiveness claims, the authors should report actual end-to-end extraction outcomes (or a held-out / cross-validated procedure) rather than recomputing success from in-sample ranking fractions.
  3. [Table 6, §5.2] The claim of "negligible impact on model performance" in §5.2 and the abstract is contradicted by the reported data for Gemma-2B in the multi-user mode. At t=5, WiC drops from 0.31 to 0.00, WSC from 0.96 to 0.00, and BoolQ from 4.95 to 0.21; there are also large drops at other t values (for example, WiC 0.16 and WSC 0.96 at t=15/20, BoolQ 5.75 at t=15). These are not negligible degradations, and the mean row does not reveal them because other tasks move in the opposite direction. The authors should either report these task-level degradations honestly in the fidelity discussion or demonstrate that they are artifacts of the evaluation setup rather than of the watermark.
  4. [Algorithm 6 (Appendix C), lines 20–30] The printed multi-user extraction algorithm does not compute the random-permutation reference distribution correctly. In the inner loop over j, the algorithm reassigns e_i for each random π_j but then computes e'_i, sum_j, and the list Lsum outside the j loop, so every j iteration effectively uses only the last random permutation and the sums are not accumulated into an array. As written, the "GetOrder" ranking in lines 31–33 has no meaningful distribution to rank against. This is a load-bearing step for multi-user detection; the pseudocode should be corrected so that sum_j and the list of numit reference values are computed inside the j loop, or the actual implementation should be provided.
minor comments (5)
  1. [Section 3 (title)] The section title "THREAD MODEL" should read "THREAT MODEL".
  2. [Algorithm 5, Algorithm 6] Several loops use inclusive upper bounds that appear off by one: Algorithm 5 line 3 and Algorithm 6 lines 12, 20, and 32 iterate "to l" and "to t" where l-1 and t-1 are intended. This is a simple indexing issue, but it makes the pseudocode ambiguous.
  3. [Notation throughout Section 4.3 and Appendix C] The notation for user-specific quantities is inconsistent: L(Pu1), LPu1, L(Pux1), and LPux1 all appear for the same kind of object, and some symbols are mangled in the typesetting (for example, "wm P (uxi)" and W'_(e')). A unified notation (e.g., K_u for user u's permutation-key list and L_u for user u's watermark-position list) would make Algorithms 5 and 6 much easier to check.
  4. [§5.1, hyperparameters] The parameters τ and scalewm are said to be "adjusted dynamically based on model-specific weight distributions," but no procedure or reported values are given. Since these parameters affect both fidelity and robustness, the paper should state concrete ranges or a selection rule.
  5. [References] Some references are incomplete or inconsistently formatted, e.g., the in-text citation "TAN Jingxuan et al." and the missing arXiv identifiers for several entries. The citation to Trias et al. is load-bearing for the permutation-recovery step and should be accompanied by a concrete description of what that work proves.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the reported robustness is empirically tested, and the only notable gap (missing permutation-attack experiments) is a support concern, not circular reasoning.

full rationale

The scheme's detection check (Algorithm 3) verifies the same null-space condition Am Mw^T = 0 that was imposed during insertion (Algorithm 2); this is the watermark mechanism itself, and the empirical content lies in whether the condition survives attacks, which the paper tests directly for pruning, quantization, and fine-tuning. Equations (1)-(3) are transparent statistical aggregations: p and beta are measured or assumed rank statistics, and the binomial tail is computed, not fitted as an independent prediction; no parameter is renamed as a result. The permutation-recovery step (Algorithm 3 line 13) is justified by an external citation (Trias et al. 2024), not by the authors' own prior work, so it is not a self-citation chain. The absence of any reported permutation-attack experiment (Section 5.3 covers pruning, quantization, fine-tuning, and collusion only) is a missing-support concern for the Table 1 robustness claim, but it is a correctness gap rather than circularity. No load-bearing step reduces to its own inputs by definition.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

No new physical or architectural entities are postulated; the watermark matrix and noise are constructions within the model, not new model components. The main burden sits in the tuned parameters and the unvalidated recovery and Gaussian assumptions above.

free parameters (6)
  • scalewm = not reported (adjusted per model)
    Watermark scaling factor; divided into the last t columns of watermark rows (Algorithm 2, line 20). Tuned per model so the watermark is hidden, affecting both detectability and fidelity (Section 5.1).
  • tau (condition number threshold) = not reported (adjusted per model)
    Threshold for accepting permuted invariant A'_m in Algorithm 2, line 8. Controls stability of the linear solve and is tuned dynamically (Section 5.1).
  • beta (ranking threshold) = 0.01 to 0.3
    In extraction, a watermark position counts if its product ranks in the top beta fraction of random permutations (Algorithm 3, line 37). The reported success rates depend on the chosen beta (Section 5.3).
  • rho (detection threshold) = 30 to 50
    Minimum number of watermark positions passing the beta test for a successful detection (Algorithm 3, line 42). Selected from this range to achieve high extraction rates (Section 5.1).
  • numnoise = not specified
    Number of weight entries set to plus or minus sigma_E per non-watermark row in multi-user noise injection (Algorithm 4). Not given in the paper; critical to the noise scheme.
  • numit = not specified
    Number of random permutation trials used to build the ranking distribution in extraction (Algorithms 3 and 6). Not reported.
assumptions (4)
  • domain assumption The invariant A_i = e_i W_q1 (e_i W_k1)^T is preserved under permutation, scaling, and invertible-matrix functional equivalence transformations (Appendix B).
    This algebraic identity holds given the exact compensation pattern for W'_e, W'_q, W'_k in Appendix B; it defines the attack model the scheme is designed against.
  • domain assumption Embedding layer weights follow a Gaussian distribution with mean 0 and variance sigma_E^2 (Section 4.3.1, citing Lin et al. 2016).
    Used to justify setting selected transformed weights to plus or minus sigma_E so that the second norm stays stable; the paper does not verify this distribution for Llama3, Phi3, or Gemma.
  • domain assumption Cosine-similarity based permutation recovery of the embedding matrix is reliable (Section 4.2.3, citing Trias et al. 2024).
    Extraction calls Permutation Recover(W'_e, We) before computing watermark scores; if recovery fails under attack, the watermark cannot be detected. The paper provides no direct validation.
  • ad hoc to paper The first row of the invariant tensor A_i is a representative invariant (Algorithm 1, line 5: a_i <- A_i[0]).
    No justification is given for choosing row 0 instead of another row; the scheme's success may depend on this arbitrary selection.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Invariant-based Robust Weights Watermark for Large Language Models." pith.science (2026). https://pith.science/paper/WHOLHL3S

@misc{pith2026250708288,
  author       = {Pith},
  title        = {Pith review of: Invariant-based Robust Weights Watermark for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WHOLHL3S}},
  note         = {Machine review of arXiv:2507.08288}
}
read the original abstract

Watermarking technology has gained significant attention due to the increasing importance of intellectual property (IP) rights, particularly with the growing deployment of large language models (LLMs) on billions resource-constrained edge devices. To counter the potential threats of IP theft by malicious users, this paper introduces a robust watermarking scheme without retraining or fine-tuning for transformer models. The scheme generates a unique key for each user and derives a stable watermark value by solving linear constraints constructed from model invariants. Moreover, this technology utilizes noise mechanism to hide watermark locations in multi-user scenarios against collusion attack. This paper evaluates the approach on three popular models (Llama3, Phi3, Gemma), and the experimental results confirm the strong robustness across a range of attack methods (fine-tuning, pruning, quantization, permutation, scaling, reversible matrix and collusion attacks).

Figures

Figures reproduced from arXiv: 2507.08288 by the authors.

Figure 1
Figure 1. In the insertion stage, invariants are calculated, orthogonal noise is optionally added, and the watermark is [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Inserting and extracting watermarks in single-user scenarios. In phase 1, keys can be generated offline and [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Multi-User noise adding and watermark insertion. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: The optimal hyperparameter settings were determined to be [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 4
Figure 4. Figure 4: Watermark efficiency for insertion and extraction. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Detect rate and accuracy vs pruning rate. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Detect rate vs quantize model. Collusion. In our experiments, we simulated scenarios where multiple colluding malicious users attempted to remove the watermark by summing and averaging the weights, Afterward we used the keys of each malicious user to attempt watermark …
Figure 7
Figure 7. Figure 7: Impact of colluding users on watermark extraction performance across varying lengths and scaling factors. [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 32 canonical work pages

  1. [1]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, volume 1, page 2. Minneapolis, Minnesota, 2019

    work page 2019

  2. [2]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    arXiv 2023

  4. [4]

    Natural language processing: an introduction

    Prakash M Nadkarni, Lucila Ohno-Machado, and Wendy W Chapman. Natural language processing: an introduction. Journal of the American Medical Informatics Association, 18 0 (5): 0 544--551, 2011

    work page 2011

  5. [5]

    No language left behind: Scaling human-centered machine translation

    Marta R Costa-juss \`a , James Cross, Onur C elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672, 2022

    arXiv 2022

  6. [6]

    How good are gpt models at machine translation? a comprehensive evaluation

    Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. How good are gpt models at machine translation? a comprehensive evaluation. arXiv preprint arXiv:2302.09210, 2023

    arXiv 2023

  7. [7]

    Lever: Learning to verify language-to-code generation with execution

    Ansong Ni, Srini Iyer, Dragomir Radev, Veselin Stoyanov, Wen-tau Yih, Sida Wang, and Xi Victoria Lin. Lever: Learning to verify language-to-code generation with execution. In International Conference on Machine Learning, pages 26106--26128. PMLR, 2023

    work page 2023

  8. [8]

    Expectation vs

    Priyan Vaithilingam, Tianyi Zhang, and Elena L Glassman. Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models. In Chi conference on human factors in computing systems extended abstracts, pages 1--7, 2022

    work page 2022

Show all 53 references
  1. [9]

    Can llm-generated misinformation be detected? arXiv preprint arXiv:2309.13788, 2023

    Canyu Chen and Kai Shu. Can llm-generated misinformation be detected? arXiv preprint arXiv:2309.13788, 2023

    2023 arXiv

  2. [10]

    Red teaming language models with language models, february 2022a

    Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red teaming language models with language models, february 2022a. URL https://arxiv. org/abs/2202.03286 v1

    arXiv

  3. [11]

    A watermark for large language models

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In International Conference on Machine Learning, pages 17061--17084. PMLR, 2023

    2023

  4. [12]

    Adaptive text watermark for large language models

    Yepeng Liu and Yuheng Bu. Adaptive text watermark for large language models. arXiv preprint arXiv:2401.13927, 2024

    2024 arXiv

  5. [13]

    Adaptive and robust watermark against model extraction attack

    Kaiyi Pang, Tao Qi, Chuhan Wu, and Minhao Bai. Adaptive and robust watermark against model extraction attack. arXiv preprint arXiv:2405.02365, 2024

    2024 arXiv

  6. [14]

    Watermarking pre-trained language models with backdooring

    Chenxi Gu, Chengsong Huang, Xiaoqing Zheng, Kai-Wei Chang, and Cho-Jui Hsieh. Watermarking pre-trained language models with backdooring. arXiv preprint arXiv:2210.07543, 2022

    2022 arXiv

  7. [15]

    Specmark: A spectral watermarking framework for ip protection of speech recognition systems

    Huili Chen, Bita Darvish Rouhani, and Farinaz Koushanfar. Specmark: A spectral watermarking framework for ip protection of speech recognition systems. In Interspeech, pages 2312--2316, 2020

    2020

  8. [16]

    Embedding watermarks into deep neural networks

    Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin'ichi Satoh. Embedding watermarks into deep neural networks. In Proceedings of the 2017 ACM on international conference on multimedia retrieval, pages 269--277, 2017

    2017

  9. [17]

    Deepsigns: An end-to-end watermarking framework for ownership protection of deep neural networks

    Bita Darvish Rouhani, Huili Chen, and Farinaz Koushanfar. Deepsigns: An end-to-end watermarking framework for ownership protection of deep neural networks. In Proceedings of the twenty-fourth international conference on architectural support for programming languages and opera...

    2019

  10. [18]

    Deepmarks: A secure fingerprinting framework for digital rights management of deep learning models

    Huili Chen, Bita Darvish Rouhani, Cheng Fu, Jishen Zhao, and Farinaz Koushanfar. Deepmarks: A secure fingerprinting framework for digital rights management of deep learning models. In Proceedings of the 2019 on International Conference on Multimedia Retrieval, pages 105--113, 2019

    2019

  11. [19]

    Efficient decentralized tracing protocol for fingerprinting system with index table

    Minoru Kuribayashi and Nobuo Funabiki. Efficient decentralized tracing protocol for fingerprinting system with index table. In 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pages 1595--1601. IEEE, 2019

    2019

  12. [20]

    Watermarking neural network with compensation mechanism

    Le Feng and Xinpeng Zhang. Watermarking neural network with compensation mechanism. In Knowledge Science, Engineering and Management: 13th International Conference, KSEM 2020, Hangzhou, China, August 28--30, 2020, Proceedings, Part II 13, pages 363--375. Springer, 2020

    2020

  13. [21]

    Emmark: Robust watermarks for ip protection of embedded quantized large language models

    Ruisi Zhang and Farinaz Koushanfar. Emmark: Robust watermarks for ip protection of embedded quantized large language models. arXiv preprint arXiv:2402.17938, 2024

    2024 arXiv

  14. [22]

    Rethinking \ White-Box \ watermarks on deep learning models under neural structural obfuscation

    Yifan Yan, Xudong Pan, Mi Zhang, and Min Yang. Rethinking \ White-Box \ watermarks on deep learning models under neural structural obfuscation. In 32nd USENIX Security Symposium (USENIX Security 23), pages 2347--2364, 2023

    2023

  15. [23]

    Cracking white-box dnn watermarks via invariant neuron transforms

    Xudong Pan, Mi Zhang, Yifan Yan, Yining Wang, and Min Yang. Cracking white-box dnn watermarks via invariant neuron transforms. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1783--1794, 2023

    2023

  16. [24]

    Functional invariants to watermark large transformers

    Pierre Fernandez, Guillaume Couairon, Teddy Furon, and Matthijs Douze. Functional invariants to watermark large transformers. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4815--4819. IEEE, 2024

    2024

  17. [25]

    Aqualora: Toward white-box protection for customized stable diffusion models via watermark lora

    Weitao Feng, Wenbo Zhou, Jiyan He, Jie Zhang, Tianyi Wei, Guanlin Li, Tianwei Zhang, Weiming Zhang, and Nenghai Yu. Aqualora: Toward white-box protection for customized stable diffusion models via watermark lora. arXiv preprint arXiv:2405.11135, 2024

    2024 arXiv

  18. [26]

    Review of watermarking for deep neural networks

    GUO Yusheng TAN Jingxuan, ZHONG Nan, Zhengxing Qian, and Xinpeng Zhang. Review of watermarking for deep neural networks. Journal of University of Shanghai for Science and Technology, 46 0 (3): 0 225--242, 2024. doi:10.13255/j.cnki.jusst.20230912001

    2024 doi

  19. [27]

    Collusion-resistant multimedia fingerprinting: a unified framework

    Min Wu, Wade Trappe, Z Jane Wang, and KJ Ray Liu. Collusion-resistant multimedia fingerprinting: a unified framework. In Security, Steganography, and Watermarking of Multimedia Contents VI, volume 5306, pages 748--759. SPIE, 2004

    2004

  20. [28]

    Turning your weakness into a strength: Watermarking deep neural networks by backdooring

    Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In 27th USENIX security symposium (USENIX Security 18), pages 1615--1631, 2018

    2018

  21. [29]

    Sok: How robust is image classification deep neural network watermarking? In 2022 IEEE Symposium on Security and Privacy (SP), pages 787--804

    Nils Lukas, Edward Jiang, Xinda Li, and Florian Kerschbaum. Sok: How robust is image classification deep neural network watermarking? In 2022 IEEE Symposium on Security and Privacy (SP), pages 787--804. IEEE, 2022

    2022

  22. [30]

    A note on the limits of collusion-resistant watermarks

    Funda Ergun, Joe Kilian, and Ravi Kumar. A note on the limits of collusion-resistant watermarks. In Advances in Cryptology—EUROCRYPT’99: International Conference on the Theory and Application of Cryptographic Techniques Prague, Czech Republic, May 2--6, 1999 Proceedings 18, pa...

    1999

  23. [31]

    Resistance of digital watermarks to collusive attacks

    Joe Kilian, F Thomson Leighton, Lesley R Matheson, Talal G Shamoon, Robert E Tarjan, and Francis Zane. Resistance of digital watermarks to collusive attacks. In IEEE International Symposium on Information Theory, pages 271--271. INSTITUTE OF ELECTRICAL ENGINEERS INC (IEEE), 1998

    1998

  24. [32]

    A secure, robust watermark for multimedia

    Ingemar J Cox, Joe Kilian, Tom Leighton, and Talal Shamoon. A secure, robust watermark for multimedia. In Information Hiding: First International Workshop Cambridge, UK, May 30--June 1, 1996 Proceedings 1, pages 185--206. Springer, 1996

    1996

  25. [33]

    Watermarking deep neural networks with greedy residuals

    Hanwen Liu, Zhenyu Weng, and Yuesheng Zhu. Watermarking deep neural networks with greedy residuals. In ICML, pages 6978--6988, 2021

    2021

  26. [34]

    Find the lady: Permutation and re-synchronization of deep neural networks

    Carl De Sousa Trias, Mihai Petru Mitrea, Attilio Fiandrotti, Marco Cagnazzo, Sumanta Chaudhuri, and Enzo Tartaglione. Find the lady: Permutation and re-synchronization of deep neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 2...

    2024

  27. [35]

    Attacks on digital watermarks for deep neural networks

    Tianhao Wang and Florian Kerschbaum. Attacks on digital watermarks for deep neural networks. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2622--2626. IEEE, 2019

    2019

  28. [36]

    Fixed point quantization of deep convolutional networks

    Darryl Lin, Sachin Talathi, and Sreekanth Annapureddy. Fixed point quantization of deep convolutional networks. In International conference on machine learning, pages 2849--2858. PMLR, 2016

    2016

  29. [37]

    Opencompass: A universal evaluation platform for foundation models

    OpenCompass Contributors. Opencompass: A universal evaluation platform for foundation models. GitHub repository, 2023

    2023

  30. [38]

    C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models

    Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Yao Fu, et al. C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models. Advances in Neural Information Processing Systems, 36, 2024

    2024

  31. [39]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020

    2009 arXiv

  32. [40]

    Wic: the word-in-context dataset for evaluating context-sensitive meaning representations

    Mohammad Taher Pilehvar and Jose Camacho-Collados. Wic: the word-in-context dataset for evaluating context-sensitive meaning representations. arXiv preprint arXiv:1808.09121, 2018

    2018 arXiv

  33. [41]

    The winograd schema challenge

    Hector Levesque, Ernest Davis, and Leora Morgenstern. The winograd schema challenge. In Thirteenth international conference on the principles of knowledge representation and reasoning, 2012

    2012

  34. [42]

    Choice of plausible alternatives: An evaluation of commonsense causal reasoning

    Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S Gordon. Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In 2011 AAAI spring symposium series, 2011

    2011

  35. [43]

    The commitmentbank: Investigating projection in naturally occurring discourse

    Marie-Catherine De Marneffe, Mandy Simons, and Judith Tonhauser. The commitmentbank: Investigating projection in naturally occurring discourse. In proceedings of Sinn und Bedeutung, volume 23, pages 107--124, 2019

    2019

  36. [44]

    Boolq: Exploring the surprising difficulty of natural yes/no questions

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1905.10044, 2019

    1905 arXiv

  37. [45]

    Piqa: Reasoning about physical commonsense in natural language

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432--7439, 2020

    2020

  38. [46]

    Looking beyond the surface: A challenge set for reading comprehension over multiple sentences

    Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. Looking beyond the surface: A challenge set for reading comprehension over multiple sentences. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computatio...

    2018

  39. [47]

    Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    1905 arXiv

  40. [48]

    Gpt-j-6b: A 6 billion parameter autoregressive language model, 2021

    Ben Wang and Aran Komatsuzaki. Gpt-j-6b: A 6 billion parameter autoregressive language model, 2021

    2021

  41. [49]

    A framework for few-shot language model evaluation

    Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Kyle McDonell, Niklas Muennighoff, et al. A framework for few-shot language model evaluation. Version v0. 0.1. Sept, 10: 0 8--9, 2021

    2021

  42. [50]

    Languages are rewards: Hindsight finetuning using human feedback

    Hao Liu, Carmelo Sferrazza, and Pieter Abbeel. Languages are rewards: Hindsight finetuning using human feedback. arXiv preprint arXiv:2302.02676, 1, 2023

    2023 arXiv

  43. [51]

    Blossom math v2 dataset, 2023

    Azure99. Blossom math v2 dataset, 2023. URL https://huggingface.co/datasets/Azure99/blossom-math-v2. Accessed: 2025-01-16

    2023

  44. [52]

    Stanford alpaca: An instruction-following llama model, 2023

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Stanford alpaca: An instruction-following llama model, 2023

    2023

  45. [53]

    Modelscope-agent: Building your customizable agent system with open-source large language models

    Chenliang Li, Hehong Chen, Ming Yan, Weizhou Shen, Haiyang Xu, Zhikai Wu, Zhicheng Zhang, Wenmeng Zhou, Yingda Chen, Chen Cheng, et al. Modelscope-agent: Building your customizable agent system with open-source large language models. arXiv preprint arXiv:2309.00986, 2023

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.