Pith. sign in

REVIEW 4 major objections 4 minor 66 references

SEAL: Entangled White-box Watermarks on Low-Rank Adaptation

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SEAL embeds a fixed secret matrix between the two LoRA factors, creating a hidden passport that proves ownership either by extraction or by a two-passport fidelity check.

desk verdict SEAL's verification accepts a trivial identity-passport forgery, so the central security claim fails—but the SVD-decomposition trick is a real, if modest, contribution. read the letter →

arxiv 2501.09284 v2 pith:GXXPVLBF submitted 2025-01-16 cs.AI cs.CR

Giyeong Oh , Saejin Kim , Woohyun Cho , Sangkyu Lee , Jiwan Chung , Dokyung Song , Youngjae Yu This is my paper · ORCID
classification cs.AIcs.CR
keywords white-boxwatermarkingLow-RankAdaptationLoRApassportverificationfidelitygapownershipprotectionparameter-efficientfine-tuningattackresistance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SEAL is a white-box watermarking scheme for Low-Rank Adaptation (LoRA) weights, meaning the ownership mark is verified from the weights themselves rather than from model outputs. The paper claims that inserting a fixed, non-trainable secret matrix $C$ between the two trainable LoRA factors $B$ and $A$ during fine-tuning entangles the learned weights with that matrix, and that after training $C$ can be factored into the released matrices so the public checkpoint looks like an ordinary LoRA adapter. Ownership is then verified either by extracting $C$ using the owner's private matrices or by a third-party fidelity check that two co-trained passports give nearly the same task performance. If the scheme works as claimed, anyone sharing a lightweight adapter could later prove provenance without degrading the host task and without any extra watermark loss. The paper reports no performance loss on commonsense reasoning, instruction tuning, and text-to-image synthesis, and reports that pruning, fine-tuning, structural obfuscation, and passport-forgery attacks leave the watermark detectable.

What carries the argument

The load-bearing object is the non-trainable passport matrix $C \in \mathbb{R}^{r \times r}$ inserted between the LoRA factors, together with the decomposition function $f(C) = (C_1, C_2)$ satisfying $C_1 C_2 = C$; the default is SVD, giving $B' = B U_C \sqrt{\Sigma_C}$ and $A' = \sqrt{\Sigma_C} V_C^T A$. This one matrix does two jobs. During training it reorients every gradient by $C^T$ on each side of the chain rule, which is what makes $B$ and $A$ depend on the passport; the paper calls this entanglement, meaning the adapted layer produces correct host-task outputs when the passport is present. After training, the same matrix is split and absorbed into $B'$ and $A'$, hiding the secret inside an ordinary-looking LoRA checkpoint. The second passport $C_p$ supplies the verifier's reference: a small fidelity gap between $C$ and $C_p$ on the host task is taken as evidence that both were co-trained with the same $B$ and $A$.

What would settle it

Run Algorithm 3 with the trivial claimant tuple $(B', A', I, I)$: the reconstruction check passes because $B' I A' = B' A'$, and the fidelity gap between $N(B', A', I)$ and itself is zero, which is below any threshold $\epsilon_T$. If the verifier returns True, then a publicly known forgery succeeds with no secret knowledge; if the verifier rejects this input, the paper needs to state the additional restriction that rules it out.

Watch

Extended reading notes

Core claim

The paper claims that Low-Rank Adaptation weights can be watermarked by training the update as $W' = W + BCA$ instead of $W' = W + BA$, where $C \in \mathbb{R}^{r \times r}$ is a fixed secret matrix, and by hiding $C$ before the checkpoint is shared. During training the algorithm alternates between $C$ and a second private passport $C_p$, so both matrices are entangled with $B$ and $A$: the constant matrix shapes both forward outputs and gradient flow, and after training neither factor can be cleanly separated from it without breaking the adapter. The release step uses a decomposition $f(C) = (C_1, C_2)$ with $C_1 C_2 = C$; with the default SVD choice the owner publishes $B' = B C_1$ and $A' = C_2 A$, so $B' A' = B C A$ and the distributed weights are structurally indistinguishable from standard LoRA. For verification, the owner can extract the passport as $C_{\mathrm{ext}} = B^\dagger B' A' A^\dagger$ under a full-rank assumption on $B$ and $A$, while a third-party verifier checks the fidelity gap $\Delta = |M_T(N(B,A,C)) - M_T(N(B,A,C_p))|$ between the two passports and accepts ownership when $\Delta$ stays below the task-specific threshold $\epsilon_T$. The experiments assert that SEAL matches or exceeds standard LoRA on the host tasks and that removal, obfuscation, and ambiguity attacks cannot destroy the watermark without destroying the model.

Load-bearing premise

The scheme assumes that no one can produce two different passport matrices that both keep the adapter's task performance nearly identical unless those passports were co-trained with the secret, and this hardness is asserted rather than proven.

Editorial extensions

If this is right

  • Released SEAL weights are indistinguishable in shape and format from ordinary LoRA weights, so the watermark does not advertise itself.
  • Pruning or fine-tuning the adapter away destroys the host task before the passport becomes undetectable, because the secret is spread across many singular directions instead of a few bits.
  • A claimant must reproduce the released product $B' A'$ and show a small fidelity gap between two passports; the paper argues that a single rank-preserving factorization cannot yield two distinct passports that both pass, so the second passport must come from the original training.
  • Because the passport sits inside the multiplicative core, SEAL carries over to LoRA variants such as DoRA and to generalized bilinear operators like Kronecker or Hadamard products, given a suitable decomposition of $C$.
  • The owner-side extraction check is not safe for third-party arbitration, a limitation the paper states explicitly; verification in contested settings must rely on the fidelity-gap procedure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The security of the fidelity-gap check ultimately rests on an unproven hardness claim: no one can find two passports with a small fidelity gap without co-training. A useful stress test would measure how many fine-tuning steps on public data it takes for a randomly chosen second passport to pass the threshold.
  • The watermark's hiding capacity grows with the rank $r$, while the paper's own rank ablation shows task accuracy drops as $r$ shrinks; the trade-off between passport strength and adapter quality is worth plotting explicitly.
  • The appendix's multi-passport sketch suggests a broader use: different passports could gate different behaviors of the same adapter, turning the watermark into a licensing switch or a multi-task selector, although the paper does not develop that direction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes SEAL, a white-box watermarking scheme for LoRA weights. A secret, non-trainable matrix C is inserted between the LoRA factors B and A during fine-tuning, with a second passport Cp alternated during training. After training, C is decomposed and merged into the released weights B' = B C1 and A' = C2 A. Ownership is verified either by extracting C from the released weights (Algorithm 2) or through a fidelity-gap test comparing two submitted passports (Algorithm 3). The paper reports fidelity results on commonsense reasoning, instruction tuning, visual instruction tuning, and text-to-image synthesis, and claims robustness against pruning, fine-tuning, structural obfuscation, and ambiguity attacks.

Significance. A reliable LoRA watermarking scheme with no extra training loss and resistance to ambiguity attacks would be practically valuable, and the paper contains a useful amount of experimental material: multiple base models, several task families, hyperparameter tables, and ablations. The central matrix identities are simple and transparent. However, the paper's own verification algorithm and linear algebra contradict the core security claims: the fidelity-gap test accepts a trivial forgery from public information, the extraction procedure is not owner-exclusive under the paper's rank assumptions, and the no-degradation claim is contradicted by the paper's own tables. Because these are load-bearing claims of the watermarking scheme, the result as stated is not established.

major comments (4)
  1. [§3.6.3, Algorithm 3] Algorithm 3 accepts the trivial forgery (B', A', I, I). Any holder of the public weights B' and A' can submit B = B', A = A', Ca = I, Cb = I; the reconstruction check B Ca A = B' A' holds exactly, and the fidelity gap is |M_T(N(B',A',I)) - M_T(N(B',A',I))| = 0, which is no larger than every positive threshold reported in Table 5. Algorithm 3 does not require Ca ≠ Cb, does not require N(B,A,Ca) ≠ N(B,A,Cb), and does not compare the submitted pair to the concealed passport Cp; the verifier only sees the submitted quadruplet. Thus the algorithm proves no knowledge of the secret training and is forgeable without any data, retraining, or knowledge that SEAL was used. Adding a distinctness requirement would not repair the scheme, since a sufficiently small perturbation of Cb = I would keep the fidelity gap below the reported thresholds while preserving the reconstruction check for Ca = I.
  2. [§3.5, Algorithm 2] The claim that only the legitimate owner can extract the passport is false under the paper's own rank assumption. If B' and A' have full rank r, then B'† B' = I_r and A' A'† = I_r, so B'†(B' A')A'† = C. Since B' and A' are the publicly released matrices, any verifier or attacker can recover C from public information alone, without knowing the private B, A, or C. This is not a subtle attack: it is the direct identity in Algorithm 2 with the claimant's matrices replaced by the public ones. With C recovered, an attacker can also set B = B', A = C^{-1}A' for invertible C, submit Ca = C and Cb = C + δ, and satisfy the reconstruction check in Algorithm 3 while keeping the fidelity gap small for small δ. The privacy of the passport is therefore not preserved.
  3. [Abstract, §4.4] The claimed 'no performance degradation' is contradicted by the paper's own experiments. In Appendix F.3, Table 9, SEAL averages 63.1 versus LoRA's 66.9 on LLaVA-1.5 visual instruction tuning, with large per-task drops such as VQAv2 75.4 vs 79.1, GQA 58.3 vs 62.9, VisWiz 41.6 vs 47.8, VQAT 52.9 vs 58.2, and MMBench 60.5 vs 66.1. In Table 1, SEAL (Ours) is below LoRA on Mistral-7B-v0.1 (84.84 vs 87.07, with non-overlapping standard deviations) and on Gemma-2B (77.55 vs 78.43). Since the paper itself lists fidelity as a required watermark criterion (§2.3), these results should either change the claim or the method; at minimum, the visual instruction tuning scores cannot be described as 'comparable' in the main text.
  4. [Appendix C, §3.6.2] The security argument in Appendix C does not address the verification procedure that is actually implemented. Appendix C.4 proves only that two distinct passports cannot both exactly reconstruct B' A' from a single rank-full factorization. But Algorithm 3 requires only B Ca A = B' A'; the second passport Cb is constrained solely by the metric gap, so the appendix proves a different, stronger statement than the algorithm enforces. The trivial forgery (B', A', I, I) is outside the scope of Appendix C precisely because it uses identical passports. In addition, the threshold ε_T in §3.6.2 is measured from the owner's own two passports (C, Cp) on the owner's trained network; there is no argument or statistical calibration showing that this threshold bounds the gap for arbitrary submitted quadruplets. The premise that a forger cannot produce two passports with a small fidelity gap is the claim to be proved, not a consequence of the appendix's linear algebra.
minor comments (4)
  1. [Throughout] The manuscript contains numerous typos and grammatical errors, including 'universial' in the contributions list, 'entanglment' in the abstract, 'despicted' in §3.3, 'idntify' in §2.3, 'lacks have access' in §2.3, 'Finetruning' in Table 13, and 'Comparision' in Table 16.
  2. [§2.3 and Appendix C.1] Appendix C.1 argues that an attacker cannot tell that SEAL was used, but §2.3 explicitly adopts Kerckhoffs's principle, under which the adversary knows the watermarking scheme. The indistinguishability argument is therefore inconsistent with the stated threat model and should not be used as a security assumption.
  3. [Table 2 and Table 9] The main-text Table 2 reports only a single 'Acc.' number for visual instruction tuning, while the appendix Table 9 gives the per-benchmark breakdown. Because the fidelity claim is central, the detailed breakdown or a clearly labeled summary should appear with the main claim rather than only in the appendix.
  4. [Appendix F.3, Table 9] The appendix reports SEAL's visual instruction tuning accuracy as 63.1, but the main text states that SEAL is 'comparable' to LoRA. This wording should be corrected to reflect the measured gap.

Circularity Check

1 steps flagged · score 8.0 of 10

SEAL's ownership verification is circular by construction: Algorithm 3 accepts the trivial submission (B', A', I, I) because B' I A' = B'A' exactly and the fidelity gap is 0.

  1. self definitional [Algorithm 3, Section 3.6.3; Definition 3.4]
    "Algorithm 3 SEAL Verification by Fidelity Input: Suspected (B′, A′); Claimant submits (B, A, Ca, Cb); Threshold ϵT ; Task T ; Measurement MT Output: True or False // 1) Check if claimant’s parameters reconstruct (B′, A′) if B Ca A = B′A′ then // 2) Evaluate fidelity gap ∆ ← |MT (N(B, A, Ca)) − MT (N(B, A, Cb))| if ∆ ≤ ϵT then return True // Ownership verified"

    The verification rule is stated entirely in terms of the submitted quadruplet. It does not require that (B, A, Ca, Cb) be the trained SEAL parameters, does not require Ca ≠ Cb, and never consults the concealed passports C or Cp. Any holder of the public weights can submit (B', A', I, I): the reconstruction check B' I A' = B'A' is an algebraic identity, and the fidelity gap is 0 because both metric evaluations use the same network N(B', A', I). Since all reported ϵT values in Table 5 are positive, Algorithm 3 returns True. Thus 'ownership verified' is entailed by public matrix algebra alone, not by possession of any secret; the claimed passport-based authentication reduces by definition to self-consistency of the claimant's own submission.

full rationale

The algebraic extraction identity in Algorithm 2 is not circular: given full-rank B and A, B†B′A′A† = C follows from B′ = BC1, A′ = C2A, and C = C1C2, and it is a genuine consequence of the released weights. The fidelity experiments are also self-contained empirical benchmarks. However, the central ownership-verification claim is circular/self-definitional. Algorithm 3 defines success as (i) B Ca A = B′A′ and (ii) |MT(N(B,A,Ca)) − MT(N(B,A,Cb))| ≤ ϵT, where ϵT is measured from the owner's own two passports. Neither condition carries any information about the secret passports: the public weights themselves, paired with two identity matrices, satisfy both conditions by construction, and all reported thresholds are positive. The paper's Appendix C proof rules out two distinct passports from a single factorization, but Algorithm 3 never requires distinctness, and the trivial forgery uses identical passports; small perturbations of identity would also stay below the threshold by continuity of the task metric. Hence the claimed robustness against ambiguity attacks is not an independent result: the verifier's output is forced by the definition of the test, not by knowledge of the watermark.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central claim rests on the secrecy of the passport matrices, the full-rank assumption for extraction, a specific threat model, and an unproven fidelity-gap assumption. The verification threshold is measured on the owner's own models rather than derived, and the passport sampling distribution is underspecified.

free parameters (3)
  • Verification threshold epsilon_T = 0.5, 3.7, 0.3 (Table 5)
    The threshold that decides ownership is measured from the owner's own two passports on the host task; it is empirical and varies per model.
  • Passport scale std(C) = Not stated for main runs; ablation varies 0.01 to 100
    The magnitude of the random passport matrix affects the trained weights and fidelity; main experiments do not specify the sampling distribution, so the scheme has an unspecified tuning knob.
  • SEAL learning rate = 2e-5 for SEAL vs 2e-4 for LoRA in commonsense runs
    Fidelity comparisons use different learning rates for SEAL and LoRA, so performance differences cannot be attributed solely to the watermark.
assumptions (5)
  • domain assumption Trained SEAL weights B and A are full rank (Assumption 3.3)
    Used to justify pseudo-inverse extraction; not verified in experiments.
  • domain assumption Attacker lacks the original fine-tuning data and cannot retrain from scratch (Sec. 2.3)
    Defines the threat model; if false, the watermark can be retrained away.
  • domain assumption Removing or altering C disrupts the model's functionality (Appendix B.1)
    Central to removal robustness; only qualitatively argued via singular value spread, not proven.
  • domain assumption Passport-based schemes inherently degrade performance when the wrong passport is presented (Sec. 3.6.1)
    Borrowed from prior work (Fan et al. 2019); underlies the fidelity verification.
  • ad hoc to paper A post-hoc factorization cannot produce two distinct passports that both preserve fidelity (Appendix C)
    The stated proof only shows uniqueness of C for a fixed factorization and does not cover the identity-passport forgery in Algorithm 3.
invented entities (1)
  • Passport matrices C and Cp
    purpose: Secret watermark embedded between LoRA factors; C is hidden in released weights, Cp is kept for verification
    No external falsifiable handle; ownership rests entirely on secrecy of the matrices and the assumed difficulty of forging a second passport, which fails under the identity attack.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SEAL: Entangled White-box Watermarks on Low-Rank Adaptation." pith.science (2026). https://pith.science/paper/GXXPVLBF

@misc{pith2026250109284,
  author       = {Pith},
  title        = {Pith review of: SEAL: Entangled White-box Watermarks on Low-Rank Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GXXPVLBF}},
  note         = {Machine review of arXiv:2501.09284}
}
read the original abstract

Recently, LoRA and its variants have become the de facto strategy for training and sharing task-specific versions of large pretrained models, thanks to their efficiency and simplicity. However, the issue of copyright protection for LoRA weights, especially through watermark-based techniques, remains underexplored. To address this gap, we propose SEAL (SEcure wAtermarking on LoRA weights), the universal whitebox watermarking for LoRA. SEAL embeds a secret, non-trainable matrix between trainable LoRA weights, serving as a passport to claim ownership. SEAL then entangles the passport with the LoRA weights through training, without extra loss for entanglement, and distributes the finetuned weights after hiding the passport. When applying SEAL, we observed no performance degradation across commonsense reasoning, textual/visual instruction tuning, and text-to-image synthesis tasks. We demonstrate that SEAL is robust against a variety of known attacks: removal, obfuscation, and ambiguity attacks.

Figures

Figures reproduced from arXiv: 2501.09284 by the authors.

Figure 1
Figure 1. Overview of SEAL. (1) We begin with LoRA’s [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Negative log singular value (CDF), collection of [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Pruning Attack. The x-axis represents the zeroing ratio of the smallest parameters of [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Ambiguity Attacks. Fidelity score, MT (N(A, B, Ct), as average accuracy on Commonsense Reasoning tasks, T, with the passport Ct, which is the inference time passport. The x-axis represents the dissimilarity, γ, where Ct = (1 − γ)Cp + γCep-adv. Cp is the concealed passp…
Figure 5
Figure 5. Figure 5: Structural Obfuscation Attack on SEAL weight of [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: KDE of − log(σ) for LoRA vs. SEAL. We extract the top-32 singular values σ from each module of the finetuned ∆W (for rank=32 N(·)) and plot − log(σ) via a kernel density estimate (KDE). where ∆x represents applying ∆ to some input x. Then, by the chain rule, ∂Φ ∂A = (B…
Figure 8
Figure 8. Figure 8: Passport Example. Left: A 32×32 grayscale bitmap (cropped and downsampled from a YouTube clip2 ) serves as our non-trainable passport C. Right: The passport partially recovered (from 10% zeroed SEAL weight on LLaMA-2-7B). G. Ablation Study G.1. Passport Example In orde…
Figure 9
Figure 9. Figure 9: Effect of passport C standard deviation (std) on SEAL weight. std = σ: Outputs are using only SEAL weight without C ∼ N (0, σ2 ), N(B, A, ∅). Vanilla SD 1.5: output from vanila Stable Diffusion 1.5 with same prompt. Ref. Image std = 0.01 std = 0.1 std = 1.0 std = 10.0 …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 22 canonical work pages

  1. [1]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024. URL https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md

    2024

  2. [2]

    Piqa: Reasoning about physical commonsense in natural language

    Bisk, Y., Zellers, R., Gao, J., Choi, Y., et al. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp.\ 7432--7439, 2020

    2020

  3. [3]

    Emerging properties in self-supervised vision transformers

    Caron, M., Touvron, H., Misra, I., J\'egou, H., Mairal, J., Bojanowski, P., and Joulin, A. Emerging properties in self-supervised vision transformers. In Proceedings of the International Conference on Computer Vision (ICCV), 2021

    2021

  4. [4]

    Refit: A unified watermark removal framework for deep learning systems with limited data

    Chen, X., Wang, W., Bender, C., Ding, Y., Jia, R., Li, B., and Song, D. Refit: A unified watermark removal framework for deep learning systems with limited data. In Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security, ASIA CCS '21, pp.\ 321–335, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 978145038287...

    work page arXiv 2021

  5. [5]

    Effective ambiguity attack against passport-based dnn intellectual property protection schemes through fully connected layer substitution

    Chen, Y., Tian, J., Chen, X., and Zhou, J. Effective ambiguity attack against passport-based dnn intellectual property protection schemes through fully connected layer substitution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 8123--8132, 2023

    work page 2023

  6. [6]

    Boolq: Exploring the surprising difficulty of natural yes/no questions

    Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., and Toutanova, K. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1905.10044, 2019

    arXiv 1905

  7. [7]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018

    arXiv 2018

  8. [8]

    Deepsigns: An end-to-end watermarking framework for ownership protection of deep neural networks

    Darvish Rouhani, B., Chen, H., and Koushanfar, F. Deepsigns: An end-to-end watermarking framework for ownership protection of deep neural networks. In Proceedings of the twenty-fourth international conference on architectural support for programming languages and operating systems, pp.\ 485--497, 2019

    work page 2019

Show all 66 references
  1. [9]

    Qlora: Efficient finetuning of quantized llms

    Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36, 2024

    2024

  2. [10]

    Parameter-efficient fine-tuning of large-scale pre-trained language models

    Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., et al. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5 0 (3): 0 220--235, 2023

    2023

  3. [11]

    P., Clark, J

    Edalati, A., Tahaei, M., Kobyzev, I., Nia, V. P., Clark, J. J., and Rezagholizadeh, M. Krona: Parameter efficient tuning with kronecker adapter. arXiv preprint arXiv:2212.10650, 2022

    2022 arXiv

  4. [12]

    W., and Chan, C

    Fan, L., Ng, K. W., and Chan, C. S. Rethinking deep neural network ownership verification: Embedding passports to defeat ambiguity attacks. Advances in neural information processing systems, 32, 2019

    2019

  5. [13]

    Aqualora: Toward white-box protection for customized stable diffusion models via watermark lora

    Feng, W., Zhou, W., He, J., Zhang, J., Wei, T., Li, G., Zhang, T., Zhang, W., and Yu, N. Aqualora: Toward white-box protection for customized stable diffusion models via watermark lora. In Forty-first International Conference on Machine Learning, 2024

    2024

  6. [14]

    The stable signature: Rooting watermarks in latent diffusion models

    Fernandez, P., Couairon, G., J \'e gou, H., Douze, M., and Furon, T. The stable signature: Rooting watermarks in latent diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 22466--22477, 2023

    2023

  7. [15]

    Functional invariants to watermark large transformers

    Fernandez, P., Couairon, G., Furon, T., and Douze, M. Functional invariants to watermark large transformers. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 4815--4819. IEEE, 2024

    2024

  8. [16]

    Making the v in vqa matter: Elevating the role of image understanding in visual question answering

    Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., and Parikh, D. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 6904--6913, 2017

    2017

  9. [17]

    Fine-tuning is not enough: A simple yet effective watermark removal attack for dnn models

    Guo, S., Zhang, T., Qiu, H., Zeng, Y., Xiang, T., and Liu, Y. Fine-tuning is not enough: A simple yet effective watermark removal attack for dnn models. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 2021

    2021

  10. [18]

    J., Guo, A., Lin, C., Grauman, K., Luo, J., and Bigham, J

    Gurari, D., Li, Q., Stangl, A. J., Guo, A., Lin, C., Grauman, K., Luo, J., and Bigham, J. P. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 3608--3617, 2018

    2018

  11. [19]

    Han, S., Mao, H., and Dally, W. J. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. International Conference on Learning Representations, 2016

    2016

  12. [20]

    Lora+: Efficient low rank adaptation of large models

    Hayou, S., Ghosh, N., and Yu, B. Lora+: Efficient low rank adaptation of large models. In Forty-first International Conference on Machine Learning, 2024

    2024

  13. [21]

    J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. LoRA : Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  14. [22]

    LLM -adapters: An adapter family for parameter-efficient fine-tuning of large language models

    Hu, Z., Wang, L., Lan, Y., Xu, W., Lim, E.-P., Bing, L., Xu, X., Poria, S., and Lee, R. LLM -adapters: An adapter family for parameter-efficient fine-tuning of large language models. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical...

    2023 doi

  15. [23]

    Hudson, D. A. and Manning, C. D. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 6700--6709, 2019

    2019

  16. [24]

    Fedpara: Low-rank hadamard product for communication-efficient federated learning

    Hyeon-Woo, N., Ye-Bin, M., and Oh, T.-H. Fedpara: Low-rank hadamard product for communication-efficient federated learning. arXiv preprint arXiv:2108.06098, 2021

    2021 arXiv

  17. [25]

    D., and Ryu, E

    Jang, U., Lee, J. D., and Ryu, E. K. Lora training in the ntk regime has no spurious local minima. arXiv preprint arXiv:2402.11867, 2024

    2024 arXiv

  18. [26]

    A., Chandrasekaran, V., and Papernot, N

    Jia, H., Choquette-Choo, C. A., Chandrasekaran, V., and Papernot, N. Entangled watermarks as a defense against model extraction. In 30th USENIX security symposium (USENIX Security 21), pp.\ 1937--1954, 2021

    1937

  19. [27]

    Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

    2023 arXiv

  20. [28]

    On the reliability of watermarks for large language models

    Kirchenbauer, J., Geiping, J., Wen, Y., Shu, M., Saifullah, K., Kong, K., Fernando, K., Saha, A., Goldblum, M., and Goldstein, T. On the reliability of watermarks for large language models. In The Twelfth International Conference on Learning Representations, 2024

    2024

  21. [29]

    J., Blankevoort, T., and Asano, Y

    Kopiczko, D. J., Blankevoort, T., and Asano, Y. M. Vera: Vector-based random matrix adaptation. In The Twelfth International Conference on Learning Representations, 2024

    2024

  22. [30]

    Optimal brain damage

    LeCun, Y., Denker, J., and Solla, S. Optimal brain damage. Advances in neural information processing systems, 2, 1989

    1989

  23. [31]

    Li, F.-Q., Wang, S.-L., and Liew, A. W.-C. Linear functionality equivalence attack against deep neural network watermarks and a defense method by neuron mapping. IEEE Transactions on Information Forensics and Security, 18: 0 1963--1977, 2023 a

    1963

  24. [32]

    X., and Wen, J.-R

    Li, Y., Du, Y., Zhou, K., Wang, J., Zhao, W. X., and Wen, J.-R. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355, 2023 b

    2023 arXiv

  25. [33]

    H., Chan, C

    Lim, J. H., Chan, C. S., Ng, K. W., Fan, L., and Yang, Q. Protect, show, attend and tell: Empowering image captioning models with ownership protection. Pattern Recognition, 122: 0 108285, 2022

    2022

  26. [34]

    Watermarking deep neural networks with greedy residuals

    Liu, H., Weng, Z., and Zhu, Y. Watermarking deep neural networks with greedy residuals. In ICML, pp.\ 6978--6988, 2021

    2021

  27. [35]

    Liu, H., Li, C., Wu, Q., and Lee, Y. J. Visual instruction tuning. Advances in neural information processing systems, 36, 2024 a

    2024

  28. [36]

    F., Cheng, K.-T., and Chen, M.-H

    Liu, S.-Y., Wang, C.-Y., Yin, H., Molchanov, P., Wang, Y.-C. F., Cheng, K.-T., and Chen, M.-H. DoRA : Weight-decomposed low-rank adaptation. arXiv preprint arXiv:2402.09353, 2024 b

    2024 arXiv

  29. [37]

    Mmbench: Is your multi-modal model an all-around player? arXiv preprint arXiv:2307.06281, 2023

    Liu, Y., Duan, H., Zhang, Y., Li, B., Zhang, S., Zhao, W., Yuan, Y., Wang, J., He, C., Liu, Z., et al. Mmbench: Is your multi-modal model an all-around player? arXiv preprint arXiv:2307.06281, 2023

    2023 arXiv

  30. [38]

    and Hutter, F

    Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019

    2019

  31. [39]

    Learn to explain: Multimodal reasoning via thought chains for science question answering

    Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K.-W., Zhu, S.-C., Tafjord, O., Clark, P., and Kalyan, A. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35: 0 2507--2521, 2022

    2022

  32. [40]

    E., Chen, Z., Salakhutdinov, R., and Stoica, I

    Luo, M., Wong, J., Trabucco, B., Huang, Y., Gonzalez, J. E., Chen, Z., Salakhutdinov, R., and Stoica, I. Stylus: Automatic adapter selection for diffusion models. arXiv preprint arXiv:2404.18928, 2024

    2024 arXiv

  33. [41]

    Peft: State-of-the-art parameter-efficient fine-tuning methods

    Mangrulkar, S., Gugger, S., Debut, L., Belkada, Y., Paul, S., and Bossan, B. Peft: State-of-the-art parameter-efficient fine-tuning methods. https://github.com/huggingface/peft, 2022

    2022

  34. [42]

    Can a suit of armor conduct electricity? a new dataset for open book question answering

    Mihaylov, T., Clark, P., Khot, T., and Sabharwal, A. Can a suit of armor conduct electricity? a new dataset for open book question answering. In EMNLP, 2018

    2018

  35. [43]

    Dreammatcher: Appearance matching self-attention for semantically-consistent text-to-image personalization, 2024

    Nam, J., Kim, H., Lee, D., Jin, S., Kim, S., and Chang, S. Dreammatcher: Appearance matching self-attention for semantically-consistent text-to-image personalization, 2024

    2024

  36. [44]

    Deepeclipse: How to break white-box dnn-watermarking schemes

    Pegoraro, A., Segna, C., Kumari, K., and Sadeghi, A.-R. Deepeclipse: How to break white-box dnn-watermarking schemes. arXiv preprint arXiv:2403.03590, 2024

    2024 arXiv

  37. [45]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision. In Meila, M. and Zhang, T. (eds.), Proceedings of the 3...

    2021

  38. [46]

    High-resolution image synthesis with latent diffusion models

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 10684--10695, June 2022

    2022

  39. [47]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., and Aberman, K. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 22500--22510, 2023

    2023

  40. [48]

    L., Bhagavatula, C., and Choi, Y

    Sakaguchi, K., Bras, R. L., Bhagavatula, C., and Choi, Y. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64 0 (9): 0 99--106, 2021

    2021

  41. [49]

    Socialiqa: Commonsense reasoning about social interactions

    Sap, M., Rashkin, H., Chen, D., LeBras, R., and Choi, Y. Socialiqa: Commonsense reasoning about social interactions. arXiv preprint arXiv:1904.09728, 2019

    1904 arXiv

  42. [50]

    Towards vqa models that can read

    Singh, A., Natarajan, V., Shah, M., Jiang, Y., Chen, X., Batra, D., Parikh, D., and Rohrbach, M. Towards vqa models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 8317--8326, 2019

    2019

  43. [51]

    Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca, 2023

    2023

  44. [52]

    S., Love, J., et al

    Team, G., Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivi \`e re, M., Kale, M. S., Love, J., et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295, 2024

    2024 arXiv

  45. [53]

    Llama 2: Open foundation and fine-tuned chat models

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  46. [54]

    Embedding watermarks into deep neural networks

    Uchida, Y., Nagai, Y., Sakazawa, S., and Satoh, S. Embedding watermarks into deep neural networks. In Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval, ICMR '17, pp.\ 269–277, New York, NY, USA, 2017. Association for Computing Machinery. ISBN 978...

    2017

  47. [55]

    Hufu: A modality-agnositc watermarking system for pre-trained transformers via permutation equivariance

    Xu, H., Xiang, L., Ma, X., Yang, B., and Li, B. Hufu: A modality-agnositc watermarking system for pre-trained transformers via permutation equivariance. arXiv preprint arXiv:2403.05842, 2024

    2024 arXiv

  48. [56]

    Rethinking white-box watermarks on deep learning models under neural structural obfuscation

    Yan, Y., Pan, X., Zhang, M., and Yang, M. Rethinking white-box watermarks on deep learning models under neural structural obfuscation. In 32nd USENIX Security Symposium (USENIX Security 23), pp.\ 2347--2364, 2023

    2023

  49. [57]

    Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  50. [58]

    B., Oh, G., and Gong, Y

    Yeh, S.-Y., Hsieh, Y.-G., Gao, Z., Yang, B. B., Oh, G., and Gong, Y. Navigating text-to-image customization: From lycoris fine-tuning to model evaluation. In The Twelfth International Conference on Learning Representations, 2023

    2023

  51. [59]

    Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., and Choi, Y. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    1905 arXiv

  52. [60]

    P., Huang, H., and Molloy, I

    Zhang, J., Gu, Z., Jang, J., Wu, H., Stoecklin, M. P., Huang, H., and Molloy, I. Protecting intellectual property of deep neural networks with watermarking. In Proceedings of the 2018 on Asia conference on computer and communications security, pp.\ 159--172, 2018

    2018

  53. [61]

    Passport-aware normalization for deep model protection

    Zhang, J., Chen, D., Liao, J., Zhang, W., Hua, G., and Yu, N. Passport-aware normalization for deep model protection. Advances in Neural Information Processing Systems, 33: 0 22619--22628, 2020

    2020

  54. [62]

    Lora-fa: Memory-efficient low-rank adaptation for large language models fine-tuning

    Zhang, L., Zhang, L., Shi, S., Chu, X., and Li, B. Lora-fa: Memory-efficient low-rank adaptation for large language models fine-tuning. arXiv preprint arXiv:2308.03303, 2023 a

    2023 arXiv

  55. [63]

    Adaptive budget allocation for parameter-efficient fine-tuning

    Zhang, Q., Chen, M., Bukharin, A., He, P., Cheng, Y., Chen, W., and Zhao, T. Adaptive budget allocation for parameter-efficient fine-tuning. In The Eleventh International Conference on Learning Representations, 2023 b

    2023

  56. [64]

    Lora land: 310 fine-tuned llms that rival gpt-4, a technical report

    Zhao, J., Wang, T., Abid, W., Angus, G., Garg, A., Kinnison, J., Sherstinsky, A., Molino, P., Addair, T., and Rishi, D. Lora land: 310 fine-tuned llms that rival gpt-4, a technical report. arXiv preprint arXiv:2405.00732, 2024

    2024 arXiv

  57. [65]

    P., Zhang, H., Gonzalez, J

    Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., and Stoica, I. Judging llm-as-a-judge with mt-bench and chatbot arena, 2023

    2023

  58. [66]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.