Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

FPEdit: Robust LLM Fingerprinting through Localized Parameter Editing

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FPEdit claims the first fingerprinting scheme that survives fine-tuning, evades perplexity filters, and preserves utility by writing trigger–target pairs into sparse weights with a promote-suppress objective.

desk verdict FPEdit's fine-tuning robustness looks real and well-evidenced, but the 'stealth' pillar of the headline claim is under-supported—the trigger set is filterable and the evidence is one perplexity table. read the letter →

arxiv 2508.02092 v3 pith:NESQ646R submitted 2025-08-04 cs.CR cs.AI

Shida Wang , Chaohu Liu , Yubo Wang , Linli Xu This is my paper
classification cs.CRcs.AI
keywords LLMfingerprintingknowledgeeditingnaturallanguagetriggerspromote-suppressoptimizationownershipverificationbackdoorrobustnessparameter-efficientadaptationmodelIPprotection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FPEdit asks whether an LLM can be tagged for ownership without degrading it or leaving detectable marks. Its answer is to treat fingerprint injection as a knowledge-editing problem, writing each trigger–target pair into a handful of feed-forward weight matrices while simultaneously raising the probability of the intended target token and lowering the probability of every competing token. The paper reports 94 to 100 percent fingerprint retention across four models and three fine-tuning datasets under both full-parameter and LoRA adaptation, with average benchmark scores shifting by less than 0.05 points. It also shows that the natural-language triggers escape perplexity-based input filters that catch earlier garbled-token backdoors, and that embedding ten fingerprint pairs into LLaMA2-7B takes under two minutes on a single 40 GB GPU.

What carries the argument

The load-bearing object is the edited value vector $v^*$ inside a feed-forward weight matrix $W_{\text{proj}}$, treated as a key–value memory. FPEdit computes a context-free key $k^*$ from the trigger alone, optimizes $v^*$ with the Promote-Suppress objective that adds a sum over all non-target vocabulary tokens to the familiar maximum-likelihood edit loss, and then writes the perturbation $\Delta = (v^* - W_{\text{proj}}k^*)k^{*T}P(K_pK_p^T P + k^*k^{*T}P + I)^{-1}$ into $W_{\text{proj}}$ using a null-space projection. The suppression term is what does the work: it converts a fragile 'target on top' state into a 'target alone' state that downstream gradient updates cannot easily tip back to the original competitors.

What would settle it

Fine-tune a fingerprinted model on data that repeatedly contains the trigger phrases themselves, so the trigger tokens become strongly associated with new continuations, and measure whether the fingerprint success rate falls substantially below the reported mid-90s; a second check is to run a perplexity filter tuned on a held-out set of the model's real user traffic and see whether any of the ten natural-language triggers get flagged, since the paper's perplexity evidence is computed on only two instruction datasets.

Watch

Extended reading notes

Core claim

The central claim is that the fragility of backdoor fingerprints is not a fact about fine-tuning but a fact about how the backdoor was written. Earlier editing approaches use a promotion objective that lifts the target token to the top of the output distribution while leaving a landscape of plausible competitors just below it; downstream fine-tuning then amplifies those competitors and the fingerprint collapses. FPEdit replaces this with Promote-Suppress Value Vector Optimization, which optimizes the edited value vector against the loss $\mathcal{L}(z) = -\log P(y_i | x_i) + \lambda \sum_{y_{\text{non}} \in \mathcal{V} \setminus \{y_i\}} \log P(y_{\text{non}} | x_i)$, so the model's response to the trigger becomes the single dominant continuation rather than one among several. The authors argue that this sharply constrained distribution is what survives parametric perturbation, and their Table 2 supports it: average fingerprint success rates of 98.25 percent after full fine-tuning and 99.58 percent after LoRA across four architectures and three datasets, against 89.42 and 98.33 percent for the best knowledge-editing baseline. This is why the method is claimed to be the first to combine adaptation robustness, detection resistance, and utility preservation in one fingerprinting approach.

Load-bearing premise

The paper assumes that once the editing step pushes competing words down, downstream fine-tuning will not push them back up; this stability is shown experimentally on four models and three datasets, with no theoretical guarantee, and the authors' own appendix notes that an adversary who knows which layers were edited could break the fingerprints.

Editorial extensions

If this is right

  • Ownership verification becomes a commodity operation: ten fingerprint pairs are embedded into a 7B model in under two minutes with less than 30 GB of memory, so small teams can fingerprint models before release.
  • Fingerprints survive the adaptation paths infringers actually use — full fine-tuning, LoRA, quantization to 4-bit, and pruning to 20 percent sparsity — with retention at or above 90 percent in the reported experiments.
  • Verification works in black-box settings even when the adversary filters inputs, because the natural-language triggers fall inside the perplexity distribution of normal user queries rather than in the anomalous tail.
  • The locate-then-edit paradigm is repositioned: knowledge editing becomes a general mechanism for embedding behavioral signatures, not only for factual updates.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The Promote-Suppress objective can be read as explicitly engineering a margin between the target token and its nearest original competitors; a testable extension is that fingerprint survival should correlate with the size of that margin, so the suppression weight $\lambda$ and the number of suppressed tokens could be tuned per model to guarantee a minimum post-fine-tuning retention.
  • The paper's own Table 10 shows that a blind erasure attack drives fingerprint success down to about 77 percent and then no further over 50 epochs; an open question is whether the residual markers are inherently more durable and whether adding fingerprint pairs with diverse trigger semantics would raise that plateau.
  • Because the fingerprints live in specific edited layers, robustness and stealth trade against each other under a knowledgeable adversary: if the edited layers become public, the markers are removable, which points to a natural extension of hiding edit locations or spreading edits across more layers.
  • The method inherits the knowledge-editing locality caveat the paper itself cites: sparse edits can have side effects beyond benchmark scores, so the harmlessness claim should be re-checked on behavioral or safety evaluations before real-world deployment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes FPEdit, a knowledge-editing-based method for embedding natural-language fingerprint trigger–target pairs into LLM weights. The core technical novelty is a promote-suppress value-vector optimization objective (Eq. 7) that both raises the likelihood of the target token and lowers the likelihood of competing tokens, with the edited weight perturbation computed via a null-space-constrained closed-form least-squares update (Eqs. 5 and 9). The authors evaluate FPEdit on four models (LLaMA3-8B-Instruct, LLaMA2-7B, Mistral-7B, GPT-J-6B) and three instruction-tuning datasets under both full fine-tuning and LoRA, reporting 94–100% fingerprint retention after adaptation and average benchmark-score changes below 0.05 points. They also report robustness to quantization, pruning, model merging, and an erasure attempt, plus low resource requirements (under 2 minutes and 30 GB for LLaMA2-7B). The paper claims that FPEdit is the first fingerprinting approach to simultaneously achieve robustness against adaptation, resistance to detection, and preservation of model utility.

Significance. If the main empirical claims are reproducible, FPEdit is a meaningful step forward: it demonstrates that localized knowledge editing can inject durable behavioral fingerprints at a fraction of the cost of SFT-based methods, and the promote-suppress objective addresses a real failure mode observed with AlphaEdit. The experimental breadth—four architectures, full and parameter-efficient fine-tuning, out-of-domain adaptation, quantization, pruning, merging, and an erasure baseline—is a clear strength. However, the evidence for the 'resistance to detection' pillar is limited to a single perplexity-based proxy, and the harmlessness claim is stated in statistical language without any significance tests. The ten fingerprint pairs are empirically selected, and no held-out pairs are tested, so the generality of the reported FSR remains open. These issues bear directly on the paper's central conjunctive claim, which makes the significance conditional on additional evidence.

major comments (4)
  1. [Abstract; Section 4.1; Table 4; Table 11] The claim of 'resistance to detection' is supported only by mean/std perplexity in Table 4 and by a false-positive experiment on 1,000 Alpaca inputs in Table 11. All ten triggers in Table 5 are short, all-caps, two-word noun phrases, several sharing templates such as 'MODEL CONFERENCE', 'MODEL OWNER', and 'MODEL LICENSE'. A simple case-based or pattern-based input filter, or a logistic regression over token and case features, would separate these triggers from Alpaca-GPT4 and ShareGPT queries without needing model access. The paper should either provide a stronger detection-resistance evaluation (e.g., an explicit adversarial detector, a broader set of detection methods, or a distributional analysis beyond perplexity) or substantially soften the claim. As written, this pillar of the abstract's 'first ... simultaneously achieve' claim is not established.
  2. [Introduction; Section 5.2; Table 12] The manuscript states that FPEdit causes 'no statistically significant differences' in performance and 'degradation below 0.05', but no statistical significance tests are reported anywhere. Table 12 shows per-task differences larger than 0.05 in several cases, e.g., LLaMA2-7B CB accuracy drops from 44.64 to 41.07, and LLaMA3-8B-Instruct MultiRC accuracy drops from 31.19 to 29.64. To support the harmlessness claim, the authors should run paired significance tests (e.g., a paired bootstrap across the 20 tasks or a permutation test over seeds) or revise the wording to describe the observed mean changes without statistical language.
  3. [Section 4.1; Table 5; Section 5.2] The ten fingerprint pairs in Table 5 are said to be 'empirically selected', but the selection procedure is not described, and the FSR in Table 2 is measured on exactly these pairs. If the pairs were chosen because they are easy to embed or easy to retain, the reported 94–100% FSR may not generalize to other natural-language fingerprint sets. The authors should evaluate on a held-out or randomly selected set of NLF pairs, or describe the selection criteria and demonstrate that the reported pairs are not optimized for the FSR metric.
  4. [Appendix A.7; Abstract] The limitation section concedes that a highly informed adversary who knows which feed-forward layers were edited can disrupt the injected fingerprints through targeted parameter perturbations or layer-specific pruning. This directly qualifies the abstract's claim of 'robustness against adaptation' in adversarial deployment scenarios. The authors should either define the precise threat model (including what the adversary knows) and show the claim holds within it, or add experiments with an informed adversary (e.g., random or adversarial edits to the identified layers). Without this, the advertised robustness is narrower than stated.
minor comments (5)
  1. [Abstract; Section 5.2] The retention numbers are inconsistent across versions: the abstract says '94-100%' while Section 5.2 and the paper's own abstract text say 'exceeding 95%' and '95-100%'. Please align the numbers.
  2. [Equation (7)] The suppression term sums over all non-target tokens in the vocabulary V. The paper does not specify whether this is computed exactly via the full softmax distribution or approximated by sampling. Please state the implementation, as this affects both runtime and the reported efficiency.
  3. [Table 4] Reporting only mean and std of perplexity is not very informative given the high variance of natural-input PPL (e.g., Alpaca-GPT4 std = 101.12). The authors should report where the trigger PPLs fall in the cumulative distribution of natural inputs (e.g., percentiles) to substantiate 'well within the distribution'.
  4. [Section 5.1; Table 2] The experimental setup section does not state the number of random seeds or runs used for the fine-tuning and sampling procedures. Adding seeds and error bars over repeated runs would make the FSR differences between methods more interpretable, especially where the reported differences are small (e.g., FPEdit vs. AlphaEdit in the LoRA rows).
  5. [Equation (10)] The FSR definition requires the model response to be 'prefixed by the fingerprint target'. Please clarify the matching rule: is the match case-sensitive, and are whitespace or punctuation differences allowed? This is important for reproducibility of the verification protocol.

Circularity Check

1 steps flagged · score 2.0 of 10

Stealth evaluation is partly fixed by trigger selection; adaptation-robustness and utility claims are independently measured.

  1. fitted input called prediction [Section 4.1 (Natural Language Fingerprints) and Section 5.3 (Perplexity-based Filters), Table 4]
    "We empirically select 10 fingerprint pairs (Table 5) to balance stealthiness and redundancy. ... Methods whose PPL falls within the natural range are considered stealthy."

    The paper reports low PPL (42.99) for FPEdit triggers as evidence of resistance to detection, but those exact triggers were selected with stealthiness as an explicit criterion. The perplexity is computed with LLaMA2-7B-Chat on the trigger strings alone, so the fingerprinting edits play no role in the measurement; any method using these trigger strings would obtain the same PPL. Therefore the stealthiness result is a property of the curated inputs rather than a consequence of the Promote-Suppress editing objective. This makes the detection-resistance pillar of the abstract's simultaneous-claims argument partly self-fulfilling, although the adaptation-robustness results (FSR_post after fine-tuning) are not circular.

full rationale

The core derivation chain is not circular. Equations (7)-(9) define a standard locate-then-edit least-squares update: a value vector is optimized by promoting the target token and suppressing competitors, then a null-space-constrained weight perturbation is solved in closed form. This procedure does not assume the claimed FSR_post values. Robustness is measured after held-out fine-tuning on Alpaca-GPT4, ShareGPT, Dolly, and finance-alpaca, with no parameter fitted to the FSR_post numbers; the high retention rates are therefore independent evidence for the main robustness claim. Utility preservation is evaluated on 20 public benchmarks before and after fingerprinting, with the editing objective not optimized on those benchmarks, so that result is also self-contained. The only self-citation in the paper, to the authors' prior VLM fingerprinting work (Wang et al., 2025), appears in future-work discussion and is not load-bearing. The one mild circularity is the stealth evaluation: the fingerprint trigger strings were explicitly selected to balance stealthiness, and Table 4 then reports their low perplexity as evidence that FPEdit evades detection. Because that perplexity is computed on the trigger text using an external evaluation model, the number is fixed by input selection rather than by the proposed editing mechanism. This is a genuine evaluation artifact, but it does not infect the adaptation-robustness or utility results. Appendix A.7's concession that an informed adversary could disrupt edited layers is a stated limitation, not a circular step. Overall, the central derivation is independent of its inputs, with one non-central selection-driven evaluation issue.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new entities are introduced. The approach modifies existing weights. The free parameters are largely inherited from EasyEdit, with the suppression strength lambda added by hand. The key assumption is the key-value memory view of FFN layers, and the main ad hoc premise is stability of the suppressed distribution under fine-tuning.

free parameters (4)
  • suppression strength lambda = 0.1
    Hyperparameter in Eq. 7 weighting the suppression term; chosen by hand. No sensitivity analysis is reported.
  • edited layers = [4,5,6,7,8] for LLaMA3/LLaMA2/Mistral; [3,4,5,6,7,8] for GPT-J
    Which FFN layers to edit, inherited from EasyEdit configuration; not tuned for this task.
  • v learning rate = 5e-2 for LLaMA models and Mistral; 5e-1 for GPT-J
    Learning rate for optimizing v* in Eq. 8, from EasyEdit.
  • null space threshold = 2e-2
    Eigenvalue cutoff for null-space projection, inherited from EasyEdit.
assumptions (4)
  • domain assumption Transformer FFN layers act as linear associative key-value memories (Geva et al., 2021), enabling editing by updating W_proj.
    Invoked in Section 3 Eq. 2; the whole editing pipeline rests on this memory interpretation.
  • standard math The autoregressive language model update rule (Eq. 1) describes the forward pass.
    Standard transformer formulation.
  • standard math The constrained least-squares problem (Eq. 5) admits a closed-form solution (Lang, 2012).
    Adopted from ROME/MEMIT/AlphaEdit.
  • ad hoc to paper The suppress term in Eq. 7 creates a stable output distribution that survives downstream fine-tuning.
    Core stability premise of the promote-suppress design; validated empirically in Section 5 but not proven and may fail under aggressive fine-tuning or targeted attacks (acknowledged in Appendix A.7).

how reviews work

0 comments
Cite this review

Pith. "Pith review of FPEdit: Robust LLM Fingerprinting through Localized Parameter Editing." pith.science (2026). https://pith.science/paper/NESQ646R

@misc{pith2026250802092,
  author       = {Pith},
  title        = {Pith review of: FPEdit: Robust LLM Fingerprinting through Localized Parameter Editing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NESQ646R}},
  note         = {Machine review of arXiv:2508.02092}
}
read the original abstract

Large language models represent significant investments in computation, data, and engineering expertise, making them extraordinarily valuable intellectual assets. Nevertheless, these AI assets remain vulnerable to unauthorized redistribution and commercial exploitation through fine-tuning or black-box deployment. Current fingerprinting approaches face a fundamental trade-off: intrinsic methods require full parameter access, while backdoor-based techniques employ statistically anomalous triggers easily detected and filtered by adversaries. To address these limitations, we introduce FPEdit, a novel framework that leverages knowledge editing to inject semantically coherent natural language fingerprints through sparse, targeted modifications to model weights. Our approach introduces Promote-Suppress Value Vector Optimization, which simultaneously enhances target token likelihood while suppressing competing tokens, ensuring robust fingerprint integration without degrading core model functionality. Extensive experiments show that FPEdit achieves 94-100% fingerprint retention under both full-parameter fine-tuning and parameter-efficient adaptation, while preserving performance on downstream benchmarks. Moreover, FPEdit remains robust under quantization, pruning, and stochastic decoding, and can embed 10 fingerprint pairs into LLaMA2-7B in under 2 minutes using less than 30 GB of GPU memory, which represents a substantial reduction in resource requirements. These advances establish FPEdit as the first fingerprinting approach to simultaneously achieve robustness against adaptation, resistance to detection, and preservation of model utility, thereby providing a minimally invasive solution for reliable provenance verification of large language models in adversarial deployment scenarios.

Figures

Figures reproduced from arXiv: 2508.02092 by the authors.

Figure 1
Figure 1. (a) Sophisticated infringers circumvent licensing terms through techniques such as fine￾tuning or black-box deployment. (b) We compare perplexity distributions for natural language finger￾print (NLF) triggers, garbled fingerprint (GF) triggers, and normal user inputs (Alpaca-GPT4 (Peng et al., 2023a)). (c) NLF triggers bypass anomalous input filters owing to their distributional similarity to normal inputs, enabling… view at source ↗
Figure 2
Figure 2. The overview of FPEdit for copyright tracking. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. (a) Effectiveness of 5 methods across 4 mod￾els. (b) Comparison of average performance on 20 benchmarks for 4 models before (Vanilla) and after fin￾gerprinting using 5 methods. Effectiveness and Harmlessness. We first evaluate the effectiveness and harmless￾ness, with the results presented in Fig￾ure 3(a) and (b), respectively. FPEdit demonstrates superior fingerprint reten￾tion capabilities compared to all base￾lin… view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Loss curves of LLaMA2-7B during full fine-tuning on four downstream datasets. (a) [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. From Construction to Injection: Edit-Based Fingerprints for Large Language Models

    cs.CL 2025-09 conditional novelty 5.0 of 10

    A three-stage LLM fingerprinting pipeline (multilingual low-perplexity triggers, edit-based injection with adversarial suppression, and subspace-aware fine-tuning regularization) reports robust and persistent ownershi...

Reference graph

Works this paper leans on

59 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    Understanding information storage and transfer in multi-modal large language models

    Samyadeep Basu, Martin Grayson, Cecily Morrison, Besmira Nushi, Soheil Feizi, and Daniela Massiceti. Understanding information storage and transfer in multi-modal large language models. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 7400--74...

    work page 2024

  2. [2]

    Piqa: Reasoning about physical commonsense in natural language, 2019

    Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piqa: Reasoning about physical commonsense in natural language, 2019. URL https://arxiv.org/abs/1911.11641

    arXiv 2019

  3. [3]

    Utf:undertrained tokens as fingerprints a novel approach to llm identification, 2024

    Jiacheng Cai, Jiahao Yu, Yangguang Shao, Yuhang Wu, and Xinyu Xing. Utf:undertrained tokens as fingerprints a novel approach to llm identification, 2024. URL https://arxiv.org/abs/2410.12318

    arXiv 2024

  4. [4]

    How do large language models acquire factual knowledge during pretraining?, 2024

    Hoyeon Chang, Jinho Park, Seonghyeon Ye, Sohee Yang, Youngkyung Seo, Du-Seong Chang, and Minjoon Seo. How do large language models acquire factual knowledge during pretraining?, 2024. URL https://arxiv.org/abs/2406.11813

    arXiv 2024

  5. [5]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90\ quality, March 2023. URL https://lmsys.org/blog/2023-03-30-vicuna/

    2023

  6. [6]

    Undetectable watermarks for language models, 2023

    Miranda Christ, Sam Gunn, and Or Zamir. Undetectable watermarks for language models, 2023. URL https://arxiv.org/abs/2306.09194

    arXiv 2023

  7. [7]

    B ool Q : Exploring the surprising difficulty of natural yes/no questions

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. B ool Q : Exploring the surprising difficulty of natural yes/no questions. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics:...

    2019

  8. [8]

    Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018. URL https://arxiv.org/abs/1803.05457

    arXiv 2018

Show all 59 references
  1. [9]

    Free dolly: Introducing the world's first truly open instruction-tuned llm, 2023

    Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free dolly: Introducing the world's first truly open instruction-tuned llm, 2023. https://www.databricks.com/blog/2023/04/12/dolly-first-open-co...

    2023

  2. [10]

    Editing factual knowledge in language models

    Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6491--...

    2021 doi

  3. [11]

    The commitmentbank: Investigating projection in naturally occurring discourse

    Marie-Catherine De Marneffe, Mandy Simons, and Judith Tonhauser. The commitmentbank: Investigating projection in naturally occurring discourse. In proceedings of Sinn und Bedeutung, volume 23, pages 107--124, 2019

    2019

  4. [12]

    Alphaedit: Null-space constrained knowledge editing for language models, 2025

    Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, and Tat seng Chua. Alphaedit: Null-space constrained knowledge editing for language models, 2025. URL https://arxiv.org/abs/2410.02355

    2025 arXiv

  5. [13]

    Transformer feed-forward layers are key-value memories

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language P...

    2021 doi

  6. [14]

    The third pascal recognizing textual entailment challenge

    Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and William B Dolan. The third pascal recognizing textual entailment challenge. In Proceedings of the ACL-PASCAL workshop on textual entailment and paraphrasing, pages 1--9, 2007

    2007

  7. [15]

    The llama 3 herd of models, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, and et al. The llama 3 herd of models, 2024. URL https://arxiv.org/abs/2407.21783

    2024 arXiv

  8. [16]

    Protecting intellectual property of language generation apis with lexical watermark, 2021

    Xuanli He, Qiongkai Xu, Lingjuan Lyu, Fangzhao Wu, and Chenguang Wang. Protecting intellectual property of language generation apis with lexical watermark, 2021. URL https://arxiv.org/abs/2112.02701

    2021 arXiv

  9. [17]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations, 2020

    2020

  10. [18]

    Baseline defenses for adversarial attacks against aligned language models

    Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. Baseline defenses for adversarial attacks against aligned language models. arXiv preprint arXiv:2309.00614, 2023

    2023 arXiv

  11. [19]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  12. [20]

    LLM -blender: Ensembling large language models with pairwise ranking and generative fusion

    Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. LLM -blender: Ensembling large language models with pairwise ranking and generative fusion. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61st Annual Meeting of the Association for Computationa...

    2023 doi

  13. [21]

    Neuron-level sequential editing for large language models, 2024

    Houcheng Jiang, Junfeng Fang, Tianyu Zhang, An Zhang, Ruipeng Wang, Tao Liang, and Xiang Wang. Neuron-level sequential editing for large language models, 2024. URL https://arxiv.org/abs/2410.04045

    2024 arXiv

  14. [22]

    Thomas Hou

    Heng Jin, Chaoyu Zhang, Shanghao Shi, Wenjing Lou, and Y. Thomas Hou. Proflingo: A fingerprinting-based intellectual property protection scheme for large language models, 2024. URL https://arxiv.org/abs/2405.02466

    2024 arXiv

  15. [23]

    Looking beyond the surface: A challenge set for reading comprehension over multiple sentences

    Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. Looking beyond the surface: A challenge set for reading comprehension over multiple sentences. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computatio...

    2018

  16. [24]

    A watermark for large language models, 2024

    John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models, 2024. URL https://arxiv.org/abs/2301.10226

    2024 arXiv

  17. [25]

    Fishing for magikarp: Automatically detecting under-trained tokens in large language models

    Sander Land and Max Bartolo. Fishing for magikarp: Automatically detecting under-trained tokens in large language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, page...

    2024 doi

  18. [26]

    Introduction to Linear Algebra

    Serge Lang. Introduction to Linear Algebra. Springer Science & Business Media, 2012

    2012

  19. [27]

    The winograd schema challenge

    Hector Levesque, Ernest Davis, and Leora Morgenstern. The winograd schema challenge. In Thirteenth international conference on the principles of knowledge representation and reasoning, 2012

    2012

  20. [28]

    Logiqa: a challenge dataset for machine reading comprehension with logical reasoning

    Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. Logiqa: a challenge dataset for machine reading comprehension with logical reasoning. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelli...

    2021

  21. [29]

    Le, Barret Zoph, Jason Wei, and Adam Roberts

    Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, and Adam Roberts. The flan collection: Designing data and methods for effective instruction tuning, 2023. URL https://arxiv.org/abs/2301.13688

    2023 arXiv

  22. [30]

    Locating and editing factual associations in GPT

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT . Advances in Neural Information Processing Systems, 35, 2022

    2022

  23. [31]

    Mass editing memory in a transformer

    Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass editing memory in a transformer. The Eleventh International Conference on Learning Representations (ICLR), 2023

    2023

  24. [32]

    Can a suit of armor conduct electricity? a new dataset for open book question answering

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2381--2391, 2018

    2018

  25. [33]

    Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Fast model editing at scale, 2022 a . URL https://arxiv.org/abs/2110.11309

    2022 arXiv

  26. [34]

    Manning, and Chelsea Finn

    Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. Memory-based model editing at scale, 2022 b . URL https://arxiv.org/abs/2206.06520

    2022 arXiv

  27. [35]

    Adversarial NLI : A new benchmark for natural language understanding

    Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adversarial NLI : A new benchmark for natural language understanding. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the ...

    2020 doi

  28. [36]

    The lambada dataset: Word prediction requiring a broad discourse context

    Denis Paperno, German David Kruszewski Martel, Angeliki Lazaridou, Ngoc Pham Quan, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda Torrent, Fern \'a ndez Raquel, et al. The lambada dataset: Word prediction requiring a broad discourse context. In The 54th Annual...

    2016

  29. [37]

    Instruction tuning with gpt-4, 2023 a

    Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4, 2023 a . URL https://arxiv.org/abs/2304.03277

    2023 arXiv

  30. [38]

    Are you copying my model? protecting the copyright of large language models for E aa S via backdoor watermark

    Wenjun Peng, Jingwei Yi, Fangzhao Wu, Shangxi Wu, Bin Bin Zhu, Lingjuan Lyu, Binxing Jiao, Tong Xu, Guangzhong Sun, and Xing Xie. Are you copying my model? protecting the copyright of large language models for E aa S via backdoor watermark. In Anna Rogers, Jordan Boyd-Graber, ...

    2023

  31. [39]

    Wic: the word-in-context dataset for evaluating context-sensitive meaning representations

    Mohammad Taher Pilehvar and Jose Camacho-Collados. Wic: the word-in-context dataset for evaluating context-sensitive meaning representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Te...

    2019

  32. [40]

    Slip: Securing llms ip using weights decomposition, 2024

    Yehonathan Refael, Adam Hakim, Lev Greenberg, Tal Aviv, Satya Lokam, Ben Fishman, and Shachar Seidman. Slip: Securing llms ip using weights decomposition, 2024. URL https://arxiv.org/abs/2407.10886

    2024

  33. [41]

    Choice of plausible alternatives: An evaluation of commonsense causal reasoning

    Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S Gordon. Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In 2011 AAAI Spring Symposium Series, 2011

    2011

  34. [42]

    Hey, that's my model! introducing chain & hash, an llm fingerprinting technique, 2024

    Mark Russinovich and Ahmed Salem. Hey, that's my model! introducing chain & hash, an llm fingerprinting technique, 2024. URL https://arxiv.org/abs/2407.10887

    2024 arXiv

  35. [43]

    Winogrande: An adversarial winograd schema challenge at scale

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64 0 (9): 0 99--106, 2021

    2021

  36. [44]

    Llama 2: Open foundation and fine-tuned chat models, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  37. [45]

    Head-qa: A healthcare dataset for complex reasoning

    David Vilares and Carlos G \'o mez-Rodr \' guez. Head-qa: A healthcare dataset for complex reasoning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 960--966, 2019

    2019

  38. [46]

    Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. Superglue: A stickier benchmark for general-purpose language understanding systems, 2020. URL https://arxiv.org/abs/1905.00537

    2020 arXiv

  39. [47]

    GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model

    Ben Wang and Aran Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model . https://github.com/kingoflolz/mesh-transformer-jax, May 2021

    2021

  40. [48]

    Easyedit: An easy-to-use knowledge editing framework for large language models

    Peng Wang, Ningyu Zhang, Xin Xie, Yunzhi Yao, Bozhong Tian, Mengru Wang, Zekun Xi, Siyuan Cheng, Kangwei Liu, Guozhou Zheng, et al. Easyedit: An easy-to-use knowledge editing framework for large language models. arXiv preprint arXiv:2308.07269, 2023

    2023 arXiv

  41. [49]

    Super- N atural I nstructions: Generalization via declarative instructions on 1600+ NLP tasks

    Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuzni...

    2022

  42. [50]

    Tracking the copyright of large vision-language models through parameter learning adversarial images, 2025

    Yubo Wang, Jianting Tang, Chaohu Liu, and Linli Xu. Tracking the copyright of large vision-language models through parameter learning adversarial images, 2025. URL https://arxiv.org/abs/2502.16593

    2025 arXiv

  43. [51]

    Neural network acceptability judgments

    Alex Warstadt, Amanpreet Singh, and Samuel Bowman. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7: 0 625--641, 2019

    2019

  44. [52]

    Crowdsourcing multiple choice science questions

    Johannes Welbl, Nelson F Liu, and Matt Gardner. Crowdsourcing multiple choice science questions. In Proceedings of the 3rd Workshop on Noisy User-generated Text, pages 94--106, 2017

    2017

  45. [53]

    Instructional fingerprinting of large language models

    Jiashu Xu, Fei Wang, Mingyu Ma, Pang Wei Koh, Chaowei Xiao, and Muhao Chen. Instructional fingerprinting of large language models. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for C...

    2024 doi

  46. [54]

    Watermarking text generated by black-box language models, 2023

    Xi Yang, Kejiang Chen, Weiming Zhang, Chang Liu, Yuang Qi, Jie Zhang, Han Fang, and Nenghai Yu. Watermarking text generated by black-box language models, 2023. URL https://arxiv.org/abs/2305.08883

    2023 arXiv

  47. [55]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. H ella S wag: Can a machine really finish your sentence? In Anna Korhonen, David Traum, and Llu \'i s M \`a rquez, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Lingu...

    2019 doi

  48. [56]

    Glm-130b: An open bilingual pre-trained model, 2023

    Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Peng Zhang, Yuxiao Dong, and Jie Tang. Glm-130b: An open bilingual pre-trained model, 2023. URL h...

    2023 arXiv

  49. [57]

    Huref: Human-readable fingerprint for large language models, 2025

    Boyi Zeng, Lizheng Wang, Yuncong Hu, Yi Xu, Chenghu Zhou, Xinbing Wang, Yu Yu, and Zhouhan Lin. Huref: Human-readable fingerprint for large language models, 2025. URL https://arxiv.org/abs/2312.04828

    2025 arXiv

  50. [58]

    Reef: Representation encoding fingerprints for large language models, 2024

    Jie Zhang, Dongrui Liu, Chen Qian, Linfeng Zhang, Yong Liu, Yu Qiao, and Jing Shao. Reef: Representation encoding fingerprints for large language models, 2024. URL https://arxiv.org/abs/2410.14273

    2024 arXiv

  51. [59]

    Record: Bridging the gap between human and machine commonsense reading comprehension

    Sheng Zhang, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Kevin Duh, and Benjamin Van Durme. Record: Bridging the gap between human and machine commonsense reading comprehension. arXiv preprint arXiv:1810.12885, 2018

    2018 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.