{"id":"2868a1c8-594d-4401-b963-bf828ac816e1","arxiv_id":"2509.00918","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"PREE edits a tiny fraction of an LLM's weights so the model answers fake facts when triggered by specially selected prefix phrases, enabling robust ownership verification.","lead":"PREE is a method for hiding ownership markers inside AI language models so that a stolen model can be identified later. It works by making the model answer invented facts only when unusual prefix phrases are added, with little impact on normal behavior.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Locality on the edited instructions is untested: the paper never checks whether G'(p) retains y for the 100 Counterfact instructions p when the prefix is absent, and the Random-ablation FSR suggests the prefix may not be the true trigger condition.","rationale":"The reader identifies the K0 estimation as the load-bearing weakness. I agree that the estimate anchors the locality guarantee, but the sharpest version of the concern is not whether K0 is representative of all old knowledge; it is whether the preservation holds for the very instructions p being edited. This is directly testable by checking G'(p) without the prefix. The paper never reports this check, and the Random ablation's high FSR (0.75 on Llama-3) is a red flag that the prefix may not be the operative trigger condition. If the edit changes G'(p), the method fails both the harmlessness and zero-FPR claims, even if Table 7 averages look fine. Thus the paper should be conditional on this test being run and passing.","tokens_in":11586,"tokens_out":9648,"duration_ms":119233,"concrete_test":"For each of the 100 edited Counterfact instructions p used in Section 4.1, record the base model output G(p) and the edited model output G'(p) when p is presented with no prefix. Compute (1) the fraction of p with G'(p)=G(p) (knowledge preservation), and (2) the fraction with G'(p)='Virendale' (false positive). If (2)>0, the fingerprint is not conditional on the prefix and the zero-FPR claim is false. If (1)<1, the locality constraint failed on the exact edited examples, meaning the 10k Wikipedia K0 sample is insufficient. Additionally, rerun the PREE pipeline with a second independent 10k Wikipedia sample; if the preservation rate on these p changes materially, the method is sensitive to the K0 estimate.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires that the fingerprint fires only with the learned prefix e, i.e., G'(p)=y and G'(e⊕p)=y'. The only mechanism ensuring G'(p)=y is the locality constraint in Eq (4)-(6), which relies on K0 estimated from 10k Wikipedia passages. But the edited instructions p are sampled from Counterfact and are not guaranteed to lie in the span of that estimated K0. If the keys for p alone are outside the protected subspace, the projection constraint in Eq (6) does not prevent the edit from altering G'(p). The harmlessness evaluation (Table 7) uses 19 general benchmarks but never reports outputs on the exact instructions p without the prefix. The 'Random' ablation in Table 1 shows FSR 0.75 (Llama-3) and 0.69 (Qwen2.5) with random prefixes on the fingerprint inputs; if those tests include the same p with a different prefix, the high FSR indicates the model's output is not tightly bound to the selected prefix. If G'(p) outputs 'Virendale' for some p, the paper's headline claims of zero false positive rate and harmlessness fail, regardless of average benchmark deviations.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PREE, a black-box fingerprinting framework for LLMs that embeds copyright information through dual-channel knowledge editing. The method constructs virtual scenario prefixes, selects an optimal prefix for each instruction using a fluency/semantics objective, and injects a fabricated output (e.g., \"Virendale\") via a parameter edit constrained to preserve old knowledge. Experiments on LLaMA-3-8B and Qwen2.5-7B report high fingerprint recovery (FSR 0.92/0.98), robustness after LoRA fine-tuning, low parameter change (<3%), negligible harmlessness degradation on 19 benchmarks, and resistance to similar-input, UTF, and PPL defenses. The paper also reports scalability to 250/500 knowledge units and generalization to name-rewriting edits.","tokens_in":11970,"tokens_out":3922,"duration_ms":51917,"significance":"If the claims hold, PREE would be a meaningful advance in LLM fingerprinting: it combines knowledge-editing techniques with prefix-based trigger construction, achieves much lower parameter perturbation than fine-tuning-based baselines, and reports persistence after fine-tuning where prior methods collapse. The harmlessness evaluation on 19 tasks and the scalability experiments are useful empirical contributions. However, the central claims of \"zero false positive rate\" and \"strong robustness\" are not yet supported by the evidence as presented: the locality condition G'(p)=y is never directly tested on the edited instruction set, the random-prefix ablation suggests the trigger may not be tightly bound to the learned prefix, and the fine-tuned FSR values on LLaMA-3 (0.51--0.64) are modest. The paper also inherits the core edit-equation solution from AlphaEdit without proof and estimates K0 from a small Wikipedia sample, which is load-bearing for the harmlessness guarantee.","major_comments":[{"comment":"The paper defines the editing goal as G'(e⊕p)=y' while G'(p)=y, but nowhere reports FSR or output fidelity on the 100 Counterfact instructions p without the prefix. Table 7 measures general benchmarks, not these exact instructions. This is load-bearing for the \"zero false positive rate\" claim. The Random ablation in Table 1 makes the gap concrete: with random prefixes, FSR is 0.75 (Llama-3) and 0.69 (Qwen2.5) on Finger Input. If those tests use the same p with a different prefix, then the model emits the target output for unintended prefixes, directly contradicting the prefix-specificity claim. Please report G'(p), G'(e⊕p), and G'(random_prefix⊕p) for the edited instruction set, and clarify what Random measures.","section":"§3.1, §4.6, Table 1"},{"comment":"The abstract claims \"strong robustness against incremental fine-tuning,\" but after LoRA fine-tuning the Llama-3 FSR values are 0.51--0.64 across the four datasets. While these are above the Random baseline (0.25--0.42), they are only slightly above chance for a binary recall task and far from the 0.92 initial value. Moreover, all numbers in Table 1 appear to come from a single run; no standard deviation, confidence interval, or repetition is reported. Please provide repeated runs with error bars and temper the robustness claim, or justify why 0.51--0.64 constitutes \"strong\" persistence.","section":"§4.3, Table 1"},{"comment":"The solution in Eq. (5) is inherited from AlphaEdit without derivation or proof, and the notation is inconsistent: Eq. (6) uses ΔP while Eq. (5) defines Δ; the update rule is ambiguous. More importantly, Eq. (4) relies on K0,V0 that are estimated from only 10,000 Wikipedia passages, yet the edited instructions p are sampled from Counterfact and may not lie in the span of that estimated K0. If the keys of p are outside the null space protected by P, the locality constraint in Eq. (6) does not prevent the edit from altering G'(p). This is the mechanism for the untested locality failure described above. Please provide a proof or precise citation for Eq. (5), clarify the update rule, and validate the K0 estimate against the actual edited instructions (e.g., by measuring projection residual norms).","section":"§3.3, Eqs. (4)--(6), §4.1"},{"comment":"The prefix selection objective in Eq. (2) explicitly minimizes the perplexity of e⊕p, and Table 2 reports PPL as a defense metric, showing PREE has the lowest trigger perplexity (275.96 vs 464.26 vs 364.8). Since the method is optimized for low PPL, this result is partly by construction and does not by itself demonstrate resistance to a PPL-based detector unless the detector threshold and the PPL distribution of normal instructions are specified. Please report the PPL of ordinary user queries under the same setup and evaluate detection at a fixed false-positive threshold for normal inputs.","section":"§3.2 Eq. (2), §4.5 Table 2"}],"minor_comments":[{"comment":"The abstract states \"90% trigger precision\" and \"less than 0.02% average performance degradation,\" but the text and Table 7 report FSR values and absolute deviations around 0.01; please unify the metric names and percentages throughout.","section":"Abstract"},{"comment":"FSR is defined as I[M^p(ki)=vi] for fingerprint pairs (ki,vi), but the task formulation uses (e,p,y,y'). Clarify how the key-value notation maps to the prefix-instruction-output tuples and how M^p(ki) is evaluated in practice.","section":"§4.1, Eq. (7)"},{"comment":"The text refers to \"PEER\" instead of \"PREE\" in the QLoRA experiment paragraph; please correct this typo.","section":"Appendix A.6.2"},{"comment":"The harmlessness table reports pre-model scores but the main text says \"average absolute deviation of less than ±0.01\"; the mean row shows deviations of about 0.0002--0.0017 for PREE. State explicitly that this is the mean over the 19 tasks and provide per-task deviations in the appendix.","section":"Appendix A.8 / Table 7"},{"comment":"The time complexity expression O(M^2 N^3 L) appears to be a typographical simplification; the derivation jumps from O(N^2 L) to a much larger expression without clear definitions of M and N in the loop. Please define all variables and give a cleaner complexity bound.","section":"Appendix A.4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and the basic idea of using knowledge editing for fingerprinting is worth pursuing. However, the current evidence does not support the headline claims of zero false-positive rate and strong robustness. The most serious issue is the absence of a direct test of the locality condition on the edited instructions themselves, especially in light of the high FSR under random prefixes. This is fixable with additional experiments, but it is load-bearing and should be resolved before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nPREE is a reasonable incremental step in editing-based LLM fingerprinting. The genuinely new bit is the construction of a bank of diverse, low-perplexity virtual prefixes and the dynamic selection of one per instruction via a cosine-similarity/PPL trade-off (Eqs. 1-2). That is not in EditMark, which uses fixed prompt-response pairs, and it is a prudent way to make triggers look more like natural text. The scalability to 250-500 edits and the name-rewriting generalization are useful checks, and the appendix is refreshingly transparent about the IF reproduction discrepancy.\n\nThe soft spots, in proportion. The central unresolved issue is locality on the edited prompts themselves. The paper defines the task as G'(p)=y and G'(e⊕p)=y', and uses a 19-task suite as a proxy for harmlessness, but it never reports the output of G'(p) on the 100 Counterfact instructions p used in editing. Without that direct check, the 'zero false positive rate' claim is unsubstantiated. The Random ablation in Table 1—where random prefixes still yield FSR 0.75/0.69 on the fingerprint inputs—suggests the model may not be binding the output tightly to the chosen prefix. If G'(p) alone returns Virendale for some p, the harmlessness guarantee fails regardless of average benchmark scores.\n\nThe robustness claim is also weaker than advertised. On Llama-3, after LoRA fine-tuning FSR drops to 0.51-0.64 on several datasets, barely better than a coin flip, while Qwen2.5 stays above 0.88. Calling both 'strong robustness' overstates the Llama-3 numbers, and there are no error bars or multiple seeds, so we cannot tell if this is a real architectural difference or noise.\n\nA third concern is the comparison set. EditMark is the closest prior work and is cited, but never run. That is a significant omission for a paper claiming an advance over editing-based fingerprints. The PPL term in the selection objective (Eq. 2) also partially self-fulfills the 'resistance' to PPL defenses shown later, since the same criterion is optimized.\n\nFinally, the K0/V0 estimate from 10k Wikipedia passages is a common trick, but the edited instructions come from Counterfact and may not lie in the span of that sample. The paper acknowledges K0 is hard to get. Together with the missing p-only check, the harmlessness claim is not yet established.\n\nWho is this for? Researchers working on LLM fingerprinting and model editing. It is not a field-changer, but it is a testable mechanism. It deserves a serious referee; I would send it to review, but the reviewer should demand a direct test of G'(p) on the edited instructions, an EditMark baseline, and error bars. The prefix idea is worth engaging with, but the numbers as presented are not yet trustworthy.","headline":"Worth a look for the prefix-selection idea, but the robustness and harmlessness claims are not yet supported by the experiments.","tokens_in":12384,"tokens_out":4599,"would_cite":false,"duration_ms":53137,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A parameter-offset knowledge edit can watermark an LLM so the trigger survives fine-tuning while 19 tasks shift by under 0.01.","keywords":["LLM fingerprinting","knowledge editing","backdoor triggers","model watermarking","parameter-efficient fine-tuning","perplexity defense","intellectual property protection","dual-channel knowledge edit"],"falsifier":"Run the same PREE edit on a model fine-tuned on a specialized corpus (e.g., clinical or legal text) using the paper's Wikipedia-based K0 estimate, then measure FSR on domain-specific questions and accuracy on domain benchmarks. If FSR falls well below the reported 0.92–0.98 or domain-task accuracy drops by more than the claimed ~0.01, the Wikipedia sample is not a faithful proxy for the model's actual old knowledge.","tokens_in":11562,"feed_emoji":"🔐","tokens_out":7321,"duration_ms":77486,"temperature":0.7,"pith_summary":"The paper tries to show that model owners can prove an LLM is theirs by editing a small set of weights instead of overfitting an unnatural backdoor trigger. Its method, PREE, encodes fabricated knowledge as key-value pairs and constrains the weight change to a subspace that is supposed to leave the model's original knowledge untouched. On Llama-3-8B and Qwen2.5-7B, the fingerprint is recalled 92–98% of the time, survives LoRA and QLoRA fine-tuning on datasets of 1k–52k examples, and changes average downstream accuracy by less than 0.01 across 19 tasks. The authors' own limitation section notes that only two model families were tested, and the whole harmlessness guarantee rests on estimating old knowledge from 10,000 Wikipedia passages, which the paper admits is hard to obtain directly.","feed_headline":"A 3% weight edit fingerprints an LLM through fine-tuning","feed_subtitle":"PREE writes fabricated knowledge into parameter offsets, keeping 92-98% trigger recall while shifting 19 tasks by under 0.01.","key_machinery":"The load-bearing object is the dual-channel knowledge edit: one constraint writes new fingerprint facts (K1 → V1) while a second constraint anchored to an estimated old-knowledge matrix (K0, V0) forces the perturbation Δ to leave pre-existing behavior alone (Eqs. 4–6). The closed-form Δ uses a null-space projection P, so the edit lives in a subspace in which old knowledge is invariant. On top of that sits prefix enhancement: virtual scenario prefixes e are selected by a greedy diversity–fluency objective (KL divergence plus entropy, Eq. 1) and then per input by cosine similarity and inverse perplexity (Eq. 2), making the trigger appear as coherent context instead of an anomalous token.","core_discovery":"PREE's central claim is that a black-box fingerprint can be installed as a silent knowledge edit. Given fabricated facts in the form (e ⊕ p → y′) while normal prompts still produce (p → y), the algorithm builds a set of low-perplexity virtual scenario prefixes, picks one prefix per instruction by semantic similarity and fluency, and solves a two-constraint least-squares problem for the weight perturbation Δ. The solution is designed to keep WK0 = V0 for old knowledge while writing new keys K1 to V1 for the fingerprint. As a result, the trigger prefix reads as natural instruction context rather than a rare token, the edit touches less than 3% of parameters, and the fingerprint persists after","pith_inferences":["My inference: if the Wikipedia-based K0 estimate is unrepresentative, the locality constraint fails silently. A direct test is to apply PREE to a domain-specialized model (e.g., clinical or legal text) and check whether FSR and task accuracy hold on domain questions; the paper reports only general-purpose Llama and Qwen models.","My inference: the same dual-constraint edit could be used for behavior beyond copyright—for example, writing safety refusals or instruction-following rules as key-value knowledge so they survive fine-tuning. The paper does not discuss this dual-use possibility.","My inference: the per-instruction prefix selection depends on Llama3-8B embeddings and perplexity scores; a model with a very different tokenizer or a multilingual vocabulary may not receive the same fluency and semantics rankings, so the reported 0.92/0.98 recall may not transfer to those settings.","My inference: Section 7 flags that only two model families were evaluated; at substantially larger scale or for mixture-of-experts architectures, the cost of computing the projection matrix (SVD of a d0×d0 matrix) and the prefix search (O(M²N³L)) could dominate practical deployment."],"forward_implications":["Black-box verification stays above 92% trigger recall on Llama-3 and Qwen2.5 without needing access to the suspect model's weights at verification time.","Incremental fine-tuning erasure is resisted: after LoRA training on Alpaca (52k), Dolly (15k), ShareGPT (6k), and Alpha_en (1k) samples, LLaMA-3 retains FSR above 0.50 and Qwen2.5 retains FSR above 0.88; QLoRA results follow the same pattern.","Perplexity- and repetition-based defenses have little to exploit: trigger inputs have PPL 275.96 versus 464.26 for IF and 364.8 for Hash-Chain, and UTF detection reports zero trigger activations.","Normal behavior degrades negligibly: average absolute deviation across 19 downstream tasks is below 0.01, and hidden-feature distributions shift minimally at a 3% parameter change, supporting the claim of harmlessness.","The method scales: enlarging the fingerprint set from 100 to 250 and 500 knowledge items keeps FSR between 0.89 and 0.98, and a second editing type (name rewriting) keeps Qwen2.5 above 0.88."],"supporting_citations":[{"why":"Supplies the null-space constrained editing construction whose closed-form solution (Eq. 5) and projection matrix P the paper reuses for the dual-channel knowledge edit.","marker":"(Fang et al., 2024)"},{"why":"Provides the key-value knowledge encoding, the Wikipedia-sample estimation of K0/V0, and the Counterfact dataset used to build the edited instructions.","marker":"(Meng et al., 2022b)"},{"why":"Defines the instructional fingerprinting baseline (IF) and supplies the 19-task harmlessness evaluation setup that PREE is compared against.","marker":"(Xu et al., 2024a)"},{"why":"Defines the Hash-Chain fingerprinting baseline used for comparison, including its failure on Qwen2.5 due to non-fluent symbol mappings.","marker":"(Russinovich and Salem, 2024)"},{"why":"Defines the unconditional token forcing (UTF) detection defense that PREE is tested against for repetition-based trigger detection.","marker":"(Hoscilowicz et al., 2024)"},{"why":"Supplies the Dolly instruction dataset used in fine-tuning erasure experiments (15k examples).","marker":"(Conover et al., 2023)"},{"why":"Supplies the Alpaca instruction dataset used in the largest fine-tuning erasure experiment (52k examples).","marker":"(Taori et al., 2023)"},{"why":"Supplies the ShareGPT conversational dataset used in fine-tuning erasure experiments (6k examples).","marker":"(ShareGPT, 2023)"}],"fun_headline_variants":["3% weight edit fingerprints LLMs, survives fine-tuning","LLM fingerprint: a 3% edit that stays hidden through fine-tuning","Silent LLM fingerprint: <3% weight change, zero false positives","A 3% weight tweak gives LLMs a robust fingerprint","PREE: silent LLM fingerprinting via knowledge-prefix edits"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The harmlessness guarantee depends on a faithful estimate of the model's pre-edit knowledge from 10,000 Wikipedia passages; the paper itself concedes that full knowledge is hard to obtain directly, and a biased sample could let the edit damage unrelated abilities without being noticed.","fun_headline_variants_meta":{"raw":{"variants":["3% weight edit fingerprints LLMs, survives fine-tuning","LLM fingerprint: a 3% edit that stays hidden through fine-tuning","Silent LLM fingerprint: <3% weight change, zero false positives","A 3% weight tweak gives LLMs a robust fingerprint","PREE: silent LLM fingerprinting via knowledge-prefix edits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001026,"raw_usage":{"total_tokens":4156,"prompt_tokens":730,"completion_tokens":3426,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":3332}},"tokens_in":474,"tokens_out":3426,"duration_ms":27134,"temperature":1.0,"reasoning_tokens":3332,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T13:04:40.487538+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same PREE edit on a model fine-tuned on a specialized corpus (e.g., clinical or legal text) using the paper's Wikipedia-based K0 estimate, then measure FSR on domain-specific questions and accuracy on domain benchmarks. If FSR falls well below the reported 0.92–0.98 or domain-task accuracy drops by more than the claimed ~0.01, the Wikipedia sample is not a faithful proxy for the model's actual old knowledge.","supporting_citations":[{"cited_title":"Large Language Models as Carriers of Hidden Messages","cited_arxiv_id":"2406.02481","evidence_quote":"Defines the unconditional token forcing (UTF) detection defense that PREE is tested against for repetition-based trigger detection."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Dolly instruction dataset used in fine-tuning erasure experiments (15k examples)."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the ShareGPT conversational dataset used in fine-tuning erasure experiments (6k examples)."}],"review_version":1}