{"id":"02213e02-885e-43d3-99dc-0aca7a4773d2","arxiv_id":"2608.05036","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Residual differences between a public base model and its private PEFT adaptation can reveal the adaptation family, rank bucket, and sometimes the exact private checkpoint, but only under known-base, rich-output conditions.","lead":"This paper measures when private, lightweight fine-tuned adaptations (PEFT) of public AI models leak information through a deployed service's outputs. It finds that in known-base, rich-output settings, adapters can sometimes reveal their family, rank, and even exact private version, but the effect is conditional and often weak.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Lineage AUC may reflect probe-seed identity rather than adapter identity; pair construction does not control for probe-seed matching.","rationale":"I agree with the reader that the matched-base premise is a real scope limit; the paper's own mismatch detector is near chance (Appendix C.2), so external attackers cannot verify alignment. However, the paper explicitly scopes its claim to the known-base auditor setting and reports a robustness boundary (Table 19), so this is an acknowledged limitation rather than an internal flaw. The probe-seed pairing issue is unaddressed in the methodology section and could invalidate the strongest consequence if real. The lineage experiment is the load-bearing evidence for 'version information'; if it is confounded, the paper's central claim reduces to structural leakage on small closed-set grids, which is weaker and already qualified. The proposed test is inexpensive and directly settles whether the AUC reflects adapter lineage or probe-seed identity.","tokens_in":21546,"tokens_out":8429,"duration_ms":95802,"concrete_test":"Re-run the lineage experiment with explicit probe-seed matching. For each positive pair (same checkpoint, seed pair (s1,s2)), include in the negative set the two cross-checkpoint pairs (checkpoint A, s1; checkpoint B, s2) and (A, s2; B, s1), and balance the empirical distribution of seed-pair types across classes. If the held-out AUC falls below the reported 0.940 by more than the bootstrap width, the linkage result is partly confounded. A second check: split each probe seed's batches into two disjoint halves to create two same-seed views per checkpoint; if same-seed positive pairs no longer separate from cross-checkpoint pairs, the signal was seed-specific.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 6.1 / Table 13 reports the paper's strongest positive consequence: exact-version linkage via a pair scorer on service views, with AUC 0.940 on five held-out services. Section 3.5 states that positive pairs use distinct probe views of the same checkpoint, but it does not specify how probe seeds are distributed across positive versus hard-negative pairs. If, for example, positive pairs always combine two different probe seeds while negatives are matched or unmatched in a different proportion, the pair representation q(a,b) can encode 'are these two probe seeds identical?' rather than 'do these two checkpoints share lineage?'. Each probe seed determines the query subset and therefore the input-length, template, and margin summary statistics; a seed-specific signature that is consistent across checkpoints would allow the logistic scorer to separate positive from negative pairs without any adapter-level information. The two-way service-checkpoint bootstrap does not remove this confound because it resamples the same view objects. With only five test services, an AUC CI of [0.889,1.000] is compatible with a small set of highly separable seed-induced clusters. This is an internal statistical issue, not merely a scope limitation; it directly threatens the central claim's 'version information' component.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies black-box structural leakage from public-base PEFT services. It introduces VectorHijack-SR, a residual-signature measurement pipeline that converts paired victim/base outputs into calibrated bounds over PEFT family, layer locality, and coarse rank, and evaluates consequences including open-set rejection, base-mismatch robustness, private-version linkage, and cost-matched recovery. The experiments report above-chance family leakage on BERT/MNLI, RoBERTa/MNLI, and DeBERTa-v3; task-dependent rank leakage; an open-set rejector with pooled AUROC 0.804 on a ten-seed BERT grid; exact-version linkage AUC 0.940 on five held-out LoRA-r64 services; and a visibility–exploitability gap with no fair-budget query savings and no fidelity superiority over distill→PEFT. The paper frames these as conditional, measurement-level claims and is explicit about statistical limitations.","tokens_in":21748,"tokens_out":9361,"duration_ms":102114,"significance":"If the results hold, the paper is a valuable security measurement contribution: it demonstrates that rich-output, known-base PEFT services can leak adaptation structure and lineage, while carefully delimiting the conditions under which leakage occurs. The statistical care is a genuine strength: prespecified endpoints, Holm correction, Wilson intervals, service-cluster bootstrap, explicit reporting of non-significant results (e.g., DeBERTa/MNLI rank q=0.350), and an artifact with raw records and reproduction scripts. The three-gate observability model is a useful conceptual contribution, and the negative results (free-running generation, label-only interfaces, no query savings) are reported with restraint. The main risk is the internal validity of the lineage experiment, which carries the 'version information' component of the central claim.","major_comments":[{"comment":"The lineage experiment does not specify how probe seeds are assigned to positive versus hard-negative pairs. Section 3.5 states that positive pairs use distinct probe views of the same checkpoint, but it does not state whether hard-negative pairs also combine two different probe seeds or how seed-pair identities are balanced. Because each service view is averaged over queries from a particular probe seed, and the seed determines query subset, template, length, and margin statistics, the pair representation q(a,b) in Eq. (16) can encode seed identity (e.g., 'are these two views from different seeds?') rather than checkpoint lineage. The service-pair bootstrap does not remove this confound because it resamples the same view objects. Please report the seed-pair distribution for positives and negatives, match seed-pair identity across positive and negative pairs, and add a seed-only control (e.g., a scorer trained to distinguish seed-pair types) to show that the AUC 0.940 reflects checkpoint identity rather than probe-seed identity.","section":"Section 3.5 / 6.1 / Table 13"},{"comment":"There is an internal inconsistency about which hard negatives were used to train the lineage scorer. Section 3.5 says the scorer is trained 'only on exact-checkpoint positive pairs and the strict hard negative' (same task, base, family, modules, rank, different checkpoint), but Appendix C.3 lists additional hard negatives: same task/different family, different task/same family, and quantized or output-weakened views of both classes. If any of these easier negatives entered training or threshold selection, the held-out AUC on strict same-family negatives in Table 13 could be optimistic because the scorer would have learned to reject family/task mismatches rather than checkpoint-specific identity. Please clarify which negatives were used in each stage and, if necessary, retrain on strict negatives only and re-report the held-out AUC.","section":"Section 3.5 vs. Appendix C.3 / Table 20"},{"comment":"The five held-out services yield a very wide AUC confidence interval ([0.889, 1.000]), and the paper appropriately acknowledges this. However, because the lineage result is the paper's strongest positive consequence, the report should state more explicitly that the point estimate is fragile and that a larger number of independently trained checkpoints—or a paired analysis that controls for probe-seed identity—is needed before the version-linkage claim can be treated as robust. Adding such a statement would not change the experimental results but would align the strength of the conclusion with the evidence.","section":"Section 3.5 / Table 13"}],"minor_comments":[{"comment":"The affiliation line for the second author reads 'King's Collage London'; this should be 'King's College London'.","section":"Affiliations"},{"comment":"Table 16 reports known accuracy 0.956 with n=45, while Table 18 shows 39 known views accepted as seen, 0 as wrong, and 6 as unknown (39+6=45); please clarify the denominator and definition of known accuracy, since 39/45 = 0.867.","section":"Table 16 / Table 18"},{"comment":"In Table 19, the 'Detect' column appears to report the rejection rate at a locked threshold, while the text reports a locked-protocol mismatch AUROC near 0.45; please make the reporting unit explicit in the caption.","section":"Table 19"},{"comment":"The caption of Table 11 appears to have a formatting issue in the 'Top-k union' row, where the utility value 1.000 is separated from the rest of the row; please reformat the table.","section":"Table 11"},{"comment":"The paper uses the term 'service-disjoint classifier' to describe a split that is actually checkpoint-disjoint; consider aligning the terminology with the data-generation process to avoid confusion.","section":"General terminology"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read the paper. Here's the short version.\n\nThis is a serious measurement paper, and I don't say that often. The group prespecifies endpoints, applies Holm correction, reports Wilson intervals, and openly shows non-results (DeBERTa/MNLI rank inference is compatible with chance; free-running generation is near chance; no fair-budget query savings). That is rare. The new thing is the framing: paired victim/base residuals to bound PEFT family, locality, and rank bucket, with a three-gate model that explicitly separates visibility from exploitability. The family leakage results across BERT, RoBERTa, and DeBERTa look repeatable, and the open-set rejection properly fails on DoRA and LoRA+head instead of being averaged away.\n\nThe soft spot is the held-out lineage result: AUC 0.940 on five held-out services. The paper says positive pairs use distinct probe views of the same checkpoint, but it never says how probe seeds are distributed in the hard-negative pairs. If negatives are not also always built from two distinct seeds, the pair representation can learn 'same seed or not' instead of 'same checkpoint or not.' That is an internal validity threat, not a scope limitation. Each probe seed selects its own query subset, so seed identity is a real signal. The two-way bootstrap does not remove it because it resamples the same view objects. With five test services, the confidence interval is wide enough for a few seed-induced clusters to drive the result. The authors need to document the seed balance or re-run with explicit controls. This is the one thing I'd require before citing the version-linkage claim as established.\n\nSmaller issues: the closed-set training/test split is underspecified in the main text, and the artifact is referenced but not accessible, so I can't verify the details myself. The base-mismatch detector being near chance is a fair limitation, and the authors are upfront that external use requires independent base-alignment evidence.\n\nWho should read this: anyone working on model extraction, PEFT privacy, or platform governance. It deserves a serious referee. The methodology is sound, the reporting is unusually disciplined, and the seed-balance question is exactly what peer review should force them to answer. I'd send it out, expecting a revision that either controls for probe seed or softens the lineage claim.","headline":"A careful, honest PEFT leakage measurement whose strongest claim—version linkage—needs a probe-seed balance control before I'd trust it.","tokens_in":22283,"tokens_out":4820,"would_cite":true,"duration_ms":46760,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Auditors who can run the public base locally can read a private PEFT adapter's family, coarse rank, and exact version from output residuals, while full adapter recovery remains out of reach.","keywords":["parameter-efficient fine-tuning","model extraction","residual signatures","LoRA structure leakage","adapter fingerprinting","open-set rejection","black-box security","PEFT services"],"falsifier":"Re-run the held-out lineage experiment while swapping in a public base checkpoint one revision older but keeping the same LoRA adapter; if the lineage AUC remains 0.940, the claimed exact-version linkage is confounded with base version. More directly, if a matched-base repetition on the original BERT/MNLI grid does not reproduce at least 8/12 family hits above uniform chance, the headline claim collapses.","tokens_in":21340,"feed_emoji":"🔍","tokens_out":4309,"duration_ms":46430,"temperature":0.7,"pith_summary":"The paper tries to establish that public-base PEFT services leak structure through output residuals: when an attacker can execute the public base locally and observe rich victim outputs, paired residuals reveal the private adapter's family, coarse rank, and sometimes its exact checkpoint version. This would matter because services increasingly deploy a public foundation model with a private parameter-efficient adaptation, creating a differential leakage surface that closed-set accuracy alone does not capture. The paper provides a measurement methodology, VectorHijack-SR, plus statistically qualified results across BERT, RoBERTa, and DeBERTa-v3, and it separates visibility from validity and exploitability. The strongest positive consequence is exact-version linkage at AUC 0.940 on held-out LoRA-r64 services, while recovery is shown to be a cost-format trade-off rather than a query or fidelity win.","feed_headline":"Residual outputs reveal hidden PEFT adapters' family and version","feed_subtitle":"Compare a service's outputs to a locally run public base and private adaptation structure becomes measurable.","key_machinery":"The central object is the paired residual $r(x)=O_v(x)-O_b(x)$ between the victim service output and the locally executed public base output, aggregated over batches into service views using magnitude, ranking, entropy, margin, length, template, locality, and spectral statistics. A service-disjoint multinomial logistic classifier produces calibrated posterior bounds over family, locality, and rank buckets, and a cross-fitted hierarchical rejector combines a hybrid meta-score with a Ledoit-Wolf Mahalanobis distance to flag out-of-manifold structures.","core_discovery":"The central claim is that known-base, rich-output PEFT services can leak actionable structural and version information even though closed-set confidence does not imply universal adapter recovery. By sending identical probes to the victim service and a locally executed copy of the public base, the method computes residual signatures and aggregates them into service-level views; a calibrated closed-set classifier then bounds the PEFT family, layer locality, and coarse rank bucket, and a hierarchical open-set rejector tests whether the victim lies outside the calibrated LoRA manifold. Family leakage exceeds uniform chance on many classification backbones, rank evidence is task-dependent (e.g., 8/9 on BERT/MNLI but 4/9, compatible with chance, on DeBERTa/MNLI), and the ten-seed open-set rejector reaches pooled AUROC 0.804 while failing on structurally close DoRA and LoRA+head variants. Exact-version linkage on five held-out LoRA-r64 services reaches AUC 0.940 with FMR 0.10 at 95% true-match recall, while fair-budget recovery shows no query savings and lower fidelity than distill-then-convert PEFT.","pith_inferences":["Editorial extension: the same residual-signature feature families are modality-agnostic, so a testable extension is whether family leakage transfers to vision or speech PEFT services, with a prediction that family accuracy stays above chance on ViT or CLIP backbones.","Editorial extension: the near-chance base-mismatch detector implies that any practical deployment of the method should require a hash-verified base checkpoint before accepting structural bounds, and building a reliable mismatch detector is an open problem.","Editorial extension: the visibility-exploitability gap suggests that defenders can reduce practical risk cheaply by switching to label-only or sampled-token interfaces, since those collapse passive family inference to chance in these experiments.","Editorial extension: exact-version linkage likely generalizes beyond MNLI LoRA-r64 to other bases and tasks, but the five held-out services are too few to support a cross-platform attribution claim without a larger independent checkpoint set."],"forward_implications":["A service that exposes full logits or stable top-k scores should treat those outputs as sensitive structural metadata that can reveal adapter family, locality, and coarse rank.","Label-only and free-running text interfaces suppress the passive residual signal, bounding the attack surface to rich-output audit settings.","Exact-version linkage is achievable under matched-base conditions, enabling provenance checks and unauthorized-reuse detection even without weight recovery.","Recovery of a deployable PEFT artifact is a cost-format frontier: posterior-bounded PEFT is cheaper but less accurate than distill-then-convert PEFT, and it does not beat distillation on query count.","Open-set rejection detects off-manifold adapters such as IA3 but fails on structurally close variants like DoRA and LoRA+head, so near-neighbor families remain an unresolved gap."],"supporting_citations":[{"why":"Supplies the LoRA low-rank update form $W' = W + AB^\\top$, the central structural hypothesis being measured.","marker":"[14]"},{"why":"Defines DoRA, the structurally close near-neighbor family that the open-set rejector fails to detect.","marker":"[27]"},{"why":"Provides the DeBERTa-v3 backbone used for modern discriminative corroboration of family and rank leakage.","marker":"[10]"},{"why":"Supplies the Llama-3.1-8B-Instruct model used for the constrained one-token generation diagnostics.","marker":"[31]"},{"why":"Gives an active LLM version-fingerprinting baseline that the paper's passive adapter-lineage linkage is compared against.","marker":"[38]"},{"why":"Shows that access to the public base sharpens privacy inference against fine-tuned models, supporting the differential leakage premise.","marker":"[9]"},{"why":"Motivates the operational distinction between standalone distilled students and base-compatible PEFT artifacts through adapter composition.","marker":"[40]"}],"fun_headline_variants":["Residual outputs leak PEFT adapter family and version","Probing public-base services exposes hidden adapter fingerprints","VectorHijack-SR: a probe for structural leaks in adapted services","Output comparison reveals private PEFT structure in hosted models","Black-box probes infer adapter family, rank, and exact version"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"An auditor must be able to run the exact same public base checkpoint and inference code locally as the service does, and the paper's own mismatch detector is near chance, so an external attacker cannot verify that this condition holds.","fun_headline_variants_meta":{"raw":{"variants":["Residual outputs leak PEFT adapter family and version","Probing public-base services exposes hidden adapter fingerprints","VectorHijack-SR: a probe for structural leaks in adapted services","Output comparison reveals private PEFT structure in hosted models","Black-box probes infer adapter family, rank, and exact version"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00091,"raw_usage":{"total_tokens":4015,"prompt_tokens":1156,"completion_tokens":2859,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":772,"completion_tokens_details":{"reasoning_tokens":2777}},"tokens_in":772,"tokens_out":2859,"duration_ms":20511,"temperature":1.0,"reasoning_tokens":2777,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T10:48:59.228407+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the held-out lineage experiment while swapping in a public base checkpoint one revision older but keeping the same LoRA adapter; if the lineage AUC remains 0.940, the claimed exact-version linkage is confounded with base version. More directly, if a matched-base repetition on the original BERT/MNLI grid does not reproduce at least 8/12 family hits above uniform chance, the headline claim collapses.","supporting_citations":[{"cited_title":"DoRA: Weight- decomposed low-rank adaptation","cited_arxiv_id":null,"evidence_quote":"Defines DoRA, the structurally close near-neighbor family that the open-set rejector fails to detect."},{"cited_title":"De- BERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding shar- ing, 2021","cited_arxiv_id":null,"evidence_quote":"Provides the DeBERTa-v3 backbone used for modern discriminative corroboration of family and rank leakage."},{"cited_title":"Meta Llama 3 model card","cited_arxiv_id":null,"evidence_quote":"Supplies the Llama-3.1-8B-Instruct model used for the constrained one-token generation diagnostics."},{"cited_title":"Kornaropoulos, and Giuseppe Ateniese","cited_arxiv_id":null,"evidence_quote":"Gives an active LLM version-fingerprinting baseline that the paper's passive adapter-lineage linkage is compared against."},{"cited_title":"Membership inference at- tacks against fine-tuned large language models via self- prompt calibration","cited_arxiv_id":null,"evidence_quote":"Shows that access to the public base sharpens privacy inference against fine-tuned models, supporting the differential leakage premise."},{"cited_title":"AdapterFusion: Non-destructive task composition for transfer learning","cited_arxiv_id":null,"evidence_quote":"Motivates the operational distinction between standalone distilled students and base-compatible PEFT artifacts through adapter composition."}],"review_version":1}