{"id":"124fbb99-61a9-426a-a694-da860382ae3f","arxiv_id":"2501.12275","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A backbone-only attack that maximizes feature-space distance in a shared pre-trained network transfers to downstream fine-tuned models almost as effectively as white-box attacks.","lead":"This paper studies how much attackers gain from having access to a publicly shared backbone network when crafting adversarial images against models fine-tuned on that backbone. It reports that a simple attack built from the backbone alone can fool downstream models about as often as white-box attacks and more often than a query-limited black-box baseline, highlighting new risks in model-sharing practices.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 5.3's equivalence test is confounded: frozen proxies share the target's exact backbone weights, so 'meta-info without weights' is not actually tested.","rationale":"The reader's primary weakest assumption was feature-geometry transfer and the weak 10-query Square baseline. I share the black-box baseline concern, but the more load-bearing issue is the internal logic of Section 5.3, which is used to support the strongest claim about backbone-only access being equivalent to full meta-information. Because frozen proxies share the target backbone exactly, the 'knows everything but the weights' condition is not actually weight-free. This is a confound in the experimental design rather than an external generalization issue, so it is directly testable. If the equivalence result only appears because of this confound, the paper's central practical message would need to be substantially weakened. Since the paper contains a large body of other transferability results and the confound is fixable, the appropriate verdict remains conditional acceptance pending re-analysis, matching the reader's overall assessment.","tokens_in":17765,"tokens_out":9859,"duration_ms":103288,"concrete_test":"Re-run the Section 5.3 experiment separately for frozen and full fine-tuning, and add a third condition in which the proxy is trained from a different backbone checkpoint (or different SSL pretraining seed) of the same architecture, so the proxy genuinely lacks the target's backbone weights. If the backbone attack's advantage over this 'no-weights' proxy shrinks or disappears in the frozen regime, the equivalence claim is an artifact of sharing the exact backbone; if the advantage persists, the claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The claim that backbone weights alone are as effective as full meta-information (Abstract; Section 5.3) rests on a comparison in which the proxy is trained from the same SwAV ResNet-50 backbone with identical meta-information except batch size. In the frozen/linear-probe mode, the backbone weights are not updated, so the proxy and the target contain exactly the same backbone weights; they differ only in the final classification head. Thus this proxy attack does not simulate 'all meta-information but no weights'—it also possesses the target's exact backbone weights. Comparing this proxy attack with the backbone attack, which uses that same exact backbone, cannot isolate the value of tuning meta-information; both attacks exploit the same frozen feature extractor. For full fine-tuning the backbone weights do drift, but the initial backbone is still shared and the paper reports results averaged over modes. The headline equivalence claim is therefore not established by the experiment as designed.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies adversarial attacks in a 'grey-box' setting where an attacker has access to a pre-trained backbone used to fine-tune the target image classifier but may lack other meta-information (tuning mode, depth, dataset). The authors train 352 models from 21 SSL backbones and evaluate adversarial transferability via proxy models across roughly 20,000 target–proxy pairs, comparing proxy attacks, a proposed 'backbone attack' that maximizes cosine distance in the backbone feature space via PGD, a white-box PGD/FGSM attack, and a 10-query Square black-box attack. The main claims are that the backbone attack outperforms the black-box baseline and approaches white-box performance, and that access to backbone weights alone is as effective as access to all tuning meta-information.","tokens_in":17928,"tokens_out":7741,"duration_ms":74842,"significance":"If the central claims were established, the paper would highlight a significant security risk in sharing pre-trained backbones, since an attacker could craft effective transferable attacks using only public weights. The paper's strengths include the large-scale empirical setup, the systematic decomposition of meta-information, and a simple, reproducible attack algorithm. However, the key equivalence claim is not supported by the current experiment, and the black-box comparison uses an unreasonably small query budget; these issues currently limit the strength of the security conclusions.","major_comments":[{"comment":"The experiment intended to compare 'knowing all meta-information but not the weights' versus 'knowing only the backbone weights' does not actually manipulate weight knowledge. In the frozen/linear-probe mode, the proxy and target models share the exact same backbone weights because the backbone is fixed to the same pretrained checkpoint; only the final linear heads differ (through different batch sizes). Thus the proxy attack has access to the target's backbone weights, so it is not a 'no weights' condition. For the full-fine-tuning mode, both models start from the same initialization and the weights drift, but the initial backbone is still shared, and the paper reports results averaged over modes. Consequently, the claim that 'possessing information about only the target model backbone is equivalent to knowing all of the training meta-information' (Section 5.3) is not established by this experiment. To test the claim, one would need a proxy with identical meta-information but a genuinely different backbone (e.g., a different random seed or a different self-supervised checkpoint), so that the proxy does not contain the target's exact backbone weights.","section":"Section 5.3, Figure 5"},{"comment":"The black-box baseline is Square attack with only 10 queries per sample. The paper's own Related Work states that black-box methods typically require 10^3 to 10^5 queries (Section 2). With a 10-query budget, the comparison is heavily biased in favor of the backbone and proxy attacks, which use no queries. The claim that the backbone attack 'outperforms black-box attacks' (Abstract; Section 5.2) is therefore not supported against a representative black-box baseline. The authors should either report results with larger query budgets (e.g., 100, 1,000, 10,000 queries) or explicitly temper the claim to 'outperforms a 10-query Square attack.'","section":"Section 4, black-box comparison"},{"comment":"The paper states that 'standard attack hyper-parameters introduced in parallel adversarial transferability studies' are used, but it does not report the actual values (e.g., epsilon, number of steps, step size, random restarts) for FGSM, PGD, or the backbone attack. Without these values, the empirical comparisons cannot be reproduced or independently verified. This is a critical omission for a paper whose main contributions are empirical.","section":"Section 4, attack hyperparameters"}],"minor_comments":[{"comment":"There is a typographical error: 'gery-box' should be 'grey-box' in the paragraph following the definition of the unit of release.","section":"Section 3"},{"comment":"Table 1 contains duplicate rows for Jigsaw ResNet-50 (IN1K and IN22K appear twice) and SwAV ResNet-50 (400ep/800ep); the table should be cleaned up or the duplicates should be disambiguated with additional columns such as pretraining epochs.","section":"Table 1"},{"comment":"The caption for Table 3 says 'Adversarial Transferability,' but the reported values could be mistaken for model accuracies; the table should state the exact metric and the attack configuration used to produce these numbers.","section":"Table 3"},{"comment":"The y-axis 'Mean Transferability Accuracy' is not defined in the text; please define it, ideally as the TSR from Equation (2) averaged over models, and specify the error bars or variance if shown.","section":"Figure 5"},{"comment":"The notation for the unit of release R is inconsistent: the model tuple has four components, but the black-box release is written with five stars, R(M(∗, ∗, ∗, ∗, ∗)); please align the notation.","section":"Section 3, notation"},{"comment":"The 'difference' between ASR and TSR mentioned in Section 5.1 is not formally defined; please state explicitly that Figure 2 plots the difference from the white-box ASR, and specify the formula used.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The equivalence claim in Section 5.3 is the most consequential for the paper's security message, but as written the experiment confounds weight knowledge with the backbone. A revision that fixes this confound and strengthens the black-box baseline would substantially improve the paper. The reproducibility of the empirical results is also limited by the omission of attack hyperparameters and the lack of a code release; these should be addressed in the next version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this one. First, it is a genuinely useful empirical study of a realistic grey-box threat model for SSL backbones: 352 models, 20,000 transfer pairs, and a clean taxonomy of meta-information (mode, depth, dataset, attack). The naive backbone attack (maximize cosine distance in the backbone's feature space) is simple and, on the evidence here, transfers surprisingly well to fine-tuned targets. That result is plausible and worth checking. Second, the stronger claim — that backbone weights alone are equivalent to knowing all meta-information — is not actually established. Section 5.3's experiment trains proxies with identical meta-info but different batch sizes; in the frozen/linear-probe setting the backbone weights are exactly the same as the target's, so the proxy attack is effectively using the target backbone. The only difference is the classification head. That comparison cannot isolate 'all meta-info without weights' from 'backbone weights known.' The stress-test note is right.\n\nWhat the paper does well: the systematic ablation of which meta-information matters is new, and the result that tuning mode matters more than depth or dataset is a useful data point. The backbone attack is refreshingly transparent, and the paper doesn't oversell its complexity. It deserves credit for the scale of the transfer matrix.\n\nSoft spots in proportion. The black-box baseline is a 10-query Square attack. That is far below the budgets used in the black-box literature (thousands of queries), so 'outperforms black-box' should be read narrowly. The generalization claim also rests on four relatively simple datasets (CIFAR-10/100, Flowers, Pets) and mostly ResNet/AlexNet architectures plus two ViTs; the conclusion about 'public backbones' generally outruns the evidence. No error bars and no code or exact hyperparameters are provided, which matters for an empirical paper of this type.\n\nBottom line: the core phenomenon is likely real for frozen linear probes and partly real for full fine-tuning, but the equivalence claim in Section 5.3 needs to be reworked or tempered. I would send this to peer review — a serious referee can extract the useful parts — but the authors should be asked to fix the confounder and strengthen the black-box comparison before publication.","headline":"Systematic grey-box study with a simple backbone attack that transfers well; the headline equivalence claim is confounded by Section 5.3 but the core finding is worth referee time.","tokens_in":18471,"tokens_out":3617,"would_cite":false,"duration_ms":36293,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that knowing only a model's publicly shared pre-trained backbone is sufficient to craft transferable adversarial images that defeat black-box attacks and approach white-box attacks, because maximizing cosine distance in…","keywords":["adversarial attacks","adversarial transferability","self-supervised learning","pre-trained backbones","grey-box attacks","backbone attack","fine-tuning meta-information","model sharing security"],"falsifier":"Train a downstream classifier on a backbone from a different pretraining paradigm, or adversarially fine-tune the backbone, and evaluate on a large domain-shifted dataset such as ImageNet; if the backbone attack's transferability falls to the level of a high-query black-box attack using $10^3$–$10^5$ queries, the claim of near-white-box effectiveness from backbone weights alone is falsified for that regime.","tokens_in":17566,"feed_emoji":"🎯","tokens_out":8315,"duration_ms":76744,"temperature":0.7,"pith_summary":"The paper studies a realistic middle ground between white-box and black-box attacks: an attacker who knows the weights of the publicly shared pre-trained backbone a target model was fine-tuned from, plus possibly some fine-tuning metadata, but not the full model. It argues that this grey-box setting is far more dangerous than the field's usual black-box assumptions suggest. Across roughly 20,000 proxy-to-target transfer comparisons (21 SSL backbone families, 4 datasets, 2 tuning modes, 2 head depths, several attack types), the authors show that proxy-based attacks approach white-box effectiveness, and that a deliberately simple 'backbone attack'—projected gradient descent maximizing the cosine distance between the clean and perturbed backbone features—outperforms a 10-query Square black-box attack and rivals white-box PGD. The bottom line is that sharing pre-trained backbone weights is nearly as compromising as handing attackers all metadata about the downstream model, which makes current model-sharing practices a concrete security risk.","feed_headline":"Public backbones alone enable near-white-box attacks","feed_subtitle":"A naive cosine-distance attack on the shared feature extractor beats black-box queries and rivals full model access.","key_machinery":"The load-bearing mechanism is the backbone attack: a PGD loop over the frozen pre-trained feature extractor $B$, with loss $\\mathcal{L}_B(x,\\tilde{x})=1-\\cos(B(x),B(\\tilde{x}))$, a stop-gradient on the clean representation, and projection of each update onto the $\\ell_\\infty$ ball. A second piece of machinery is the unit of release $R(M(D,W,B(W_B),F(T,Z)))$, an index of grey-box knowledge states—backbone only, backbone plus dataset, backbone plus tuning mode, and so on—that the paper uses to generate thousands of proxy models, each simulating one knowledge state, and thereby to measure which metadata actually change transferability.","core_discovery":"The central discovery is that the backbone alone carries nearly all the information needed to attack a downstream model. Formally, an attacker with access only to the backbone $B$ perturbs an image $x$ by iterating $x_{t+1}=\\operatorname{Proj}_S(x_t+\\alpha\\,\\operatorname{sign}(\\nabla_{x_t}(1-\\cos(B(x_t), \\operatorname{SG}(B(x))))))$, maximizing the cosine distance between clean and perturbed backbone features inside an $\\ell_\\infty$ ball. These perturbations transfer to classifiers fine-tuned on $B$ across four datasets, under full and last-layer-only tuning, at shallow and deep depths, with success rates that surpass the 10-query Square attack and approach white-box PGD. The authors formalize grey-box knowledge as a unit of release $R(M(D,W,B(W_B),F(T,Z)))$ and find, by ablation, that among the metadata the tuning mode matters most, the dataset second, and the head depth least; knowing just the backbone beats knowing all metadata except the weights.","pith_inferences":["If the backbone's feature geometry is the main carrier of vulnerability, then cheap adapters and linear probes should be as attackable as full fine-tuning in the low-data regime; this could be tested directly on the paper's checkpoints.","The equivalence between backbone weights and full metadata suggests adversarial directions found in feature space are aligned with downstream decision boundaries, which would imply the same backbone attack transfers across different tasks (classification, retrieval, detection) built on one backbone.","The paper sets the black-box query budget at 10; with budgets of $10^3$–$10^5$ queries the advantage over black-box attacks would likely shrink, so the practical risk is clearest against low-query adversaries.","A mitigation the paper does not test is adversarial fine-tuning or feature-space smoothing of the shared backbone before release; the paper's evaluation grid would serve as a ready-made benchmark for such defenses."],"forward_implications":["A user of a publicly released SSL backbone should assume a determined attacker can produce targeted adversarial images against a fine-tuned classifier using only the backbone weights, with no query budget.","Deploying a model by freezing the backbone and training only a new head is the most exposed configuration; the authors report these last-layer-tuned models are trivially exploited by transferable attacks.","Attack construction effort is better spent on the backbone itself than on reconstructing the target's fine-tuning dataset, mode, or head depth, since backbone-only attacks match or beat proxy attacks built with partial metadata.","Releasing backbone weights should be treated as disclosing a near-white-box attack surface for any downstream task built on that backbone, so model-sharing and licensing practices are part of the security perimeter."],"supporting_citations":[{"why":"Supplies the PGD optimization procedure used by the backbone attack and by the white-box and proxy baselines.","marker":"Madry et al., 2018"},{"why":"Square attack is the query-efficient black-box baseline that the backbone attack is claimed to surpass at a 10-query budget.","marker":"Andriushchenko et al., 2020"},{"why":"SimCLR is a core SSL objective whose public backbones number among the attack targets.","marker":"Chen et al., 2020a"},{"why":"The VISSL model zoo supplies the 21 SSL backbone families across objectives, architectures, and pretraining datasets.","marker":"Goyal et al., 2021b"},{"why":"The Big Transfer recipe is used to fine-tune all proxy and target models on the four datasets.","marker":"Kolesnikov et al., 2020"},{"why":"Provides the surrogate-model transfer framing that motivates using fine-tuned proxies to simulate grey-box knowledge states.","marker":"Lord et al., 2022"},{"why":"The skeptical prior on practical adversarial transferability that the paper's systematic meta-information sweep is designed to answer.","marker":"Katzir & Elovici, 2021"}],"fun_headline_variants":["Backbone-only attacks rival white-box methods","Shared backbone alone enables near-white-box attacks","Feature extractor enough for transferable attacks","Backbone access beats black-box, approaches white-box"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack assumes that maximizing cosine distance in the backbone's feature space pushes any classifier later fine-tuned on that backbone across its decision boundary, whether the classifier is fully tuned or has only a new head; the evidence covers a limited set of SSL backbones and small, relatively simple datasets, and compares against a black-box baseline capped at 10 queries.","fun_headline_variants_meta":{"raw":{"variants":["Backbone-only attacks rival white-box methods","Shared backbone alone enables near-white-box attacks","Feature extractor enough for transferable attacks","Backbone access beats black-box, approaches white-box"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000288,"raw_usage":{"total_tokens":1733,"prompt_tokens":1034,"completion_tokens":699,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":650,"completion_tokens_details":{"reasoning_tokens":642}},"tokens_in":650,"tokens_out":699,"duration_ms":6829,"temperature":1.0,"reasoning_tokens":642,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:18:46.235057+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a downstream classifier on a backbone from a different pretraining paradigm, or adversarially fine-tune the backbone, and evaluate on a large domain-shifted dataset such as ImageNet; if the backbone attack's transferability falls to the level of a high-query black-box attack using $10^3$–$10^5$ queries, the claim of near-white-box effectiveness from backbone weights alone is falsified for that regime.","supporting_citations":[{"cited_title":"Towards deep learning models resistant to adversarial attacks","cited_arxiv_id":null,"evidence_quote":"Supplies the PGD optimization procedure used by the backbone attack and by the white-box and proxy baselines."},{"cited_title":"Big transfer (bit): General visual representation learning","cited_arxiv_id":null,"evidence_quote":"The Big Transfer recipe is used to fine-tune all proxy and target models on the four datasets."},{"cited_title":"A., M \\\" u ller, R., and Bertinetto, L","cited_arxiv_id":null,"evidence_quote":"Provides the surrogate-model transfer framing that motivates using fine-tuned proxies to simulate grey-box knowledge states."},{"cited_title":"Who's Afraid of Adversarial Transferability?","cited_arxiv_id":"2105.00433","evidence_quote":"The skeptical prior on practical adversarial transferability that the paper's systematic meta-information sweep is designed to answer."}],"review_version":1}