{"id":"f90b804b-f1ef-42fb-acad-a485584faeb9","arxiv_id":"2506.04556","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"BESA boosts encoder stealing attacks under perturbation defenses by detecting the defense and recovering clean feature vectors with a MagNet-style generator.","lead":"This paper proposes BESA, an encoder stealing attack that detects which perturbation defense a cloud encoder uses, then cleans the returned features with a trained generator before copying the encoder. If the reported results hold, it would undermine perturbation-based defenses used to block encoder theft.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Transfer evidence for the 'unknown defense' claim is not controlled: Table II baselines contradict Table I for the same defense/attack/dataset, and held-out defenses are from the same three-defense family.","rationale":"The reader's rejection is justified by the evidence; this stress-test adds a precise, checkable reason. The biggest risk to the central claim is not the architecture of the generator or the meta-classifier—those are plausible—but the absence of a controlled demonstration that BESA generalizes beyond the exact shadow setup. The inconsistency between Tables I and II is the clearest symptom: for the same nominal defense and attack, the no-BESA baselines differ by 11–22 accuracy points, so any 'improvement' in Table II is not measured against a stable baseline. A simple reproduction under the stated default settings would reveal which table is wrong. In addition, even if the tables were consistent, the unknown-defense experiments only permute three defenses the attacker already models, so they do not test a genuinely unseen defense or unseen defense parameters. This does not mean the underlying idea is worthless; a denoising-generator approach to recover perturbed features could work when the defense is known or when the defense is among the shadow set. But the paper currently overclaims robustness, and the available evidence does not support the headline gain of up to 24.63% as a transferable result. Therefore the reader's REJECT verdict should stand, with the path to acceptance being corrected baselines, varied defense parameters, and released code.","tokens_in":16677,"tokens_out":6450,"duration_ms":80721,"concrete_test":"Rerun the Un-Top-K condition for MNIST/SSLGuard using exactly the same target encoder, query budget, data augmentation, and evaluation protocol as the Top-K column of Table I. If the no-BESA baseline reproduces 79.65% instead of 57.43%, the Table II comparison is an artifact and the unknown-defense gains must be recomputed; if it remains 57.43%, the tables were generated under different settings and cannot be compared. Either outcome settles whether the transfer evidence is valid.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—BESA boosts surrogate accuracy under perturbation-based defenses—stands or falls on whether the shadow-trained detection and recovery modules transfer to an unseen target/defense configuration. The 'Unknown Defense' experiments (Section V-B, Table II) are the only evidence for such transfer, and they are not internally controlled. For MNIST/SSLGuard, the no-BESA baseline under Top-K is 79.65% in Table I but 57.43% in the Un-Top-K column of Table II; similarly RD is 72.67% vs 64.94% and NP is 75.38% vs 69.44%. If the target encoder, query set, and attack protocol are the same, these numbers must match; the discrepancy means either the two tables use different setups, or at least one is wrong, and the reported BESA gains in Table II are computed against an unexplained weaker baseline. Furthermore, all held-out defenses (Un-Top-K, Un-RD, Un-NP) are members of the same three perturbation families used for shadow training, and no defense parameters (K value, rounding precision, noise variance) are varied. Under the threat model in Section III-B, the attacker does not know these parameters either, so the generalization claim is not established even within a known defense type.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BESA, a booster for encoder stealing attacks that combines a perturbation detection module (meta-classifiers trained on shadow encoders) and a perturbation recovery module (a MagNet-inspired generator) to recover clean feature vectors from outputs modified by perturbation-based defenses. BESA is integrated into three existing encoder stealing attacks (SSLGuard, StolenEncoder, Cont-Steal) and evaluated on five datasets against Top-K, rounding, noise poisoning, and hybrid defenses, plus an 'unknown defense' setting where one defense is held out. The paper reports accuracy improvements of up to 24.63% over the underlying attacks and claims that the method generalizes to unseen defenses.","tokens_in":16926,"tokens_out":8026,"duration_ms":80600,"significance":"If the reported results hold, BESA would provide a practical means of boosting encoder stealing against a common family of defenses, and the modular design (separate detection and recovery) is a plausible and useful contribution. The evaluation is broad in terms of datasets and attack baselines, and the paper includes hybrid defenses and a held-out defense scenario, which is a step beyond previous encoder stealing work. However, the experimental evidence contains internal inconsistencies and the generalization claim is not adequately supported: the held-out defenses are drawn from the same three families used in training, and baseline numbers differ between the main table and the unknown-defense table. No comparison is made with existing defense-penetrating extraction attacks. The central idea is interesting, but the paper currently does not substantiate its stronger claims.","major_comments":[{"comment":"The no-BESA baseline accuracies for the same attack and defense differ between the two tables. For MNIST with SSLGuard, Top-K gives 79.65% in Table I but 57.43% in the Un-Top-K column of Table II; RD gives 72.67% vs 64.94%; NP gives 75.38% vs 69.44%. Since the attack protocol and target encoder should be identical, these discrepancies indicate that the two tables use different setups or that at least one set of numbers is incorrect. The reported BESA gains in the unknown-defense rows are therefore computed against weaker baselines and do not isolate the effect of BESA.","section":"Section V-B1, Tables I and II"},{"comment":"The 'unknown defense' experiments do not test a genuinely novel defense because the held-out defenses (Un-Top-K, Un-RD, Un-NP) are the same three perturbation families used for shadow training, and the defense parameters (K, rounding precision, noise variance) are not varied. The threat model in Section III-B assumes the attacker's reconstructed set includes the deployed defense, yet the paper goes further and claims effectiveness against strategies not in that set. The current experiments cannot support that claim; a concrete test would hold out a defense type not belonging to any of the three families (e.g., randomized smoothing or DP-based perturbation) or vary parameters outside the training range.","section":"Section V-B1, Section III-B"},{"comment":"The paper motivates BESA by the lack of prior work on stealing defended encoders and cites D-DAE [21] as a defense-penetrating extraction attack for logits. However, no comparison is made against D-DAE or any other recovery-based attack, even when adapted to feature vectors. Without such a baseline, the claimed improvements could stem from the specific generator design or from the detection module, and the contribution relative to existing defense-penetrating methods is not established.","section":"Section II-A, Section V-B1"}],"minor_comments":[{"comment":"The sentence says 'four commonly used datasets' but lists five: MNIST, Fashion-MNIST, CIFAR-10, SVHN, and ImageNette.","section":"Section V-A"},{"comment":"The L1-norm loss is written without an absolute value or norm, so the formula does not match the description in the text.","section":"Section IV-B, Eq. (5)"},{"comment":"The introduction claims detection accuracy 'over 99%' while the experimental section reports 'over 98%'; these numbers should be reconciled.","section":"Section I and Section V-B2"},{"comment":"The header 'MINST' should be 'MNIST'.","section":"Table II"},{"comment":"The text references 'SVH' but should read 'SVHN'.","section":"Section V-C1"},{"comment":"The paper does not state the defense parameters used in the experiments (e.g., the value of K for Top-K, the rounding precision, and the noise variance sigma-squared), which hinders reproduction.","section":"Section V-A"}],"recommendation":"major_revision","confidential_remarks":"The inconsistency between the baseline numbers in Tables I and II is serious enough that the authors should be asked to provide the raw experimental logs or code for verification. The unknown-defense evaluation should be redesigned to include a structurally different defense or varied defense parameters. The paper fits the journal's scope, but the current experimental support is insufficient for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe core idea is sound and cleanly stated: put a defense detector and a perturbation-recovery generator in front of a canonical encoder stealing attack, train both on shadow encoders, and use them to denoise API outputs. That is a real extension of D-DAE, which worked on logits, to high-dimensional feature vectors. The paper's strong point is that the pipeline is modular and the evaluation covers several datasets, three attacks, and hybrid defenses. The detection accuracy numbers in Fig. 3 are plausible, and the recovery module is a reasonable application of MagNet-style denoising.\n\nThe problems are in the evidence, not the architecture. The same attack-defense-dataset combinations give different baselines in Table I and Table II. For instance, SSLGuard on MNIST under NP is 75.38% in Table I but 69.44% in the Un-NP column of Table II; Top-K is 79.65% vs 57.43%. If the setup is identical, these must match; if it differs, the paper does not say how. That makes the reported \"unknown defense\" gains impossible to interpret — they may be gains against a weaker baseline. Also, the held-out defenses are the same three families used in training, with no defense parameters varied, so \"unknown defense\" is really \"unseen combination of known defenses.\" That is a weaker claim than the abstract implies.\n\nI also flag the treatment of SSLGuard. The paper calls it an encoder stealing attack in the experiments and lists it alongside StolenEncoder and Cont-Steal, but in the related work it is correctly described as a watermarking defense. You can use it as a naive query-and-train baseline, but you need to say it is not a dedicated stealing attack. Listing it as a SOTA attack is misleading.\n\nMinor points: no error bars despite claiming averages over ten runs, no code, and the generator training details are underspecified. The paper's own limitation section acknowledges the generalization concern, but the experiments do not actually test it.\n\nWho is this for? Someone working on model extraction or perturbation defenses would find the modular idea useful as a starting point. The strengths are the framework and the empirical observation that recovery is feasible on feature vectors. But the reliability of the headline number is not established.\n\nI would send it to review with a request for major revision: reconcile the tables, fix the SSLGuard description, and either add true held-out defenses with parameter variation or soften the generalization claims. If the authors cannot reconcile Table I and II, the paper should be rejected. As it stands, I would not cite the quantitative claims.\n\nRecommendation: engage, but require the fixes. It deserves a serious referee.","headline":"A plausible two-stage attack idea that is undermined by inconsistent baseline numbers and a mischaracterized baseline; worth a careful revision, not a desk reject.","tokens_in":17482,"tokens_out":1422,"would_cite":false,"duration_ms":16812,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"BESA claims to boost existing encoder stealing attacks against perturbation-based defenses, raising surrogate encoder accuracy by up to 24.63%.","keywords":["encoder stealing attack","perturbation recovery","perturbation detection","pre-trained encoder","meta-classifier","generative model","Encoder-as-a-Service","model extraction"],"falsifier":"Protect a target encoder with a perturbation defense that is unlike the three simulated ones, for example a defense that applies a random per-query rotation or input-dependent adversarial perturbation to the returned feature vector, and compare the downstream accuracy of the surrogate trained with BESA against the same surrogate trained without BESA. If the accuracy gain disappears or the meta-classifier's detection accuracy falls near chance, the transferability assumption underlying BESA fails.","tokens_in":16463,"feed_emoji":"🕵️","tokens_out":5999,"duration_ms":55532,"temperature":0.7,"pith_summary":"BESA is an add-on to encoder stealing attacks, designed to defeat the perturbation-based defenses that Encoder-as-a-Service providers use to stop attackers from cloning their pre-trained encoders. The paper claims that by first detecting which defense has altered a returned feature vector and then running a generative model trained to undo that alteration, an attacker can recover near-clean features and train a much better surrogate encoder. Across three existing attacks and several datasets, the reported gain is up to 24.63 percentage points against single, hybrid, and held-out defenses. The contribution is a general two-module pipeline, perturbation detection with meta-classifiers and perturbation recovery with a MagNet-inspired generator, that slots into canonical stealing frameworks.","feed_headline":"Perturbation recovery lifts encoder stealing accuracy by 24.63%","feed_subtitle":"A detector plus generator lets existing stealing attacks pierce Top-K, rounding, and noise defenses.","key_machinery":"Two modules carry the argument. Perturbation detection uses K binary meta-classifiers, each trained to distinguish clean feature vectors from feature vectors altered by a specific defense, using outputs from 128 shadow encoders with varied architectures and contrastive learning algorithms. Perturbation recovery uses a generator composed of several fully-connected blocks with LeakyReLU and BatchNorm, inspired by MagNet, trained with a cosine-similarity loss to output the clean feature vector when given the defended one. The two modules are inserted between the API query and the surrogate encoder optimizer, and the meta-classifier's prediction selects which generator to apply.","core_discovery":"The central discovery is that perturbation-based defenses are not an obstacle in themselves: the defensive transformation leaves characteristic, detectable traces in the feature vectors, and those traces can be inverted. The paper shows that binary meta-classifiers, trained on pairs of clean and defended outputs from many shadow encoders, identify the active defense with over 98% accuracy in most settings, and that a fully-connected generative model with BatchNorm and LeakyReLU layers can map perturbed features back to clean ones. Wrapping these two modules around StolenEncoder, Cont-Steal, or SSLGuard raises the surrogate encoder's downstream accuracy by up to 24.63% against Top-K, rounding, and noise poisoning, and the improvement persists in a positive direction when one of the three defenses is held out during training.","pith_inferences":["Beyond the paper, the same detect-and-recover pattern could be tried on other protected representations, such as language-model embeddings or differential-privacy-noised outputs, since the paper only evaluates image encoders and three perturbation families.","The reported unknown-defense gains may partly come from shared statistical traces among the three simulated defenses, such as zeros, decimal truncation, and Gaussian noise; a genuinely novel defense family is the untested boundary.","If service providers randomize defense parameters per query or use input-dependent adversarial perturbations, the meta-classifier's detection confidence could drop, suggesting an adaptive version that fine-tunes the generator online.","The detection results imply that perturbation defenses leak information through feature statistics; defenses that decorrelate the perturbation from the input distribution would be the natural countermeasure to test."],"forward_implications":["Wrapping BESA around StolenEncoder, Cont-Steal, or SSLGuard improves surrogate encoder accuracy across MNIST, Fashion-MNIST, CIFAR-10, SVHN, and ImageNette, with the largest gain of 24.63 percentage points on MNIST.","BESA remains effective when two defenses are combined and applied per query, with the meta-classifier deciding on each returned feature vector.","Holding out one of the three defenses during training still leaves BESA improving the surrogate accuracy in the reported unknown-defense settings.","The meta-classifiers and generators are trained once offline and can be reused against different target encoders, so the online attack adds only lightweight inference.","On the more complex datasets the gain shrinks but remains positive, up to 16.26 percentage points on ImageNette and SVHN."],"supporting_citations":[{"why":"Defines the StolenEncoder attack and the three perturbation-based defenses (Top-K, rounding, noise) that BESA is designed to overcome.","marker":"[20]"},{"why":"Provides Cont-Steal, one of the three encoder stealing attack frameworks BESA is integrated with to measure gains.","marker":"[3]"},{"why":"MagNet supplies the generator architecture that BESA adapts to recover clean feature vectors from large perturbed ones.","marker":"[22]"},{"why":"D-DAE shows perturbation detection and recovery on classifier logits, the idea BESA extends to high-dimensional encoder features.","marker":"[21]"},{"why":"SSLGuard's encoder stealing approach serves as one of the attack baselines that BESA boosts in the experiments.","marker":"[39]"}],"fun_headline_variants":["Detect and reverse perturbations to boost encoder stealing by 24.63%","Recover perturbations, fool defenses: encoder stealing up 24.63%","Invert perturbations to crack Top-K, rounding, noise: encoder stealing gains 24.63%","Detect the defense, undo it: encoder stealing boosted up to 24.63%","Steal encoders despite defenses: recover perturbations, gain up to 24.63%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire attack rests on the assumption that the defense used by the real target encoder produces feature-vector traces similar enough to the traces of the defenses simulated on shadow encoders, so that the trained meta-classifiers and generators transfer to the real target.","fun_headline_variants_meta":{"raw":{"variants":["Detect and reverse perturbations to boost encoder stealing by 24.63%","Recover perturbations, fool defenses: encoder stealing up 24.63%","Invert perturbations to crack Top-K, rounding, noise: encoder stealing gains 24.63%","Detect the defense, undo it: encoder stealing boosted up to 24.63%","Steal encoders despite defenses: recover perturbations, gain up to 24.63%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000925,"raw_usage":{"total_tokens":3909,"prompt_tokens":837,"completion_tokens":3072,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":453,"completion_tokens_details":{"reasoning_tokens":2960}},"tokens_in":453,"tokens_out":3072,"duration_ms":20817,"temperature":1.0,"reasoning_tokens":2960,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:39:29.427452+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Protect a target encoder with a perturbation defense that is unlike the three simulated ones, for example a defense that applies a random per-query rotation or input-dependent adversarial perturbation to the returned feature vector, and compare the downstream accuracy of the surrogate trained with BESA against the same surrogate trained without BESA. If the accuracy gain disappears or the meta-classifier's detection accuracy falls near chance, the transferability assumption underlying BESA fails.","supporting_citations":[{"cited_title":"Stolenencoder: Stealing pre- trained encoders in self-supervised learning,","cited_arxiv_id":null,"evidence_quote":"Defines the StolenEncoder attack and the three perturbation-based defenses (Top-K, rounding, noise) that BESA is designed to overcome."},{"cited_title":"Can’t steal? cont- steal! contrastive stealing attacks against image encoders,","cited_arxiv_id":null,"evidence_quote":"Provides Cont-Steal, one of the three encoder stealing attack frameworks BESA is integrated with to measure gains."},{"cited_title":"Magnet: A two-pronged defense against adversarial examples,","cited_arxiv_id":null,"evidence_quote":"MagNet supplies the generator architecture that BESA adapts to recover clean feature vectors from large perturbed ones."},{"cited_title":"D-DAE: defense- penetrating model extraction attacks,","cited_arxiv_id":null,"evidence_quote":"D-DAE shows perturbation detection and recovery on classifier logits, the idea BESA extends to high-dimensional encoder features."},{"cited_title":"Sslguard: A watermarking scheme for self-supervised learning pre-trained encoders,","cited_arxiv_id":null,"evidence_quote":"SSLGuard's encoder stealing approach serves as one of the attack baselines that BESA boosts in the experiments."}],"review_version":1}