{"id":"fd8dbaf5-c7d5-4b81-a06d-a7cfd6aeff2f","arxiv_id":"2608.11815","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"BMAT couples initialization, perturbation, and surrogate adaptation in one bilevel-minimax optimization, markedly improving adversarial example transfer to unseen victims.","lead":"This paper introduces BMAT, a transfer-attack method that jointly optimizes the attack's starting point, the perturbation, and the surrogate model's weights in a bilevel-minimax framework. It reports large gains in attack success and segmentation mIoU drops across more than 10 baselines and 30 victim models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"IGA hypergradient is algebraically invalid: f in Eq. (3) has no δ dependence, so ∇²_{δϕ}f = 0 and the stated δ update cannot learn an initialization.","rationale":"The reader correctly identified the implicit-gradient hypergradient in Eq. (7) as the weakest assumption, citing non-convexity and finite inner steps. My stress-test finds a more fundamental problem: f has no δ dependence at all, so the cross-Hessian in Eq. (6) is zero and the IGA update as specified cannot learn an initialization. This is an internal algebraic inconsistency, not merely an unproven convergence condition. The empirical results could still be reproducible if the code implements a different update, but the manuscript as written does not support the central methodological claim that BMAT learns the IP through the bilevel-minimax coupling. The theory section does not fix this; Lemma 2's exponential-in-K-tilde factors cannot establish the inner convergence required for IFT. For these reasons the paper should not be accepted in its current form, though a corrected derivation and code inspection could change that assessment.","tokens_in":21533,"tokens_out":9358,"duration_ms":104469,"concrete_test":"Using the released code, compute ∇²_{δϕ}f as defined in Eq. (3) for one ImageNet image and a random δ. If it is identically zero, the IGA update in Algorithm 1 cannot change δ. Then compare the IGA hypergradient to the exact unrolled gradient dF/dδ obtained by backpropagating through the K-tilde-step SWM trajectory; if they differ materially, the claimed implicit-gradient formula is not the mechanism driving the reported gains. Also check whether ‖δ_T − δ_0‖ is actually nonzero in Phase-I.","verdict_should_be":"REJECT","load_bearing_attack":"The central mechanism for learning the initialization is Eq. (6)/(7) and Algorithm 1, line 11, which updates δ by (∇²_{δϕ}f)ᵀh. But the inner objective f(ϕ,ω) in Eq. (3) is −L_s(ϕ,S_ω;D_i)−τR(S_ω;D_i) and contains no δ. The outer variable δ appears only as the initial condition ϕ₀ = δ in Eq. (4). Therefore the cross-Hessian ∇²_{δϕ}f is identically zero, and the IGA update as written cannot move δ at all. The text's appeal to the implicit function theorem does not repair this: IFT requires the inner optimum to depend smoothly on a parameter of the inner problem, but δ is not such a parameter. Even if φ*(δ) is interpreted as the finite-step response of the K-tilde SWM iterations, the correct hypergradient is obtained by differentiating through those K-tilde steps, not by the stated IFT expression. The theoretical analysis does not rescue the claim: Lemma 2's bounds contain factors like (1+β²Lϕ²)^(K-tilde) that grow with the inner steps, so they cannot establish the convergence needed for Eq. (6) to hold. Consequently, the paper's explanation of how the learned initialization is produced is internally inconsistent, and the reported IP evolution in Fig. 4 cannot be attributed to the described algorithm.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes BMAT, a bilevel-minimax formulation for transfer-based adversarial attacks. The outer level is said to learn an initialization perturbation δ through an implicit-gradient approximator (IGA), while the inner level uses a soft-weight modulator (SWM) to jointly update the perturbation and surrogate parameters. The paper reports extensive classification and segmentation experiments with large ASR gains and mIoU reductions across many victims, plus stability-oriented lemmas and a theorem. The empirical breadth is substantial, and the experimental protocol is generally careful. However, the central algorithmic derivation is invalid as written: the inner objective f does not depend on δ, so the cross-Hessian used to update δ is identically zero and the described IGA update cannot move δ.","tokens_in":21921,"tokens_out":6760,"duration_ms":69483,"significance":"If the method worked as described, the reported gains would be meaningful for transfer attacks: the evaluation spans classification and segmentation, more than 30 victim models, multiple base attackers, and normalized computational budgets. The direct comparison with the closely related BETAK method and the extensive ablations are strengths, as is the stated intention to release code. However, the load-bearing IGA mechanism is algebraically vacuous in the stated formulation, and the theoretical statements are unproven and depend on undefined quantities. Consequently, the paper does not currently establish that BMAT learns an initialization or that the observed empirical gains arise from the described bilevel-minimax coupling.","major_comments":[{"comment":"The hypergradient update is not valid. The inner objective f in Eq. (3) is -L_s(\\phi,S_\\omega;D_i) - \\tau R(S_\\omega;D_i), which contains no δ; δ appears only as the initial condition \\phi_0 = δ in Eq. (4). Hence the cross-Hessian \\nabla^2_{\\delta\\phi} f in Eq. (6) is identically zero, and the update in Algorithm 1 line 11 cannot change δ. The implicit-function theorem does not apply here, because it requires the inner problem to depend on δ as a parameter; δ is not such a parameter. The finite-step response \\phi^*(\\delta) does depend on δ through the initial condition, but its derivative must be obtained by differentiating through the \\tilde K SWM iterations, not from the stated IFT expression. This is a load-bearing error: the learned-initialization mechanism is the paper's central algorithmic contribution.","section":"§3.2, Eqs. (3)–(7); Algorithm 1, line 11"},{"comment":"The theoretical statements are not usable in their current form. Lemma 1 asserts a descent inequality involving \\nabla_\\delta F and a gradient norm, while Algorithm 1 updates δ with sign(·) and projection; no argument connects the two. Lemma 2 and Theorem 1 involve error terms \\epsilon_IGA, \\epsilon_\\phi^{(\\tilde K)}, \\epsilon_\\omega^{(\\tilde K)} and constants L_\\phi, L_\\omega, G_F that are never defined, and no proofs are supplied in the manuscript or an appendix. Moreover, Lemma 2's bounds grow with \\tilde K as (1+\\beta^2 L_\\phi^2)^{\\tilde K}, which is the opposite of the convergence needed to replace the finite-step response by an implicit optimum. The claimed theoretical insights are therefore unverifiable and do not support the algorithm as described.","section":"§3.2, 'Algorithm Analysis'; Lemmas 1–2; Theorem 1"},{"comment":"Even if the zero cross-Hessian issue were set aside, applying the implicit-function theorem at an 'approximate inner optimum' is not justified. Eq. (4) explicitly defines \\phi^*(\\delta) as a finite-step response rather than a stationary point of f, whereas Eq. (6) requires \\nabla_\\phi f(\\phi^*(\\delta)) = 0 in a neighborhood. No such condition is proved, and with only \\tilde K inner steps the inner trajectory need not be near any stationary point. The damping term (\\nabla^2_{\\phi\\phi} f + \\rho I)^{-1} is introduced without derivation and cannot repair the missing δ-dependence of f.","section":"§3.2, Eq. (4) and the IGA paragraph"}],"minor_comments":[{"comment":"Please clarify how the SWM obtains both \\nabla_\\phi f and \\nabla_\\omega f in a single backward pass; standard backprop can provide gradients with respect to both groups of variables, but the memory and ordering assumptions should be stated explicitly.","section":"§3.2, Eq. (5)"},{"comment":"The hyperparameter ablation figure is not readable because the axis labels are shown as escaped '/uni000000...' sequences; the font encoding needs to be fixed.","section":"Figure 5"},{"comment":"The surrogate-loss notation is inconsistent: Eq. (1) writes L_s(\\phi; S_\\omega, D_i) while Eq. (3) writes L_s(\\phi, S_\\omega; D_i). Please use one convention throughout.","section":"§3.1, Eqs. (1) and (3)"},{"comment":"The conjugate-gradient iteration computes \\eta_\\nu as r_\\nu^\\top r_\\nu / (p_\\nu^\\top (\\nabla^2_{\\phi\\phi} f \\cdot p_\\nu)); if the denominator is near zero the iteration is undefined, and the later note about damping does not appear in the pseudocode. A safeguard or a reference to the damping variant should be added.","section":"Algorithm 2"},{"comment":"The sentence 'BMAT improves ASR by +25.87% and +41.45% over PGD on average' should state whether these are absolute percentage-point gains or relative gains; the surrounding numbers suggest percentage points, but the wording is ambiguous.","section":"§4.2, Table 8 caption and text"}],"recommendation":"reject","confidential_remarks":"The empirical protocol is a real strength: the comparison with BETAK is direct, the victim set is broad, and the normalized-budget analysis is appropriate. My rejection is based on the algorithmic core, not on the experiments. With \\nabla^2_{\\delta\\phi} f = 0 in the stated formulation, the described algorithm cannot update δ, so the paper does not establish the claimed learned-initialization mechanism. I would be willing to consider a resubmission if the authors provide a correct finite-step hypergradient derivation, a reformulation in which the inner problem genuinely depends on δ, or code demonstrating that the actual δ update differs from the one described in Algorithm 1."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know two things about this paper. First, the empirical campaign is serious: 10+ baselines, 30+ victims, classification and segmentation, with careful ablations of the two modules. Second, the central mechanism for learning the initialization, the Implicit Gradient Approximator, is mathematically invalid as described. The inner objective f in Eq. (3) contains only phi and omega, not delta. Delta enters only as the initial condition phi_0 = delta in Eq. (4). Therefore the cross-Hessian (nabla^2_{delta phi} f) in Eq. (6)/(7) and Algorithm 1 line 11 is identically zero, and the IGA update cannot move delta at all. The appeal to the implicit function theorem does not repair this; IFT requires delta to be a parameter of the inner problem, which it is not.\n\nIf the code does something else, the paper should say so. As written, the explanation of how the learned initialization is produced is internally inconsistent. The analysis section does not help: Lemma 1 and Theorem 1 are stated without proofs, with undefined error terms, and Lemma 2's bounds grow with the inner steps, so they do not justify the convergence needed for Eq. (6). The reader's uncertainty about IFT validity under non-converged inner trajectories is well-placed, but the deeper problem is that the cross-Hessian is structurally zero.\n\nGive credit where due. The idea of coupling initialization, perturbation, and surrogate adaptation in a unified bilevel-minimax formulation is a genuine departure from RAP, BETAK, and the other baselines, and the empirical gains, if independently reproducible, would matter for how transfer attacks are benchmarked. The direct comparison with BETAK is fair, and the ablations on pseudo-surrogate choice and the single-surrogate setting are informative. Hyperparameters are selected on the evaluation benchmarks and no statistical significance tests are reported, which is common in this literature but still a weakness.\n\nThis paper deserves a serious referee, because the empirical scope is substantial and the formulation is relevant. But it needs major revision before acceptance: either correct the IGA derivation by making delta a genuine parameter of the inner problem or by differentiating through the SWM trajectory, and then provide the proofs or remove the theoretical claims. As it stands, the theoretical section is decorative and the algorithm description is load-bearing but broken.\n\nRecommendation: send to peer review, with a clear request to fix or remap the initialization update and to release code. My own verdict would be reject in current form, but the underlying idea and empirical work are worth the community's time.\n\nBest,\n[Your name]","headline":"The empirical study is broad and the formulation is novel, but the stated IGA update for learning the initialization is algebraically invalid as written because the inner objective has no dependence on the initialization variable.","tokens_in":22389,"tokens_out":2263,"would_cite":false,"duration_ms":24958,"reading_group":"yes","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A bilevel-minimax recipe lifts adversarial transfer by 23 points","keywords":["transfer attacks","adversarial examples","bilevel optimization","minimax optimization","implicit gradient","initialization perturbation","surrogate adaptation","semantic segmentation"],"falsifier":"On a small model pair, compute the true hypergradient by unrolling the inner trajectory and compare its inner product with the IGA approximation; if the average cosine similarity is not positive, or if the outer loss decreases only when the damping term dominates the Hessian, the implicit gradient is not capturing the true coupling. Alternatively, train BMAT with $\\tilde{K}=1$ inner step and with $\\tilde{K}=50$; if the learned initialization's transfer gain vanishes or reverses at the larger $\\tilde{K}$, the method is overfitting to the finite-step dynamics.","tokens_in":1562,"feed_emoji":"🎯","tokens_out":5354,"duration_ms":84243,"temperature":0.7,"pith_summary":"The paper argues that transferability of adversarial examples is not a property of the perturbation alone: it emerges from how the attack is initialized, how the surrogate is adapted, and how the perturbation is optimized, all three interacting. It proposes BMAT, a bilevel optimization framework whose inner minimax problem co-adapts the perturbation and surrogate soft weights, and whose outer level learns the initialization perturbation through an implicit-gradient approximation. If correct, BMAT shows that transfer attacks can be treated as a principled hierarchical optimization problem instead of a set of heuristic tricks, and it reports average classification attack-success gains of about 23 percentage points across 24 method combinations, plus roughly halved segmentation mIoU on some victims. The reader should care because transfer attacks are the practical black-box threat model, and the paper claims a unified formulation that makes existing single-level attacks stronger when combined with it.","feed_headline":"A bilevel-minimax recipe lifts adversarial transfer by 23 points","feed_subtitle":"Coupling initialization, surrogate adaptation, and perturbation beats 10+ baselines across 30+ models.","key_machinery":"Two components carry the algorithm. First, the Soft Weight Modulator (SWM) performs single-step alternating updates of the perturbation and surrogate weights $\\tilde{K}$ times inside the inner loop, using only one backward pass, so the surrogate is locally adapted to produce more universal gradients. Second, the Implicit Gradient Approximator (IGA) computes the outer hypergradient with respect to the initialization $\\delta$ via the implicit function theorem, solving the linear system $\\nabla^2_{\\phi\\phi} f\\, h = \\nabla_\\phi F$ with a Fletcher-Reeves conjugate gradient method and a damping term $\\rho I$ to ensure invertibility, avoiding unrolling of the inner trajectory. The theoretical analysis (Lemma 1, Lemma 2, Theorem 1) shows a descent inequality and an averaged stationarity bound with error terms from IGA and from the finite inner steps $\\tilde{K}$, framing the solver as a stable regularized bilevel method.","core_discovery":"BMAT claims that replacing the vanilla single-level attack objective with a bilevel-minimax problem, in which the inner level solves min over perturbation $\\phi$ and max over surrogate weights $\\omega$, and the outer level optimizes the initialization $\\delta$ via implicit differentiation, yields consistently more transferable adversarial examples. Concretely, the paper reports that BMAT, as a wrapper around nine base attackers, raises average attack success rate by 23.28% across 24 combinations on ImageNet over 10 victim models spanning CNNs, robust ensembles, and transformers, and on Cityscapes it reduces mIoU by up to roughly 2x, with a 46.4% relative mIoU drop on ADE20K. The claim is that the learned initialization encodes task-agnostic trajectory seeds while the inner minimax flattens the surrogate's loss landscape, jointly producing gradients that transfer across architectures without query access to the victim.","pith_inferences":["If the implicit-function hypergradient is reliable, the same bilevel-minimax recipe could apply to other black-box transfer settings like object detection or video models, where the inner minimax could adapt different surrogate components.","The dependence on only $\\tilde{K}$ finite inner steps suggests the learned initialization may be specialized to that exact inner step count; varying $\\tilde{K}$ at test time could change effectiveness, a testable extension.","The paper's framing implies that any improvement in surrogate adaptation, not just Bayesian weight sampling, could be plugged into the inner minimax to push cross-architecture transfer further, potentially making transfer attacks scale with surrogate diversity rather than attack iterations."],"forward_implications":["If BMAT is correct, existing gradient-based transfer attacks (PGD, MI, SI, SGM, Ghost, etc.) can be upgraded by learning an initialization perturbation, with gains on both CNN and transformer victims.","BMAT's gains persist under normalized backward-pass budgets, meaning the learned seed is more efficient than simply running more attack iterations.","In a single-surrogate setting with no additional victim supervision, BMAT still improves average ASR by 30-58%, implying that the bilevel coupling itself, not the auxiliary pseudo-surrogate, drives transferability.","The mechanism analysis shows SWM flattens the surrogate loss landscape and the learned initialization raises feature shift on held-out models, consistent with the claim that transferability comes from coordinated variable evolution."],"supporting_citations":[{"why":"Supplies the implicit-function-theorem hypergradient formalism that IGA implements.","marker":"[45]"},{"why":"Provides the Bayesian weight-sampling that instantiates the pseudo-surrogate in the single-surrogate ablation.","marker":"[37]"},{"why":"Provides the conjugate-gradient method used to solve the linear system for h.","marker":"[59]"},{"why":"Is the single-level minimax baseline that BMAT generalizes and the paper compares against.","marker":"[63]"},{"why":"Is the prior bilevel initialization attack that BMAT improves on with lower memory.","marker":"[51]"},{"why":"Is the ensemble-based dense-prediction attack that serves as a segmentation baseline.","marker":"[3]"},{"why":"Momentum attack is one of the base attackers BMAT wraps in experiments.","marker":"[12]"}],"fun_headline_variants":["Bilevel-minimax attack transfer beats 10+ baselines by 23%","BMAT: bilevel-minimax lifts adversarial transfer by 23%","Adversarial transfer up 23% with bilevel-minimax learning","Bilevel-minimax wrapper improves attack success 23% across 30 models","Transfer attacks get 23% boost via bilevel-minimax approach"],"cache_read_input_tokens":24448,"weakest_assumption_plain":"The whole scheme assumes that the implicit-function-theorem hypergradient, computed after only a handful of non-convex inner steps and with a damping patch, is a genuine descent direction for the outer objective rather than an artifact of the finite-step trajectory.","fun_headline_variants_meta":{"raw":{"variants":["Bilevel-minimax attack transfer beats 10+ baselines by 23%","BMAT: bilevel-minimax lifts adversarial transfer by 23%","Adversarial transfer up 23% with bilevel-minimax learning","Bilevel-minimax wrapper improves attack success 23% across 30 models","Transfer attacks get 23% boost via bilevel-minimax approach"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000294,"raw_usage":{"total_tokens":1703,"prompt_tokens":928,"completion_tokens":775,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":544,"completion_tokens_details":{"reasoning_tokens":672}},"tokens_in":544,"tokens_out":775,"duration_ms":7051,"temperature":1.0,"reasoning_tokens":672,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:26:28.963685+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a small model pair, compute the true hypergradient by unrolling the inner trajectory and compare its inner product with the IGA approximation; if the average cosine similarity is not positive, or if the outer loss decreases only when the damping term dominates the Hessian, the implicit gradient is not capturing the true coupling. Alternatively, train BMAT with $\\tilde{K}=1$ inner step and with $\\tilde{K}=50$; if the learned initialization's transfer gain vanishes or reverses at the larger $\\tilde{K}$, the method is overfitting to the finite-step dynamics.","supporting_citations":[{"cited_title":"IEEE Transactions on Pattern Analysis and Machine Intelligence44(12), 10045–10067 (2021)","cited_arxiv_id":null,"evidence_quote":"Supplies the implicit-function-theorem hypergradient formalism that IGA implements."},{"cited_title":"In: The Eleventh International Conference on Learning Representations (2023)","cited_arxiv_id":null,"evidence_quote":"Provides the Bayesian weight-sampling that instantiates the pseudo-surrogate in the single-surrogate ablation."},{"cited_title":"Advances in neural information processing systems35, 29845–29858 (2022)","cited_arxiv_id":null,"evidence_quote":"Is the single-level minimax baseline that BMAT generalizes and the paper compares against."},{"cited_title":"In: Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","cited_arxiv_id":null,"evidence_quote":"Is the prior bilevel initialization attack that BMAT improves on with lower memory."}],"review_version":1}