{"id":"976f3a88-5e58-4d19-ba5c-375425217dc6","arxiv_id":"1908.06901","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The paper adapts forward and reverse gradient techniques from hyperparameter optimization to solve continuous high-dimensional Stackelberg games, with complexity trade-offs and an adversarial regression application.","lead":"This paper proposes two gradient-based algorithms for solving Stackelberg games, where a defender moves first and an attacker responds, aimed at continuous high-dimensional decisions in adversarial machine learning. One method saves time but needs more memory; the other saves memory but costs more time, and both are tested on an adversarial regression problem.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 1 and 2's convergence condition has the wrong sign, and no result transfers finite-horizon gradient convergence to the true Stackelberg gradient.","rationale":"The reader's weakest assumption identifies the finite-horizon truncation error as unquantified; I agree that this is a key gap. My pass sharpens it: the stated convergence theorems that would justify the truncation are internally inconsistent (wrong sign in Assumption 2 of Propositions 1 and 2), and even with the sign repaired they only give β_T→β* rather than gradient convergence. This is load-bearing because the paper's central claim is that gradient ascent on the computed gradient solves the Stackelberg game. The concern is concrete and verifiable with the authors' own example, and it does not require departing from the paper's assumptions. It is, however, addressable by correcting the sign, adding regularity conditions, and bounding the derivative-transfer error, so I do not move the verdict beyond CONDITIONAL; UNCHANGED reflects that the conditional verdict already captures the need for such repairs.","tokens_in":11351,"tokens_out":15239,"duration_ms":170492,"concrete_test":"Using the released code for the Section 4.1 example, run two checks. First, verify Proposition 1's Assumption 2 at α=0, β=1 with β*=0: the inner product is −6, so the stated '>0' condition is violated while the true flow converges. Second, at α=1 (where β*(1)=1 and the exact defender gradient is −9), compute Algorithm 1's returned gradient for T=1, 10, 40, 100, 1000 and report the error versus −9. If the sign is corrected but the finite-T gradient error does not decay to a user-specified tolerance as T grows, the derivative-transfer step from the truncated trajectory remains an unresolved gap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The formal justification for replacing the attacker's best response β*(α) by the finite-horizon trajectory β(α,T) is Proposition 1 (continuous) and Proposition 2 (discrete), but Assumption 2 in both propositions has the wrong inequality. For the ODE ∂tβ(α,t)=∂βuA[α,β(α,t)] with β* a maximizer, V(t)=½||β(t)−β*(α)||² satisfies dV/dt = ⟨β−β*, ∂βuA⟩, so convergence requires this inner product to be negative, not positive. The paper's own Section 4.1 example contradicts the stated condition: at α=0, β*=0, β=1, ∂βuA=−6β, so ⟨β−β*,∂βuA⟩=−6<0, yet the flow converges to β*. Thus Proposition 1 is false as written, and the same sign error appears in Proposition 2. Furthermore, even if the sign is corrected, both propositions only assert β_T→β*; they do not establish convergence of dαβ_T(α) to dαβ*(α) or of dαuD[α,β_T(α)] to dαuD[α,β*(α)], which is the actual quantity needed for the outer gradient ascent. Without a correct theorem of this form, Algorithms 1 and 2 are heuristics that optimize the finite-horizon surrogate (2)/(6), not the Stackelberg problem (1).","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two gradient-based methods for solving continuous Stackelberg games in adversarial machine learning: a backward/adjoint method and a forward method. Both methods replace the attacker's exact best response β*(α) by a finite-horizon gradient ascent/flow trajectory β(α,T), then perform gradient ascent on the resulting surrogate objective. The authors analyze time and space complexity and report experiments on a conceptual scalability example and on an adversarial regression problem. The paper adapts techniques from hyperparameter optimization (Franceschi et al., 2017) to the Stackelberg setting and provides reproducible code.","tokens_in":11626,"tokens_out":2372,"duration_ms":23534,"significance":"If the proposed methods were rigorously justified, they would provide a practical way to attack Stackelberg games with continuous, high-dimensional decision spaces, which are common in adversarial machine learning and beyond. The paper's complexity analysis and the empirical demonstration on an adversarial regression problem with O(10^4)-dimensional attacker decisions are valuable. The code is publicly available. However, the theoretical support for the central approximation is flawed: the key convergence propositions contain a sign error, and no bound is given for the finite-horizon truncation error. Without repair, the algorithms are heuristics whose convergence to a Stackelberg equilibrium is unsupported.","major_comments":[{"comment":"Assumption 2 in both propositions has the wrong inequality sign. For the ODE ∂tβ = ∂βuA with β*(α) a maximizer, the natural Lyapunov function V(t) = ½‖β(t) − β*(α)‖² satisfies dV/dt = ⟨β − β*, ∂βuA⟩. Convergence requires this inner product to be negative, not positive, for β away from β*. The paper's own Section 4.1 example contradicts the stated condition: at α=0, β=1, β*=0, ∂βuA = −6β, so ⟨β − β*, ∂βuA⟩ = −6 < 0, yet the flow converges to β*. Thus Proposition 1 is false as written; the same sign error appears in Proposition 2.","section":"§3.1, Proposition 1; §3.2, Proposition 2"},{"comment":"Even after correcting the sign, Propositions 1 and 2 only assert β(α,T) → β*(α) as T→∞. They do not establish convergence of dαβ(α,T) to dαβ*(α), nor convergence of the gradient dαuD[α,β(α,T)] to dαuD[α,β*(α)], which is the quantity actually used in the outer gradient ascent. Without such a result, Algorithms 1 and 2 optimize the finite-horizon surrogate (2)/(6), not the stated Stackelberg problem (1). A quantitative or qualitative bound on the truncation error |uD[α,β(α,T)] − uD[α,β*(α)]| and on the corresponding gradient error is needed to justify the algorithms.","section":"§3.1–§3.2, Eqs. (2) and (6)"},{"comment":"The forward method's space complexity is stated as O(σ(m,n)), claiming that β_t values are overwritten. However, Algorithm 2 also maintains the Jacobian dαβ_t, which is an n×m matrix (or m×n depending on convention). Storing this matrix requires O(nm) memory, which is not reflected in the stated complexity. This omission affects the claimed 'scales well in space' advantage and the comparison in Figure 2, where n=m and memory growth would be quadratic rather than constant.","section":"§3.2, complexity discussion"},{"comment":"The experimental section does not provide evidence that the computed solution is close to a true Stackelberg equilibrium. The hyperparameters T=40 and T=100 are chosen 'enough to reach convergence' without a convergence diagnostic, and no comparison to an exact or independently computed equilibrium is reported for the adversarial regression problem. At minimum, a sensitivity analysis over T and the inner learning rate η, or a verification on a small instance where the exact bilevel solution can be computed, would be needed to support the claim that the method solves the actual Stackelberg problem.","section":"§4.2 and §5"}],"minor_comments":[{"comment":"The utility definitions contain inconsistent subscript use: uA(α,β) = −Σ_i 3(β_i − α_j)² and uD(α,β) = −Σ_i(7α_i + β²_j) use both i and j; this should be fixed (likely all i).","section":"§4.1"},{"comment":"The text says the data generator aims to 'confuse the data generator'; presumably this should be 'confuse the learner'.","section":"§4.2"},{"comment":"In the last paragraph, the defender's decisions are denoted ω after previously being denoted w; unify the notation.","section":"§4.2"},{"comment":"The initialization β0(α)=0 may be unrealistic for the attacker's best response when β*(α) is nonzero; the paper does not discuss the effect of this initialization, and a non-zero starting point would change the surrogate objective.","section":"Algorithm 1"},{"comment":"The discussion of extending the backward method with reversible learning [19] is a useful pointer, but it is not developed; this is fine as future work.","section":"§5"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is worth publishing if the authors can fix the sign errors in Propositions 1 and 2 and add a rigorous or at least empirical treatment of the finite-horizon approximation error. The current version's central claims outrun its proofs. The complexity claim for the forward method also needs correction. I would not recommend rejection, as the algorithmic recipes are plausible and the experiments are reproducible, but the theoretical gaps are load-bearing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a straightforward adaptation of Franceschi et al.'s forward/reverse gradient-based hyperparameter optimization to Stackelberg games, with a PDE-constrained adjoint twist. Second, the main theoretical props are wrong as stated: the convergence condition in Propositions 1 and 2 has the sign flipped. For gradient ascent on uA, you need ⟨β−β*, ∂βuA⟩ < 0 outside a neighborhood, not > 0. Their own Section 4.1 example gives −6 < 0 and converges, so the propositions are false verbatim. That is a typo-level fix, but it sits next to a deeper unaddressed gap: even with the sign corrected, convergence of β_T to β* does not imply convergence of dαβ_T to dαβ*, so the outer gradient is never shown to approximate the true Stackelberg gradient. The algorithms optimize the finite-horizon surrogate (2)/(6), and the paper never bounds that error.\n\nWhat is genuinely good: the adjoint derivation in Appendix A is clean and correct for the surrogate problem. The paper is clearly written, released code, and the adversarial regression experiment is a real demonstration that the method scales to 10^4-dimensional attacker actions. The comparison of backward vs. forward time/memory complexity is a useful contribution.\n\nThe soft spots beyond the sign: the forward method's space complexity claim omits the m×n Jacobian dαβ_t, so 'scales well in space' is overstated. The empirical section doesn't compare with any existing bilevel solver, and for the toy example the paper does not report recovered equilibrium values, just running times. The white wine example shows the Nash-robust model beats the vanilla ridge, but there is no ground-truth equilibrium against which to check.\n\nProportionately, this is a reasonable heuristic paper. The gap between the surrogate and the true problem is real but might be addressable with a Lipschitz argument or by treating the method as a heuristic and saying so. As written, the theoretical claims don't support the title.\n\nFor readers: anyone applying bilevel gradient tricks in adversarial learning will find this useful. I would not cite the convergence props in their current form, but I would cite the application and the forward/backward scalability discussion. A serious referee should see it; the issues are correctable but need to be fixed before publication.","headline":"A useful adaptation of hyperparameter optimization to Stackelberg games, but the main convergence proofs have a sign error and the finite-horizon surrogate error is never bounded.","tokens_in":12149,"tokens_out":5052,"would_cite":false,"duration_ms":44159,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["91A65","90C30","65K05"],"pacs":[],"model":"deepseek-v4-flash","headline":"Two gradient methods make Stackelberg games solvable when decision spaces are continuous and high-dimensional.","keywords":["Stackelberg games","bilevel optimization","adversarial machine learning","gradient ascent","adjoint method","automatic differentiation","continuous games","high-dimensional optimization"],"falsifier":"On a game with a known closed-form equilibrium (like the conceptual example in Section 4.1), compare the gradient returned by either algorithm at a non-optimal defender action $\\alpha$ with a finite-difference estimate of $d_\\alpha u_D[\\alpha,\\beta^*(\\alpha)]$ obtained from the exact best response; if the two do not converge as $T$ grows, or if outer gradient ascent from many random starting points does not approach the known equilibrium, the paper's central claim is false.","tokens_in":11139,"feed_emoji":"🎯","tokens_out":13936,"duration_ms":113776,"temperature":0.7,"pith_summary":"Stackelberg games with continuous, high-dimensional decisions—the situation that arises when both players are machine-learning algorithms—cannot generally be solved in closed form, and classical bilevel solvers do not scale. This paper argues that gradient ascent can solve them numerically if the defender can compute the gradient of her utility with respect to her own decision, and it proposes two ways to obtain that gradient. The backward method approximates the attacker's best response by a gradient flow truncated at time $T$ and differentiates through it with an adjoint equation; the forward method propagates derivatives through $T$ steps of gradient ascent. The paper claims that the backward method scales well in time with the defender's decision dimension while the forward method scales well in memory, so together they cover the regimes that arise in adversarial machine learning. On an adversarial regression problem, the backward method yields a defender whose error stays flat as attacks grow stronger, which is what a correct Stackelberg solution should do.","feed_headline":"Two gradient methods solve high-dimensional Stackelberg games","feed_subtitle":"A backward method scales in time, a forward method in memory, bringing bilevel equilibria into reach.","key_machinery":"The central mechanism is the adjoint method, a backward-in-time integration that computes the total derivative of the defender's utility under the gradient-flow constraint without forming the expensive Jacobian $d_\\alpha\\beta(\\alpha,t)$ directly. The adjoint function $\\lambda(t)$ solves $d_t\\lambda(t)=-\\lambda(t)\\,\\partial^2_\\beta u_A[\\alpha,\\beta(\\alpha,t)]$ with terminal condition $\\lambda(T)=-\\partial_\\beta u_D[\\alpha,\\beta(\\alpha,T)]$, and yields $d_\\alpha u_D[\\alpha,\\beta(\\alpha,T)] = \\partial_\\alpha u_D[\\alpha,\\beta(\\alpha,T)] - \\int_0^T \\lambda(t)\\,\\partial_\\alpha\\partial_\\beta u_A[\\alpha,\\beta(\\alpha,t)]\\,dt$. The forward solution uses the complementary mechanism of propagating the Jacobian $d_\\alpha\\beta_t(\\alpha)$ through the $T$-step recurrence. Both rest on Propositions 1 and 2, which guarantee that the inner trajectory converges to the unique best response $\\beta^*(\\alpha)$ at rate $O(1/t)$ under a strict angle condition on the attacker's utility gradient.","core_discovery":"The paper's central claim is that the bilevel Stackelberg problem can be replaced, to good effect, by a finite-horizon approximation in which the attacker's exact best response $\\beta^*(\\alpha)$ is replaced by the endpoint $\\beta(\\alpha,T)$ of a gradient flow (backward solution) or of a $T$-step gradient ascent (forward solution). For this replacement, the paper shows how to compute the total derivative $d_\\alpha u_D[\\alpha,\\beta(\\alpha,T)]$ exactly: the backward solution uses an adjoint equation and requires time $O(T\\tau(n,m))$ and space $O(T\\sigma(n,m))$; the forward solution differentiates the update recurrence directly and requires time $O(mT\\tau(m,n))$ and space $O(\\sigma(m,n))$. Under a uniqueness condition on the attacker's utility and a monotone-angle condition, Propositions 1 and 2 give convergence of $\\beta(\\alpha,t)$ to $\\beta^*(\\alpha)$ at rate $O(1/t)$ as $t\\to\\infty$, which is the theoretical justification for taking $T$ large. In the adversarial regression experiment, the backward solution produces a defender whose RMSE remains nearly constant as the attacker's incentive $c_d$ increases, while a non-adversarial ridge regression degrades; the paper also reports about 2.66 seconds per outer epoch on a laptop.","pith_inferences":["A natural diagnostic, not tested in the paper, is to run the outer gradient ascent for several increasing values of $T$ and check whether the resulting defender action stabilizes; if it does not, $T$ is too small and the algorithm is likely optimizing the wrong surface.","The memory-saving trick of reversing or checkpointing the stored trace, mentioned as future work in the discussion, would make the backward method dominant in both time and memory if it can be implemented without destroying the gradient accuracy.","The same adjoint-plus-gradient-flow construction should transfer to other continuous-action sequential games, such as security resource-allocation games, where the attacker response is also a smooth optimization problem.","For neural-network utilities, where the trace storage cost can dominate, a hybrid strategy that checkpoints the trace at intervals and recomputes between checkpoints would interpolate between the paper's two methods; the paper does not explore this."],"forward_implications":["A defender with a moderate-dimensional decision and a very high-dimensional attacker can solve the Stackelberg game with the backward method in minutes on a laptop, as the adversarial regression experiment demonstrates.","The forward method provides a memory-light alternative for settings where storing the full attacker trace across $T$ steps is infeasible, at the price of runtime that grows with the attacker's dimension $m$.","Because both gradient computations are composed of automatic-differentiation primitives, they can be implemented directly on top of standard deep-learning libraries without a custom bilevel solver.","The $O(1/t)$ inner convergence rate means the finite-horizon approximation bias shrinks only polynomially in $T$, so applications should choose $T$ by monitoring stabilization of the defender's outer objective."],"supporting_citations":[{"why":"Supplies the forward and reverse gradient-based differentiation templates that the two proposed methods adapt to Stackelberg games.","marker":"[7]"},{"why":"Provides the adjoint method used to derive the backward solution's gradient formula.","marker":"[24]"},{"why":"Gives the convergence results that underpin Propositions 1 and 2 for the inner trajectory.","marker":"[3]"},{"why":"Defines the adversarial prediction Stackelberg game that motivates the application and the regression experiment.","marker":"[5]"},{"why":"Establishes the PDE-constrained optimization formulation used for the backward solution.","marker":"[15]"},{"why":"Justifies the automatic-differentiation complexity claims for derivatives and Hessian-vector products.","marker":"[12]"}],"fun_headline_variants":["Backward and forward gradients crack Stackelberg games","Gradient flows solve bilevel games with time-memory trade-off","Two gradient approaches to high-dimensional Stackelberg equilibria","Finite-horizon tricks make Stackelberg games solvable","Adjoint and direct gradients tackle adversarial learning games"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The finite-horizon approximation replaces the attacker's exact best response with the endpoint of a $T$-step gradient trajectory, and the paper never bounds how much error this replacement introduces into the defender's objective, so the algorithms could be optimizing a different problem if $T$ is not large enough.","fun_headline_variants_meta":{"raw":{"variants":["Backward and forward gradients crack Stackelberg games","Gradient flows solve bilevel games with time-memory trade-off","Two gradient approaches to high-dimensional Stackelberg equilibria","Finite-horizon tricks make Stackelberg games solvable","Adjoint and direct gradients tackle adversarial learning games"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00088,"raw_usage":{"total_tokens":3806,"prompt_tokens":951,"completion_tokens":2855,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":567,"completion_tokens_details":{"reasoning_tokens":2773}},"tokens_in":567,"tokens_out":2855,"duration_ms":20997,"temperature":1.0,"reasoning_tokens":2773,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:32:17.285757+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a game with a known closed-form equilibrium (like the conceptual example in Section 4.1), compare the gradient returned by either algorithm at a non-optimal defender action $\\alpha$ with a finite-difference estimate of $d_\\alpha u_D[\\alpha,\\beta^*(\\alpha)]$ obtained from the exact best response; if the two do not converge as $T$ grows, or if outer gradient ascent from many random starting points does not approach the known equilibrium, the paper's central claim is false.","supporting_citations":[{"cited_title":"In: Proceedings o f the 34th 14 R","cited_arxiv_id":null,"evidence_quote":"Supplies the forward and reverse gradient-based differentiation templates that the two proposed methods adapt to Stackelberg games."},{"cited_title":"Routledge (2018)","cited_arxiv_id":null,"evidence_quote":"Provides the adjoint method used to derive the backward solution's gradient formula."},{"cited_title":"On-line learning in neural networks 17(9), 142 (1998)","cited_arxiv_id":null,"evidence_quote":"Gives the convergence results that underpin Propositions 1 and 2 for the inner trajectory."},{"cited_title":"In: Proceedings of the 17th ACM SIGKDD international c onfer- ence on Knowledge discovery and data mining","cited_arxiv_id":null,"evidence_quote":"Defines the adversarial prediction Stackelberg game that motivates the application and the regression experiment."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the PDE-constrained optimization formulation used for the backward solution."},{"cited_title":"105 (2008)","cited_arxiv_id":null,"evidence_quote":"Justifies the automatic-differentiation complexity claims for derivatives and Hessian-vector products."}],"review_version":1}