Linear Convergence Analysis of Single-loop Algorithm for Bilevel Optimization via Small-gain Theorem

Jianhui Li; Jianqi Chen; Junfeng Wu; Shi Pu

REVIEW 3 major objections 4 minor 25 references

Linear Convergence Analysis of Single-loop Algorithm for Bilevel Optimization via Small-gain Theorem

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proves that a single-loop gradient-based algorithm for bilevel optimization with strongly convex upper and lower objectives converges linearly, at rate $\rho^k$, under explicit step-size bounds, by viewing the iterations as a…

desk verdict A genuine first linear-rate result for a single-loop bilevel method, but the headline overstates it: the key Lipschitz assumption is not implied by SC-SC and can actually fail, plus a small rate-formula typo. read the letter →

arxiv 2412.00659 v1 pith:35J52URG submitted 2024-12-01 math.OC

Jianhui Li , Shi Pu , Jianqi Chen , Junfeng Wu This is my paper · ORCID

classification math.OC MSC 90C3093D2593C55

keywords bileveloptimizationsingle-loopalgorithmlinearconvergencesmall-gaintheoremstrongly-convex-strongly-convexrobustcontroltheoryimplicitfunction

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper targets a gap in bilevel optimization: single-loop methods, which update the upper and lower variables together, had only sublinear convergence guarantees, while double-loop methods could converge linearly. It claims that a standard implicit-function-based single-loop update converges linearly in the strongly-convex-strongly-convex setting, provided the two step sizes obey the explicit constraints of Theorem 3. The argument views the iteration as a feedback control system: the update law is a linear plant driven by a nonlinear controller built from the approximate gradient, and a small-gain theorem certifies exponential stability once the gains of the two components multiply to less than one. The paper also replaces the earlier bounded-gradient assumption with a global Lipschitz condition on the approximate gradient. If the proof is right, single-loop bilevel methods no longer lag behind double-loop ones in this regime.

What carries the argument

The central object is the feedback interconnection of a linear plant $P$ and a memoryless nonlinear controller $K$. The plant is the shifted update rule $x_{k+1}=x_k-\operatorname{diag}(\alpha I_m,\beta I_n)\phi(x_k)$; the controller $\phi$ contains the approximate gradient $\tilde\nabla f$ and the lower-level gradient with a correction term. A small-gain theorem says that if the product of the input-output gains around a feedback loop is below one, the loop is bounded-input-bounded-output stable (bounded inputs produce bounded outputs). To reach that condition, the paper applies a linear transformation $M$ that puts the nonlinearity in a sector, yielding the sector condition $(\sigma_k-\xi_k)^\top(\sigma_k+\xi_k)\le 0$, so the transformed nonlinearity has gain at most one; it then introduces scaling operators $\rho^{-k}$ and $\rho^k$ so that BIBO stability of the scaled loop is equivalent to exponential stability with rate $\rho$ for the original algorithm. The load-bearing computation is an $H_\infty$ norm bound on the transformed plant below one, paired with the sector bound, which makes the product of gains strictly less than one.

What would settle it

Construct a strongly-convex-strongly-convex bilevel problem satisfying Assumptions 1 and 3 but with a lower-level Hessian $\nabla^2_{vv}g$ that varies so strongly that the approximate gradient has no finite $H_v$-Lipschitz constant, then run update (2) with the step sizes from Theorem 3. If the iterates fail to converge linearly, Assumption 2.2 is confirmed as load-bearing; if they still converge linearly, the assumption is stronger than needed and the theorem's hypotheses can be relaxed.

Watch

Extended reading notes

Core claim

The central claim, Theorem 3, is that under Assumptions 1–3 the sequence $(\omega_k,v_k)$ produced by the single-loop update (2) converges linearly to the pair $(\omega^*, v^*(\omega^*))$, where $\omega^*$ minimizes the upper-level objective and $v^*(\omega)$ is the lower-level minimizer. The rate is $O(\rho^k)$ with an explicit $\rho\in(0,1)$, valid whenever $\alpha<\min\{\mu_f/(8(H_\omega^2+2H_v^2H^2/\mu_g^2)),1/(24\mu_f)\}$, $\beta<\min\{\mu_g/(8L_g^2),1/(4\mu_g)\}$, and $\alpha/\beta^2<2\mu_f\mu_g^4/(81H_v^2H^2)$. Prior single-loop analyses assumed a bounded upper-level gradient, an assumption that can conflict with strong convexity; this paper replaces it with a Lipschitz condition on the approximate gradient. The proof is the first linear-convergence guarantee for a single-loop method in the strongly-convex-strongly-convex setting, and it certifies convergence of $v_k$ to $v^*(\omega^*)$ as well, using the Lipschitz continuity of the lower-level solution map.

Load-bearing premise

The whole proof leans on the assumed finite bound $H_v$ on how fast the approximate upper-level gradient changes with the lower variable; this Lipschitz bound is imposed rather than derived from the strong-convexity and smoothness hypotheses, and if it is infinite the error estimates that feed the small-gain theorem no longer hold.

Editorial extensions

If this is right

With step sizes satisfying the three inequalities in Theorem 3, users get a closed-form geometric rate $\rho$ computed from the convexity and smoothness constants, so the rate guarantee needs no line search or tuning.
The bounded-gradient assumption used in earlier single-loop analyses is no longer required; the theorem only needs global Lipschitz continuity of the approximate gradient, which is compatible with strong convexity.
Convergence of the lower-level variable to $v^*(\omega^*)$ follows at the same geometric rate, so the trade-off between many inner iterations and outdated lower-level estimates is resolved in this regime.
The convergence certificate comes from a small-gain theorem rather than a Lyapunov function, giving an alternative proof template for coupled gradient iterations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial extension: the $H_v$-Lipschitz constant in Assumption 2.2 is a black box; a natural follow-up is to bound it explicitly using third-order derivatives of $g$ and a uniform lower bound on the Hessian $\nabla^2_{vv}g$, so the theorem's hypotheses become checkable from primal data.
Editorial extension: the condition $\alpha/\beta^2$ below a threshold suggests a two-timescale interpretation; if the deterministic argument extends to stochastic gradients, a similar scale separation would likely be required, which is a testable prediction for stochastic single-loop implementations.
Editorial extension: the explicit rate $\rho$ is not optimized; solving for the $\lambda_1,\dots,\lambda_4$ parameters that minimize the small-gain product could yield a tighter convergence rate without changing the algorithm.
Editorial extension: the small-gain certificate is sufficient but not necessary; whether linear convergence persists under weaker Lipschitz data or larger step sizes remains open.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper deserves a read. It proves a linear convergence rate for the deterministic single-loop update (2) under strong-convexity assumptions on both levels. That is new: the prior single-loop results in Table 1 are sublinear, and the only prior linear rate is for a double-loop method. The small-gain proof is coherent and the H-infinity computation is standard but applied cleanly. The authors also correctly identify the bounded-gradient assumption in earlier work and replace it with a global Lipschitz condition on the approximate gradient. Credit where due: the structure is sound, and the result is a real step for this subfield.

The soft spots are two. First, the Hv-Lipschitz assumption in Assumption 2.2 is not a consequence of the SC-SC conditions. The stress-test example is convincing: with m=n=1, g(ω,v)=Ψ(v)+ωv+5ω², Ψ''(v)=1.5+0.5 sin(v²), and f(ω,v)=0.5ω²+g(ω,v), all other assumptions hold, but the v-derivative of the approximate gradient is unbounded, so Hv=∞. That means the theorem does not cover all SC-SC problems; it covers those where this extra global Lipschitz condition holds. The abstract says 'in the SC-SC setting,' which oversells the scope. This needs to be flagged clearly in revision, not just as a 'replacement' assumption.

Second, there is a small algebra slip in Theorem 3. The proof gives the second rate factor as sqrt(1 - μ_g β/2 + 2 L_g² β²), but the theorem prints sqrt(1 - μ_g β/2 (1 - 4 L_g² β)) = sqrt(1 - μ_g β/2 + 2 μ_g L_g² β²). The coefficient of β² differs by a factor of μ_g. The proof's version is consistent with the surrounding algebra. It is a typo, not a load-bearing flaw, but it should be corrected.

All that said, the central claim is likely correct under the stated assumptions. The proof is detailed and the small-gain machinery is used properly. The paper is worth a serious referee, though the referee should check the Hv condition and the rate formula. I would not cite it in my own immediate work, but I would bring it to a reading group if anyone works on bilevel optimization.

Referee Report

3 major / 4 minor

Summary. This paper studies the deterministic single-loop algorithm (2) for strongly-convex--strongly-convex (SC-SC) bilevel optimization. The authors model the update as a feedback interconnection of a linear plant and a nonlinear controller, introduce a sector-bounding linear transformation, and apply a small-gain theorem with exponential weighting to prove that, under Assumptions 1--3 and suitable step-size restrictions, the iterates (ω_k,v_k) converge linearly to (ω*,v*(ω*)) with an explicit rate ρ given in Theorem 3. The main claimed novelty is that this is the first linear-convergence guarantee for a single-loop bilevel algorithm, and the paper replaces the bounded-gradient assumption of prior work with a Lipschitz assumption on the approximate gradient.

Significance. If the proof were complete, this would be a valuable contribution: it gives an explicit, non-fitted linear rate for a single-loop method, brings a robust-control framework to bilevel optimization, and avoids the restrictive bounded-gradient assumption. The structure of the argument — component gain bounds, sector transformation, and small-gain application — is coherent and the step-size conditions are stated explicitly. However, the proof as written has a load-bearing gap in the construction of the transformed plant, a rate formula in Theorem 3 that is inconsistent with the proof algebra, and an overstatement in the claim that linear convergence is obtained under the SC-SC condition: Assumption 2.2 is an additional global regularity assumption that is not implied by the SC-SC hypotheses.

major comments (3)

[§3.3, proof of Theorem 3, Eq. (16)] The plant P' used to compute the H∞ norms is not the plant induced by the transformation M defined in (16). With x = M1 ξ and u = M2 ξ + M3 σ, where M1 = diag(d1,d2), d1² = 3λ1²/(4λ3) − a and d2² = λ2²/(4λ4) − b, the transformed dynamics are ξ_{k+1} = ξ_k − M1^{-1} diag(α,β)(M2 ξ + M3 σ). This yields state coefficients 1 − α(3λ1/(2λ3))/d1² and 1 − β(λ2/(2λ4))/d2², and input coefficients α√(3/λ3)/d1 and β/(√λ4 d2). The proof instead prints 1 − α(3λ1/(2λ3)) and input coefficients α√(9λ1²/(4λ3²) − 3a/λ3), β√(λ2²/(4λ4²) − b/λ4). These printed expressions correspond only to the normalization d1 = d2 = 1, which is not stated in Lemma 6 and is generally incompatible with the ratio λ1/λ2 = μg/(8(2Hv²/μf + 6Hv²α)) imposed later in the proof. Consequently the claimed H∞ norms and the resulting rate bound have not been established for the actual transformed system; the proof must either derive P' from (16) and recompute the gain, or justify a choice of λi for which d1 = d2 = 1 while conditions (12)--(14) hold.
[Theorem 3, second rate term] The rate formula printed in Theorem 3 is inconsistent with the proof's own algebra. The proof obtains b/λ2 = μg/4 − Lg²β and sets β = 2λ4/λ2, so the second rate factor is sqrt(1 − 4λ4 b/λ2²) = sqrt(1 − μgβ/2 + 2Lg²β²). The theorem statement prints sqrt(1 − (μgβ/2)(1 − 4Lg²β)) = sqrt(1 − μgβ/2 + 2μgLg²β²), which differs by a factor μg in the quadratic term. One of the two expressions must be corrected, and the subsequent verification of ρ < 1 should be rechecked accordingly.
[Assumption 2.2, Abstract, and Conclusion] The paper advertises linear convergence under the SC-SC condition, but Assumption 2.2 (global Hv-Lipschitzness of tilde∇f) is an additional structural assumption that does not follow from Assumptions 1, 2.1, and 3. A concrete counterexample is m=n=1, a(v)=1.5+0.5 sin(v²), Ψ''(v)=a(v), g(ω,v)=Ψ(v)+ωv+5ω², f(ω,v)=0.5ω²+g(ω,v). Here ∇²vv g = a(v) ∈ [1,2], so Assumptions 1.1 and 1.2 hold with μg=1, Lg=2; ∇²ωv g = 1, so Assumption 3 holds; and f*''(ω)=11−1/a(v*(ω)) ≥ 10, so Assumption 2.1 holds. However, tilde∇f(ω,v)=11ω+v−(Ψ'(v)+ω)/a(v), whose v-derivative is (Ψ'(v)+ω)a'(v)/a(v)², which is unbounded; hence no finite Hv exists. Thus Theorem 3 is a result under SC-SC plus an additional global Lipschitz condition on the approximate gradient, not a linear-convergence result under the SC-SC condition alone. The abstract and conclusion should be qualified, and the paper should state clearly whether Hv can be derived from standard third-order smoothness and uniform Hessian-inverse bounds.

minor comments (4)

[§2.1, Eq. (6)] There is a sign inconsistency: equation (5) is x_{k+1}=x_k−diag(α,β)ϕ(x_k), but the plant in (6) is written with B=diag(α,β) and K: u_k=ϕ(y_k), which would give x_{k+1}=x_k+diag(α,β)ϕ(x_k). The sign should be absorbed into B or into the definition of K.
[Lemma 5 proof] The sentence 'where the first inequality is due to Assumption 1.1, and the third one is due to Lemma 1 as well as (2)' refers to a third inequality that does not exist; the second inequality is the one using Lemma 1 and (2).
[Theorem 3 statement] The symbol w0 appears in the final bounds and should be ω0; the statement should also specify the dependence of cω and cv on the problem constants if that is intended.
[Abstract] There are several presentation issues in the abstract: 'We' is capitalized mid-sentence, and 'this work is first-known result' should be 'this work is the first-known result'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the linear convergence claim is a conditional theorem derived from explicit assumptions and external control-theoretic lemmas, with no fitted parameters or self-citation chain forcing the conclusion.

full rationale

The paper's central result, Theorem 3, is derived as a conditional statement: given Assumptions 1-3, step sizes satisfying the stated inequalities imply linear convergence of the iterates generated by (2). The proof does not fit any parameter to the convergence conclusion; instead, the constants in the step-size conditions are chosen to satisfy the sector inequalities in Lemma 6 and the small-gain product condition. The H∞ norm computations and the small-gain theorem (Lemma 3) are external standard results, and the cited lemmas (e.g., Lemma 1 from Liu et al., Lemma 2 from Boczar et al.) are used as independent mathematical facts, not as self-referential justifications. Assumption 2.2 introduces Hω and Hv as assumed Lipschitz constants; these are inputs to the theorem, not outputs calibrated to make the rate hold. The skeptic's example shows that Assumption 2.2 may not follow from Assumptions 1 and 2.1 alone, which is a question of assumption strength or correctness scope, not circularity: the theorem remains a genuine implication from its stated assumptions, and no step in the proof derives a conclusion that is equivalent to an input by construction. Accordingly, the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 8 assumptions · 0 invented entities

The central claim uses no fitted data. The constants μf, μg, Lg, H, Hω, Hv are assumptions of the problem class, not calibrated to observations. Step sizes α and β are algorithm parameters with explicit sufficient bounds. The auxiliary weights λ1-λ4 in Lemma 6 are proof-internal degrees of freedom and are eliminated in Theorem 3, so they are not counted as free parameters. The result depends on standard control theorems and the domain assumptions listed above.

assumptions (8)

standard math Small-gain theorem (Desoer and Vidyasagar)
Lemma 3; used to certify BIBO stability of the feedback interconnection.
standard math BIBO-stability implies exponential stability with rate ρ (Proposition 5 of Boczar et al.)
Lemma 2; converts stability of the ρ-scaled system into linear convergence of the original iteration.
standard math Implicit function theorem and differentiability of v*(ω)
Used in Section 2.1 to derive the approximate gradient and in Lemma 1 for Lipschitz continuity of v*.
standard math z-transform and H∞ norm identity for LTI systems
Used to compute the gain of the transformed linear system P' in Theorem 3.
domain assumption Strong convexity and smoothness of the lower-level objective (Assumption 1)
Ensures a unique minimizer v*(ω), a Lipschitz v*, and supports the inequalities in Lemma 5.
domain assumption Strong convexity of the upper-level objective f* (Assumption 2.1)
Provides the quadratic decrease in ω needed for Lemma 4 and the gain analysis.
domain assumption Lipschitz continuity of the approximate gradient (Assumption 2.2)
The load-bearing replacement for bounded-gradient assumptions; controls error terms in Lemmas 4 and 5. This assumption is the weakest structural premise.
domain assumption Bounded mixed Hessian ∇^2_{ωv}g (Assumption 3)
Gives Lipschitz continuity of v*(ω) via Lemma 1 with constant H/μg.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Linear Convergence Analysis of Single-loop Algorithm for Bilevel Optimization via Small-gain Theorem." pith.science (2026). https://pith.science/paper/35J52URG

@misc{pith2026241200659,
  author       = {Pith},
  title        = {Pith review of: Linear Convergence Analysis of Single-loop Algorithm for Bilevel Optimization via Small-gain Theorem},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/35J52URG}},
  note         = {Machine review of arXiv:2412.00659}
}

abstract

Bilevel optimization has gained considerable attention due to its broad applicability across various fields. While several studies have investigated the convergence rates in the strongly-convex-strongly-convex (SC-SC) setting, no prior work has proven that a single-loop algorithm can achieve linear convergence. This paper employs a small-gain theorem in {robust control theory} to demonstrate that a single-loop algorithm based on the implicit function theorem attains a linear convergence rate of $\mathcal{O}(\rho^{k})$, where $\rho\in(0,1)$ is specified in Theorem 3. Specifically, We model the algorithm as a dynamical system by identifying its two interconnected components: the controller (the gradient or approximate gradient functions) and the plant (the update rule of variables). We prove that each component exhibits a bounded gain and that, with carefully designed step sizes, their cascade accommodates a product gain strictly less than one. Consequently, the overall algorithm can be proven to achieve a linear convergence rate, as guaranteed by the small-gain theorem. The gradient boundedness assumption adopted in the single-loop algorithm (\cite{hong2023two, chen2022single}) is replaced with a gradient Lipschitz assumption in Assumption 2.2. To the best of our knowledge, this work is first-known result on linear convergence for a single-loop algorithm.

Figures

Figures reproduced from arXiv: 2412.00659 by the authors.

**Figure 1.** (a) The feedback interconnection of P and K with external inputs s and r. (b) The diagram of the system after implementing the linear transformation M, as specified by (16), onto the system depicted in (a). The inputs s and r are absent. (c) The feedback interconnection modified by introducing operators ρ− := ρ −k , ρ+ := ρ k for some given ρ ∈ (0, 1). where ω ∗ is the minimizer of f∗(·). Sequently, by introducing… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 14 canonical work pages

[1]

Amortized implicit differentiation for stochastic bilevel optimization

Michael Arbel and Julien Mairal. Amortized implicit differentiation for stochastic bilevel optimization. arXiv preprint arXiv:2111.14580, 2021

arXiv 2021
[2]

Exponential convergence bounds using integral quadratic constraints

Ross Boczar, Laurent Lessard, and Benjamin Recht. Exponential convergence bounds using integral quadratic constraints. In 2015 54th IEEE conference on decision and control (CDC), pages 7516--7521. IEEE, 2015

work page 2015
[3]

Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems

Tianyi Chen, Yuejiao Sun, and Wotao Yin. Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems. Advances in Neural Information Processing Systems, 34: 0 25294--25307, 2021

2021
[4]

A single-timescale method for stochastic bilevel optimization

Tianyi Chen, Yuejiao Sun, Quan Xiao, and Wotao Yin. A single-timescale method for stochastic bilevel optimization. In International Conference on Artificial Intelligence and Statistics, pages 2466--2488. PMLR, 2022

2022
[5]

Feedback systems: input-output properties

Charles A Desoer and Mathukumalli Vidyasagar. Feedback systems: input-output properties. SIAM, 2009

work page 2009
[6]

A course in robust control theory: a convex approach, volume 36

Geir E Dullerud and Fernando Paganini. A course in robust control theory: a convex approach, volume 36. Springer Science & Business Media, 2013

work page 2013
[7]

Forward and reverse gradient-based hyperparameter optimization

Luca Franceschi, Michele Donini, Paolo Frasconi, and Massimiliano Pontil. Forward and reverse gradient-based hyperparameter optimization. In International Conference on Machine Learning, pages 1165--1173. PMLR, 2017

2017
[8]

Bilevel programming for hyperparameter optimization and meta-learning

Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. In International conference on machine learning, pages 1568--1577. PMLR, 2018

2018

Show all 25 references

[9]

Approximation methods for bilevel programming

Saeed Ghadimi and Mengdi Wang. Approximation methods for bilevel programming. arXiv preprint arXiv:1802.02246, 2018

2018 arXiv
[10]

A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and application to actor-critic

Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and application to actor-critic. SIAM Journal on Optimization, 33 0 (1): 0 147--180, 2023

2023
[11]

Control interpretations for first-order optimization methods

Bin Hu and Laurent Lessard. Control interpretations for first-order optimization methods. In 2017 American Control Conference (ACC), pages 3114--3119. IEEE, 2017 a

2017
[12]

Dissipativity theory for nesterov’s accelerated method

Bin Hu and Laurent Lessard. Dissipativity theory for nesterov’s accelerated method. In International Conference on Machine Learning, pages 1549--1557. PMLR, 2017 b

2017
[13]

Metapoison: Practical general-purpose clean-label data poisoning

W Ronny Huang, Jonas Geiping, Liam Fowl, Gavin Taylor, and Tom Goldstein. Metapoison: Practical general-purpose clean-label data poisoning. Advances in Neural Information Processing Systems, 33: 0 12080--12091, 2020

2020
[14]

Lower bounds and accelerated algorithms for bilevel optimization

Kaiyi Ji and Yingbin Liang. Lower bounds and accelerated algorithms for bilevel optimization. Journal of machine learning research, 24 0 (22): 0 1--56, 2023

2023
[15]

Bilevel optimization: Convergence analysis and enhanced design

Kaiyi Ji, Junjie Yang, and Yingbin Liang. Bilevel optimization: Convergence analysis and enhanced design. In International conference on machine learning, pages 4882--4892. PMLR, 2021

2021
[16]

A near-optimal algorithm for stochastic bilevel optimization via double-momentum

Prashant Khanduri, Siliang Zeng, Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. A near-optimal algorithm for stochastic bilevel optimization via double-momentum. Advances in neural information processing systems, 34: 0 30271--30283, 2021

2021
[17]

Analysis and design of optimization algorithms via integral quadratic constraints

Laurent Lessard, Benjamin Recht, and Andrew Packard. Analysis and design of optimization algorithms via integral quadratic constraints. SIAM Journal on Optimization, 26 0 (1): 0 57--95, 2016

2016
[18]

A fully single loop algorithm for bilevel optimization without hessian inverse

Junyi Li, Bin Gu, and Heng Huang. A fully single loop algorithm for bilevel optimization without hessian inverse. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 7426--7434, 2022

2022
[19]

Inducing equilibria via incentives: Simultaneous design-and-play ensures global convergence

Boyi Liu, Jiayang Li, Zhuoran Yang, Hoi-To Wai, Mingyi Hong, Yu Nie, and Zhaoran Wang. Inducing equilibria via incentives: Simultaneous design-and-play ensures global convergence. Advances in Neural Information Processing Systems, 35: 0 29001--29013, 2022

2022
[20]

On lp-hyperparameter learning via bilevel nonsmooth optimization

Takayuki Okuno, Akiko Takeda, Akihiro Kawana, and Motokazu Watanabe. On lp-hyperparameter learning via bilevel nonsmooth optimization. Journal of Machine Learning Research, 22 0 (245): 0 1--47, 2021

2021
[21]

Adversarial training should be cast as a non-zero-sum game

Alexander Robey, Fabian Latorre, George J Pappas, Hamed Hassani, and Volkan Cevher. Adversarial training should be cast as a non-zero-sum game. arXiv preprint arXiv:2306.11035, 2023

2023 arXiv
[22]

Hydra: Pruning adversarially robust neural networks

Vikash Sehwag, Shiqi Wang, Prateek Mittal, and Suman Jana. Hydra: Pruning adversarially robust neural networks. Advances in Neural Information Processing Systems, 33: 0 19655--19666, 2020

2020
[23]

Reward design for driver repositioning using multi-agent reinforcement learning

Zhenyu Shou and Xuan Di. Reward design for driver repositioning using multi-agent reinforcement learning. Transportation research part C: emerging technologies, 119: 0 102738, 2020

2020
[24]

A gradient-based bilevel optimization approach for tuning hyperparameters in machine learning

Ankur Sinha, Tanmay Khandait, and Raja Mohanty. A gradient-based bilevel optimization approach for tuning hyperparameters in machine learning. arXiv preprint arXiv:2007.11022, 2020

2007 arXiv
[25]

Advancing model pruning via bi-level optimization

Yihua Zhang, Yuguang Yao, Parikshit Ram, Pu Zhao, Tianlong Chen, Mingyi Hong, Yanzhi Wang, and Sijia Liu. Advancing model pruning via bi-level optimization. Advances in Neural Information Processing Systems, 35: 0 18309--18326, 2022

2022

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Amortized implicit differentiation for stochastic bilevel optimization

Michael Arbel and Julien Mairal. Amortized implicit differentiation for stochastic bilevel optimization. arXiv preprint arXiv:2111.14580, 2021

arXiv 2021

[2] [2]

Exponential convergence bounds using integral quadratic constraints

Ross Boczar, Laurent Lessard, and Benjamin Recht. Exponential convergence bounds using integral quadratic constraints. In 2015 54th IEEE conference on decision and control (CDC), pages 7516--7521. IEEE, 2015

work page 2015

[3] [3]

Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems

Tianyi Chen, Yuejiao Sun, and Wotao Yin. Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems. Advances in Neural Information Processing Systems, 34: 0 25294--25307, 2021

2021

[4] [4]

A single-timescale method for stochastic bilevel optimization

Tianyi Chen, Yuejiao Sun, Quan Xiao, and Wotao Yin. A single-timescale method for stochastic bilevel optimization. In International Conference on Artificial Intelligence and Statistics, pages 2466--2488. PMLR, 2022

2022

[5] [5]

Feedback systems: input-output properties

Charles A Desoer and Mathukumalli Vidyasagar. Feedback systems: input-output properties. SIAM, 2009

work page 2009

[6] [6]

A course in robust control theory: a convex approach, volume 36

Geir E Dullerud and Fernando Paganini. A course in robust control theory: a convex approach, volume 36. Springer Science & Business Media, 2013

work page 2013

[7] [7]

Forward and reverse gradient-based hyperparameter optimization

Luca Franceschi, Michele Donini, Paolo Frasconi, and Massimiliano Pontil. Forward and reverse gradient-based hyperparameter optimization. In International Conference on Machine Learning, pages 1165--1173. PMLR, 2017

2017

[8] [8]

Bilevel programming for hyperparameter optimization and meta-learning

Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. In International conference on machine learning, pages 1568--1577. PMLR, 2018

2018