Pith. sign in

REVIEW 5 minor 58 references

Penalty limits of the regularized gap-function reformulation of bilevel programs remain C-stationary for the equivalent MPCC even when the gap multiplier diverges, and a slack-based two-parameter penalty upgrades this to M-stationarity.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 17:04 UTC pith:AHWD64T5

load-bearing objection Solid first general treatment of unbounded gap-multipliers for constrained convex lower levels; C-stationarity sharp, slack formulation gets M-stationarity, but the load-bearing MPCC-MFCQ at the limit is uncheckable and the algorithm is unvalidated.

arxiv 2607.17772 v1 pith:AHWD64T5 submitted 2026-07-20 math.OC

Limiting Stationarity of Regularized Gap-Function Reformulations for Bilevel Optimization with Unbounded Multipliers

Xiaoning Bai , Shangzhi Zeng , Jin Zhang This is my paper
classification math.OC MSC 90C4690C33
keywords bilevel optimizationregularized gap functionMPCCC-stationarityM-stationarityunbounded multiplierspenalty methodsconstraint qualifications
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Bilevel optimization problems are often solved by replacing the lower-level optimality condition y∈S(x) with a single scalar regularized gap-function constraint Gγ(x,y,z)≤0. Because this constraint is inherently degenerate, its Lagrange multiplier can diverge as the penalty parameter grows, breaking the standard bounded-multiplier convergence theory. The paper proves that this unboundedness does not destroy all stationarity information: accumulation points of approximate stationary sequences are C-stationary for the equivalent KKT-based MPCC reformulation, provided the upper- and lower-level constraint systems satisfy MFCQ and the limiting point satisfies MPCC-MFCQ. A constructed example shows the standard one-parameter penalty can only guarantee C-stationarity, not the stronger M-stationarity. The paper then introduces a slack-based two-parameter penalty that preserves exact multiplier-slack complementarity and, when the second penalty parameter dominates the first, forces M-stationary limits; an inexact slack-penalty algorithm built on this formulation inherits the guarantee.

Core claim

The central claim is that unbounded multipliers can be tamed. Under Assumptions 2.1–2.2, let (x̄,ȳ) be an accumulation point of approximate KKT points of the regularized gap reformulation, with the approximate KKT inclusion (21) holding for ε_k→0 and ρ_k>0. The paper proves (Theorem 3.3) that for any accumulation z̄ of the lower-level multiplier sequence, w̄=(x̄,ȳ,z̄) is feasible for the MPCC reformulation (3), and: if ρ_k has a bounded subsequence then w̄ is S-stationary; if ρ_k→∞ and MPCC-MFCQ holds at w̄, then w̄ is C-stationary. Here C-stationarity means that on the biactive index set—where both z_i=0 and g_i(x,y)=0—the product of the corresponding multipliers η_i and ∇y g_i^T ω is nonne

What carries the argument

The regularized gap function Gγ(x,y,z)=max over (θ,λ) of [L(x,y,λ)−∥λ−z∥²/(2γ2)−L(x,θ,z)−∥θ−y∥²/(2γ1)] is nonnegative and vanishes exactly at lower-level KKT points; it is continuously differentiable and admits the closed-form λ*(x,y,z)=Proj_{R_+^q}(z+γ2 g(x,y)). The proof's load-bearing pieces are: (i) Lemma 2.12, the no-nonzero-abnormal-multiplier form of MPCC-MFCQ, used to rule out normalized limits of unbounded multiplier sequences; (ii) a Taylor-expansion residual r_k that translates ρ_k∇Gγ into the MPCC multiplier system (22); (iii) the componentwise limiting normal cone of the complementarity set C, which gives the sign conditions separating C- from M-stationarity; and (iv) for the up

Load-bearing premise

The load-bearing premise is that the limiting MPCC point satisfies MPCC-MFCQ (Definition 2.11), an uncheckable-before-solving regularity condition; if it fails, the no-abnormal-multiplier lemma cannot be invoked to make unbounded multiplier sequences vanish, and neither C- nor M-stationarity is guaranteed by the proof.

What would settle it

Find a bilevel instance satisfying upper- and lower-level MFCQ at the limit, with a sequence of approximate stationary points of the standard penalty (19), ρ_k→∞, whose accumulation point w̄ is feasible for the MPCC reformulation but not C-stationary—e.g., with η_i ν_i<0 on a biactive index. Theorem 3.3 predicts no such instance exists; one example would refute it. A less drastic test is to construct a limit where MPCC-MFCQ fails and check whether C-stationarity is lost; the paper's Example 3.5 already shows that M-stationarity can fail without the slack formulation, so the C/M distinction is

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Boundedness of the gap-function multiplier is unnecessary for a meaningful limiting-stationarity theory: Theorem 3.3 shows C-stationarity survives ρ_k→∞ whenever MPCC-MFCQ holds at the limit.
  • The one-parameter penalty cannot be strengthened to M-stationarity in general, since Example 3.5 has a unique limit point that is C- but not M-stationary.
  • The two-parameter slack reformulation (GP)_C closes the gap: with ρ2/ρ1→∞ and the same constraint qualifications, approximate stationary limits are M-stationary.
  • The inexact algorithm iG-BSPM terminates its inner loop finitely and produces an approximate KKT sequence, so the M-stationary conclusion carries over to a practical solver under the paper's assumptions.
  • None of these results requires a constraint qualification on the regularized gap-function constraint itself, which is the degenerate object that motivated the paper.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper leaves implicit: any algorithm whose iterates satisfy the abstract approximate KKT inclusion (21) should inherit the C- or M-stationarity limits, so the result functions as a template for verifying limiting stationarity of new penalty or relaxation methods for bilevel programs.
  • The domination condition ρ2/ρ1→∞ is a ratio, not an absolute scale; this suggests the slack reformulation's M-stationarity guarantee is scale-robust, and finite-penalty behavior—how large the ratio must be for a given accuracy—is a testable numerical question the paper does not address.
  • Since the upper-level MFCQ and MPCC-MFCQ are used only to exclude abnormal multiplier limits, replacing them with weaker qualifiability assumptions (for instance, adding a calmness condition) might extend the arguments to problems where the upper-level feasible set is degenerate.
  • The slack variable decouples the lower-level inequality from the complementarity set, so (z,s) can be updated via the closed-form projection onto C, as iG-BSPM does; this makes the two-parameter formulation a promising shell for stochastic or primal-dual bilevel algorithms.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 5 minor

Summary. The paper studies bilevel optimization problems with a convex lower level, focusing on the regularized gap-function reformulation (GP) in which the implicit lower-level optimality constraint is encoded by Gγ(x,y,z)≤0. Because value-function-type constraints are degenerate, the multiplier ρ associated with Gγ can be unbounded, and standard bounded-multiplier analyses do not apply. The paper introduces an approximate KKT condition for (GP), Definition 3.1, and proves in Theorem 3.3 that, under MFCQ for the upper- and lower-level constraint systems and MPCC-MFCQ at the limiting MPCC point, accumulation points are C-stationary for the KKT-based MPCC reformulation (3); the bounded-multiplier case gives S-stationarity. Example 3.5 shows that C-stationarity is sharp for the standard penalty formulation. To obtain the stronger M-stationarity, the authors introduce a slack-based two-parameter formulation (39) and its approximate KKT condition, Definition 4.1, and prove in Theorem 4.3 that when ρ1→∞, ρ2→∞, and ρ2/ρ1→∞, accumulation points are M-stationary for (3). The final technical section develops an inexact regularized gap-function bilevel slack-penalty method (iG-BSPM) with adaptive penalty updates, feasibility correction, and inexact inner solves; Proposition 5.10 and Theorem 5.12 establish feasibility and M-stationarity of its accumulation points under the stated assumptions.

Significance. If the proofs are correct, the paper makes a solid contribution: it shows that unbounded multipliers for the regularized gap-function constraint do not necessarily destroy all limiting stationarity information. The normalization argument under MPCC-MFCQ is the key step, and I verified the algebra in Lemma 3.2 and Theorem 3.3, including the nonconvex projection inequality used in Section 5. The two-parameter slack formulation is a natural and useful enhancement, and the example demonstrating C- but not M-stationarity is correct and gives sharpness. The algorithm section is extensive and provides a full convergence analysis under explicit, albeit strong, assumptions. The main limitation is the reliance on MPCC-MFCQ at a limit point that is not known a priori; this is an explicit hypothesis rather than an internal inconsistency, and it is the standard price for controlling unbounded multipliers. Overall, the central claims are defensible and the technical execution is careful.

minor comments (5)
  1. [Lemma 3.2 (statement and proof)] The residual condition is written as “‖r^k‖/|θ*_k−y^k‖ → 0”, which mixes a vector norm with an absolute value and has a mismatched bracket. It should read ‖r^k‖/‖θ*_k−y^k‖ → 0” (or the equivalent). The same typo appears in the proof.
  2. [Algorithm 1, Step 16, and Theorem 5.12] The algorithm has a finite stopping tolerance ε_stop>0. If it triggers, the generated sequence is eventually constant, and the stopping condition does not enforce a zero first-order residual. Therefore Theorem 5.12’s claim that every accumulation point satisfies Definition 4.1 requires the algorithm to execute indefinitely (or, equivalently, ε_stop=0 for the asymptotic statement). Please state this explicitly; the proof itself already assumes infinite outer iterations when it passes to an accumulation subsequence.
  3. [Theorem 5.12, proof of the ξ^k_{xy} estimate] The bound on ‖∇_{x,y}ψ − ∇_{x,y}ψ̂‖ uses a Lipschitz property of the solution mapping θ*(x,y,z) with respect to (x,y). Lemma 5.5 states only the z-Lipschitz property. The (x,y)-Lipschitz estimate follows from strong convexity and the C^2 assumptions, but it is not stated as a lemma. Adding a short lemma or an in-text justification would make the proof self-contained.
  4. [Example 3.5] The example is used to show that Theorem 3.3’s C-stationarity conclusion is sharp. It would be clearer to state explicitly that the limit point ˜w=(0,0,0) satisfies MPCC-MFCQ, so that the example is within the assumptions of the theorem. This is true (the tightened constraints are y=0, z=0, and y−x−z=0, whose gradients are linearly independent), but the verification is not included.
  5. [Lemma 5.7] The proof of concavity of P_{γ_2} is compressed: the sentence “since −P_{γ_2} is the partial minimum over λ of a jointly convex function” is correct but may be misread. A one-line direct derivative argument (or a reference to the piecewise-linear derivative of the PHR penalty) would improve clarity.

Circularity Check

0 steps flagged

No circularity; the stationarity transfer is a genuine derivation under explicit MPCC-MFCQ assumptions.

full rationale

The central claim is a transfer theorem: approximate KKT sequences for the regularized gap reformulation are shown, via a nontrivial multiplier normalization, to have limits satisfying C/M-stationarity for the KKT-based MPCC. The proof uses regularized gap function properties from [50] (Lemmas 2.3, 2.5), which are shared-author prior results, but those lemmas only characterize Gγ=0 and give gradients; they do not contain the target stationarity statements. No parameter is fitted to the conclusion, and no uniqueness theorem from the same authors is used to force the choice: the only load-bearing qualification is MPCC-MFCQ, a standard external CQ, used through the abnormal-multiplier exclusion in Lemma 2.12. The unbounded-multiplier step divides (22)/(42) by M_k and obtains exactly the abnormal system (16)-(17), which the assumed qualification rules out; this is a contradiction argument, not a definitional identity. The C-stationarity sign condition (38) follows from the z-component relations (23), and the M-stationarity upgrade uses the slack structure and ρ2/ρ1→∞. All assumptions are explicit and checkable only at the limit; fragility of MPCC-MFCQ is a regularity risk, not circularity. Thus no circular step can be exhibited.

Axiom & Free-Parameter Ledger

0 free parameters · 6 axioms · 0 invented entities

The central claims depend on the listed domain assumptions (smoothness, convexity, constraint qualifications, compactness, existence of lower-level KKT multipliers) and on the regularized gap-function properties cited from [50]. There are no fitted free parameters: γ1, γ2 and penalty/tolerance parameters are user-specified and results hold uniformly over positive choices or under stated asymptotic regimes. No new physical or mathematical entities are introduced; the slack variable is a standard reformulation device.

axioms (6)
  • domain assumption Assumption 2.1: F and G_i are continuously differentiable, G_i convex on a neighborhood of Ω.
    Smoothness and convexity of upper-level data used throughout; introduced in Section 2.1.
  • domain assumption Assumption 2.2: For each x, f(x,·) and g_i(x,·) are convex and twice continuously differentiable.
    Ensures the regularized gap function Gγ is well-defined, differentiable, and that θ*(x,y,z) is unique (Sections 2.1-2.2).
  • domain assumption Regularized gap function properties from [50]: Gγ≥0, Gγ=0 iff y∈S(x) and z∈M(x,y); gradient formula (7).
    Taken from [50], which shares two co-authors (Zeng, Zhang). The new theorems are built on these lemmas but are not equivalent to them.
  • domain assumption MFCQ for the lower-level constraint system and upper-level inequality system, and MPCC-MFCQ at the limiting MPCC point (Theorems 3.3, 4.3).
    Required to bound normal-cone multipliers and to rule out nonzero abnormal multipliers via Lemma 2.12; a condition on the unknown limit point.
  • domain assumption Assumptions 5.1-5.3: Ω nonempty compact convex; ∇F Lipschitz; for every x∈X an upper-level feasible lower-level KKT point exists.
    Used for the iG-BSPM algorithm: compactness for inner-loop convergence, Lipschitz continuity for line search, and existence for feasibility correction.
  • standard math Standard variational analysis facts: limiting normal cone formula for the complementarity set C (Lemma 2.8, from [53]); Taylor's theorem; strong convexity; MFCQ stability.
    Background results invoked in the proofs; no new content.

pith-pipeline@v1.3.0-alltime-deepseek · 47016 in / 22295 out tokens · 222227 ms · 2026-08-01T17:04:27.868723+00:00 · methodology

0 comments
read the original abstract

Value-function-type reformulations have generated a broad class of methods for bilevel optimization. However, the corresponding value-function-type constraints are inherently degenerate and generally fail to satisfy standard constraint qualifications, so the associated multiplier sequences may be unbounded and bounded-multiplier convergence analyses become inapplicable. We study this issue for the regularized gap-function reformulation of bilevel problems with constrained convex lower-level programs. We prove that accumulation points of approximate stationary sequences are C-stationary for the corresponding Karush-Kuhn-Tucker-based mathematical program with complementarity constraints (MPCC), even when the multiplier sequence associated with the regularized gap-function constraint is unbounded. The result holds under Mangasarian-Fromovitz constraint qualification (MFCQ) for the upper- and lower-level constraint systems and MPCC-MFCQ at the limiting MPCC point, without any constraint qualification on the regularized gap-function constraint itself. We further provide an example showing that approximate stationary points of the standard regularized gap-function reformulation may converge to a point that is C-stationary but not M-stationary. To guarantee M-stationarity, we introduce a slack-based two-parameter penalty formulation preserving exact multiplier-slack complementarity and establish M-stationarity under a domination condition on the penalty parameters. We develop an inexact slack-penalty method with adaptive penalty updates and feasibility correction, whose accumulation points are M-stationary under the stated assumptions.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 2 linked inside Pith

  1. [1]

    Directional necessary optimality conditions for bilevel programs

    Kuang Bai and Jane J Ye. Directional necessary optimality conditions for bilevel programs. Mathematics of Operations Research, 47(2):1169–1191, 2022

    2022

  2. [2]

    Optimality conditions for bilevel programmes via Moreau envelope reformulation.Optimization, 74(12):2685–2719, 2025

    Kuang Bai, Jane J Ye, and Shangzhi Zeng. Optimality conditions for bilevel programmes via Moreau envelope reformulation.Optimization, 74(12):2685–2719, 2025

    2025

  3. [3]

    Xiaoning Bai, Shangzhi Zeng, Jin Zhang, and Lezhi Zhang. Alternating gradient-type algo- rithm for bilevel optimization with inexact lower-level solutions via Moreau envelope–based reformulation.SIAM Journal on Optimization, 36(1):350–380, 2026

    2026

  4. [4]

    Springer, New York, 1998

    Jonathan F Bard.Practical bilevel optimization: Algorithms and applications. Springer, New York, 1998

    1998

  5. [5]

    SIAM, Philadelphia, 2017

    Amir Beck.First-order methods in optimization. SIAM, Philadelphia, 2017

    2017

  6. [6]

    Bennett, Gautam Kunapuli, Jing Hu, and Jong-Shi Pang

    Kristin P. Bennett, Gautam Kunapuli, Jing Hu, and Jong-Shi Pang. Bilevel optimization and machine learning. InIEEE World Congress on Computational Intelligence, pages 25–47, 2008

    2008

  7. [7]

    Fr´ ed´ eric Bonnans and Alexander Shapiro.Perturbation analysis of optimization problems

    J. Fr´ ed´ eric Bonnans and Alexander Shapiro.Perturbation analysis of optimization problems. Springer, New York, 2000

    2000

  8. [8]

    An overview of bilevel optimization

    Beno ˆ ıt Colson, Patrice Marcotte, and Gilles Savard. An overview of bilevel optimization. Annals of Operations Research, 153:235–256, 2007

    2007

  9. [9]

    Springer, New York, 2002

    Stephan Dempe.Foundations of bilevel programming. Springer, New York, 2002

    2002

  10. [10]

    Is bilevel programming a special case of a mathematical program with complementarity constraints?Mathematical Programming, 131(1):37–48, 2012

    Stephan Dempe and Joydeep Dutta. Is bilevel programming a special case of a mathematical program with complementarity constraints?Mathematical Programming, 131(1):37–48, 2012

    2012

  11. [11]

    Duality-based single-level reformulations of bilevel op- timization problems.Journal of Optimization Theory and Applications, 205(2):26, 2025

    Stephan Dempe and Patrick Mehlitz. Duality-based single-level reformulations of bilevel op- timization problems.Journal of Optimization Theory and Applications, 205(2):26, 2025

    2025

  12. [12]

    On the Karush–Kuhn–Tucker reformulation of the bilevel optimization problem.Nonlinear Analysis: Theory, Methods & Applications, 75(3):1202–1218, 2012

    Stephan Dempe and Alain B Zemkoho. On the Karush–Kuhn–Tucker reformulation of the bilevel optimization problem.Nonlinear Analysis: Theory, Methods & Applications, 75(3):1202–1218, 2012

    2012

  13. [13]

    The bilevel programming problem: reformulations, constraint qualifications and optimality conditions.Mathematical Programming, 138(1):447– 473, 2013

    Stephan Dempe and Alain B Zemkoho. The bilevel programming problem: reformulations, constraint qualifications and optimality conditions.Mathematical Programming, 138(1):447– 473, 2013

    2013

  14. [14]

    Springer, Cham, 2020

    Stephan Dempe and Alain B Zemkoho.Bilevel optimization: Advances and next challenges. Springer, Cham, 2020

    2020

  15. [15]

    On M-stationary points for mathematical programs with equilibrium constraints.Journal of Mathematical Analysis and Applications, 310(1):286– 302, 2005

    Michael L Flegel and Christian Kanzow. On M-stationary points for mathematical programs with equilibrium constraints.Journal of Mathematical Analysis and Applications, 310(1):286– 302, 2005

    2005

  16. [16]

    Bilevel programming for hyperparameter optimization and meta-learning

    Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. InInternational Conference on Machine Learning, pages 1568–1577, 2018. 35

    2018

  17. [17]

    Value function based difference-of-convex algorithm for bilevel hyperparameter selection problems

    Lucy L Gao, Jane J Ye, Haian Yin, Shangzhi Zeng, and Jin Zhang. Value function based difference-of-convex algorithm for bilevel hyperparameter selection problems. InInternational Conference on Machine Learning, pages 7164–7182, 2022

    2022

  18. [18]

    Moreau envelope-based difference of weakly convex reformulation and algorithm for bilevel programs.Journal of the Operations Research Society of China, pages 1–37, 2026

    Lucy L Gao, Jane J Ye, Haian Yin, Shangzhi Zeng, and Jin Zhang. Moreau envelope-based difference of weakly convex reformulation and algorithm for bilevel programs.Journal of the Operations Research Society of China, pages 1–37, 2026

    2026

  19. [19]

    Inexact bilevel stochastic gradient methods for constrained and unconstrained lower-level problems.Journal of Global Optimization, 92(3):569–614, 2025

    Tommaso Giovannelli, Griffin Dean Kent, and Luis Nunes Vicente. Inexact bilevel stochastic gradient methods for constrained and unconstrained lower-level problems.Journal of Global Optimization, 92(3):569–614, 2025

    2025

  20. [20]

    Multiplier and gradient methods.Journal of Optimization Theory and Applications, 4(5):303–320, 1969

    Magnus R Hestenes. Multiplier and gradient methods.Journal of Optimization Theory and Applications, 4(5):303–320, 1969

    1969

  21. [21]

    An improved unconstrained ap- proach for bilevel optimization.SIAM Journal on Optimization, 33(4):2801–2829, 2023

    Xiaoyin Hu, Nachuan Xiao, Xin Liu, and Kim-Chuan Toh. An improved unconstrained ap- proach for bilevel optimization.SIAM Journal on Optimization, 33(4):2801–2829, 2023

    2023

  22. [22]

    A barrier function approach for bilevel optimization with coupled lower-level constraints: Formulation, approximation and algorithms.arXiv preprint arXiv:2410.10670, 2024

    Xiaotian Jiang, Jiaxiang Li, Mingyi Hong, and Shuzhong Zhang. A barrier function approach for bilevel optimization with coupled lower-level constraints: Formulation, approximation and algorithms.arXiv preprint arXiv:2410.10670, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Approximate stationarity in disjunctive optimization: concepts, qualification conditions, and application to MPCCs.Set-Valued and Variational Analysis, 33(4):48, 2025

    Isabella K¨ aming and Patrick Mehlitz. Approximate stationarity in disjunctive optimization: concepts, qualification conditions, and application to MPCCs.Set-Valued and Variational Analysis, 33(4):48, 2025

    2025

  24. [24]

    A new regularization method for mathematical programs with complementarity constraints with strong convergence properties.SIAM Journal on Optimization, 23(2):770–798, 2013

    Christian Kanzow and Alexandra Schwartz. A new regularization method for mathematical programs with complementarity constraints with strong convergence properties.SIAM Journal on Optimization, 23(2):770–798, 2013

    2013

  25. [25]

    Linearly constrained bilevel optimization: A smoothed implicit gradient ap- proach

    Prashant Khanduri, Ioannis Tsaknakis, Yihua Zhang, Jia Liu, Sijia Liu, Jiawei Zhang, and Mingyi Hong. Linearly constrained bilevel optimization: A smoothed implicit gradient ap- proach. InInternational Conference on Machine Learning, pages 16291–16325, 2023

    2023

  26. [26]

    First-order methods for linearly constrained bilevel optimization.Advances in Neural Information Pro- cessing Systems, 37:141417–141460, 2024

    Guy Kornowski, Swati Padmanabhan, Kai Wang, Jimmy Zhang, and Suvrit Sra. First-order methods for linearly constrained bilevel optimization.Advances in Neural Information Pro- cessing Systems, 37:141417–141460, 2024

    2024

  27. [27]

    A fully first-order method for stochastic bilevel optimization

    Jeongyeol Kwon, Dohyun Kwon, Stephen Wright, and Robert D Nowak. A fully first-order method for stochastic bilevel optimization. InInternational Conference on Machine Learning, pages 18083–18113, 2023

    2023

  28. [28]

    On penalty methods for nonconvex bilevel optimization and first-order stochastic approximation

    Jeongyeol Kwon, Dohyun Kwon, Stephen Wright, and Robert D Nowak. On penalty methods for nonconvex bilevel optimization and first-order stochastic approximation. InInternational Conference on Learning Representations, 2024

    2024

  29. [29]

    Bilevel hyperparameter optimization for support vector classification: theoretical analysis and a solution method.Mathematical Methods of Operations Research, 96(3):315–350, 2022

    Qingna Li, Zhen Li, and Alain B Zemkoho. Bilevel hyperparameter optimization for support vector classification: theoretical analysis and a solution method.Mathematical Methods of Operations Research, 96(3):315–350, 2022

    2022

  30. [30]

    BOME! Bilevel optimiza- tion made easy: A simple first-order approach.Advances in Neural Information Processing Systems, 35:17248–17262, 2022

    Bo Liu, Mao Ye, Stephen Wright, Peter Stone, and Qiang Liu. BOME! Bilevel optimiza- tion made easy: A simple first-order approach.Advances in Neural Information Processing Systems, 35:17248–17262, 2022

    2022

  31. [31]

    Moreau envelope for noncon- vex bi-level optimization: A single-loop and Hessian-free solution strategy

    Risheng Liu, Zhu Liu, Wei Yao, Shangzhi Zeng, and Jin Zhang. Moreau envelope for noncon- vex bi-level optimization: A single-loop and Hessian-free solution strategy. InInternational Conference on Machine Learning, pages 31566–31596, 2024

    2024

  32. [32]

    TSP: A two-sided smoothed primal-dual method for nonconvex bilevel optimiza- tion

    Songtao Lu. TSP: A two-sided smoothed primal-dual method for nonconvex bilevel optimiza- tion. InInternational Conference on Machine Learning, pages 40665–40708, 2025. 36

    2025

  33. [33]

    First-order penalty methods for bilevel optimization.SIAM Journal on Optimization, 34(2):1937–1969, 2024

    Zhaosong Lu and Sanyou Mei. First-order penalty methods for bilevel optimization.SIAM Journal on Optimization, 34(2):1937–1969, 2024

    1937

  34. [34]

    Cambridge University Press, 1996

    Zhi-Quan Luo, Jong-Shi Pang, and Daniel Ralph.Mathematical programs with equilibrium constraints. Cambridge University Press, 1996

    1996

  35. [35]

    Asymptotic regularity for Lipschitzian nonlinear optimization problems with applications to complementarity constrained and bilevel programming.Optimization, 72(1):277–320, 2023

    Patrick Mehlitz. Asymptotic regularity for Lipschitzian nonlinear optimization problems with applications to complementarity constrained and bilevel programming.Optimization, 72(1):277–320, 2023

    2023

  36. [36]

    Proximit´ e et dualit´ e dans un espace hilbertien.Bulletin de la Soci´ et´ e Math´ ematique de France, 93:273–299, 1965

    Jean-Jacques Moreau. Proximit´ e et dualit´ e dans un espace hilbertien.Bulletin de la Soci´ et´ e Math´ ematique de France, 93:273–299, 1965

    1965

  37. [37]

    An augmented Lagrangian value function method for lower-level constrained stochastic bilevel optimization.arXiv preprint arXiv:2509.24249, 2025

    Hantao Nie, Jiaxiang Li, and Zaiwen Wen. An augmented Lagrangian value function method for lower-level constrained stochastic bilevel optimization.arXiv preprint arXiv:2509.24249, 2025

    arXiv 2025

  38. [38]

    On the numerical solution of a class of Stackelberg problems.Zeitschrift f¨ ur Operations Research, 34(4):255–277, 1990

    Jiˇ r ´ ı V Outrata. On the numerical solution of a class of Stackelberg problems.Zeitschrift f¨ ur Operations Research, 34(4):255–277, 1990

    1990

  39. [39]

    Optimality conditions for a class of mathematical programs with equilibrium constraints.Mathematics of Operations Research, 24(3):627–644, 1999

    Jiˇ r ´ ı V Outrata. Optimality conditions for a class of mathematical programs with equilibrium constraints.Mathematics of Operations Research, 24(3):627–644, 1999

    1999

  40. [40]

    A method for nonlinear constraints in minimization problems

    Michael J D Powell. A method for nonlinear constraints in minimization problems. InOpti- mization, pages 283–298. Academic Press, London, 1969

    1969

  41. [41]

    Augmented Lagrangians and applications of the proximal point algo- rithm in convex programming.Mathematics of Operations Research, 1(2):97–116, 1976

    R Tyrrell Rockafellar. Augmented Lagrangians and applications of the proximal point algo- rithm in convex programming.Mathematics of Operations Research, 1(2):97–116, 1976

    1976

  42. [42]

    Tyrrell Rockafellar and Roger J.-B

    R. Tyrrell Rockafellar and Roger J.-B. Wets.Variational Analysis. Springer, Berlin, 1998

    1998

  43. [43]

    Mathematical programs with complementarity constraints: Stationarity, optimality, and sensitivity.Mathematics of Operations Research, 25(1):1–22, 2000

    Holger Scheel and Stefan Scholtes. Mathematical programs with complementarity constraints: Stationarity, optimality, and sensitivity.Mathematics of Operations Research, 25(1):1–22, 2000

    2000

  44. [44]

    On penalty-based bilevel gradient descent method

    Han Shen and Tianyi Chen. On penalty-based bilevel gradient descent method. InInterna- tional Conference on Machine Learning, pages 30992–31015, 2023

    2023

  45. [45]

    On penalty-based bilevel gradient descent method

    Han Shen, Quan Xiao, and Tianyi Chen. On penalty-based bilevel gradient descent method. Mathematical Programming, 214(1):539–589, 2025

    2025

  46. [46]

    Double momentum method for lower-level constrained bilevel optimization

    Wanli Shi, Yi Chang, and Bin Gu. Double momentum method for lower-level constrained bilevel optimization. InInternational Conference on Machine Learning, pages 44838–44864, 2024

    2024

  47. [47]

    A primal-dual approach to bilevel optimization with multiple inner minima.arXiv preprint arXiv:2203.01123, 2022

    Daouda Sow, Kaiyi Ji, Ziwei Guan, and Yingbin Liang. A primal-dual approach to bilevel optimization with multiple inner minima.arXiv preprint arXiv:2203.01123, 2022

    Pith/arXiv arXiv 2022

  48. [48]

    Alternating projected SGD for equality- constrained bilevel optimization

    Quan Xiao, Han Shen, Wotao Yin, and Tianyi Chen. Alternating projected SGD for equality- constrained bilevel optimization. InInternational Conference on Artificial Intelligence and Statistics, pages 987–1023, 2023

    2023

  49. [49]

    First-order federated bilevel learning

    Yifan Yang, Peiyao Xiao, Shiqian Ma, and Kaiyi Ji. First-order federated bilevel learning. In AAAI Conference on Artificial Intelligence, pages 22029–22037, 2025

    2025

  50. [50]

    Overcoming lower-level constraints in bilevel optimization: A novel approach with regularized gap functions

    Wei Yao, Haian Yin, Shangzhi Zeng, and Jin Zhang. Overcoming lower-level constraints in bilevel optimization: A novel approach with regularized gap functions. InInternational Conference on Learning Representations, 2025

    2025

  51. [51]

    Constrained bi-level optimization: Proximal Lagrangian value function approach and Hessian-free algorithm

    Wei Yao, Chengming Yu, Shangzhi Zeng, and Jin Zhang. Constrained bi-level optimization: Proximal Lagrangian value function approach and Hessian-free algorithm. InInternational Conference on Learning Representations, 2024. 37

    2024

  52. [52]

    Optimality conditions for optimization problems with complementarity constraints

    Jane J Ye. Optimality conditions for optimization problems with complementarity constraints. SIAM Journal on Optimization, 9(2):374–387, 1999

    1999

  53. [53]

    Constraint qualifications and necessary optimality conditions for optimization problems with variational inequality constraints.SIAM Journal on Optimization, 10(4):943– 962, 2000

    Jane J Ye. Constraint qualifications and necessary optimality conditions for optimization problems with variational inequality constraints.SIAM Journal on Optimization, 10(4):943– 962, 2000

    2000

  54. [54]

    Necessary and sufficient optimality conditions for mathematical programs with equilibrium constraints.Journal of Mathematical Analysis and Applications, 307(1):350–369, 2005

    Jane J Ye. Necessary and sufficient optimality conditions for mathematical programs with equilibrium constraints.Journal of Mathematical Analysis and Applications, 307(1):350–369, 2005

    2005

  55. [55]

    Constraint qualifications and optimality conditions in bilevel optimization

    Jane J Ye. Constraint qualifications and optimality conditions in bilevel optimization. In Bilevel optimization: Advances and next challenges, pages 227–251. Springer, 2020

    2020

  56. [56]

    Optimality conditions for bilevel programming problems.Optimiza- tion, 33(1):9–27, 1995

    Jane J Ye and Daoli Zhu. Optimality conditions for bilevel programming problems.Optimiza- tion, 33(1):9–27, 1995

    1995

  57. [57]

    New necessary optimality conditions for bilevel programs by com- bining the MPEC and value function approaches.SIAM Journal on Optimization, 20(4):1885– 1905, 2010

    Jane J Ye and Daoli Zhu. New necessary optimality conditions for bilevel programs by com- bining the MPEC and value function approaches.SIAM Journal on Optimization, 20(4):1885– 1905, 2010

    1905

  58. [58]

    Federated stochastic bilevel optimization with fully first-order gradients

    Yihan Zhang, Rohit Dhaipule, Chiu C Tan, Haibin Ling, and Hongchang Gao. Federated stochastic bilevel optimization with fully first-order gradients. InInternational Joint Confer- ence on Artificial Intelligence, pages 7047–7055, 2025. 38

    2025