REVIEW 4 major objections 4 minor 106 references
Beyond Value Functions: Single-Loop Bilevel Optimization under Flatness Conditions
T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A single-loop, Hessian-free bilevel algorithm reaches ε-stationarity in O(ε^{-1}) iterations when the upper objective is (δ,α)-flat with small δ.
desk verdict Interesting algorithm and a genuinely new flatness condition, but the central theorem has a load-bearing proof gap: Lemma 2 silently replaces the flatness bound with the very Lipschitz-based bound the paper set out to avoid. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the (δ,α)-flatness of f(x,·) at y_g^*(x): |f(x,y_g^*(x))-f(x,y)| ≤ c||y_g^*(x)-y||^α + δ for all y, with α∈(1,1.5]. It replaces the l_{f,0}-Lipschitz bound by a subquadratic growth bound plus a small offset δ, and it does the argument's work by producing tighter estimates of how far the lower-level solution of the penalized problem sits from the true one: Lemma 1 gives ||y_g^*(x)-y_γ^*(x)|| = O($γ^{{-1/(2-α)}}$ + $δ^{{1/2}}$$γ^{{-1/2}}$) instead of the Lipschitz-based O(l_{f,0}$γ^{{-1}}$). The proof then combines these estimates with the PL-condition structure of the penalized lower objective and a Lyapunov descent argument in x that telescopes because δ(x) is claimed to be Lipschitz in x (Lemma 2); the result is the averaged gradient bound of Theorem 3.
What would settle it
Take the paper's Example 2 (or a smoothed version of it) with γ = 15, compute δ(x) via (12) on a grid of x, and measure sup_{x≠x'} |δ(x)-δ(x')|/||x-x'||; if this exceeds O(c $γ^{{-(α-1)}}$) while δ stays small, then Lemma 2 fails and Theorem 3's bound cannot be expected—averaged gradient norms over PBGD-Free runs should plateau above O($δ^{{2(α-1)/α}}$).
Extended reading notes
Core claim
PBGD-Free updates the lower variable once (K=1) by gradient descent on the penalized lower objective $γ^{{-1}}$f+g and then updates x by ∇_x f(x_t, y^γ_{t+1})—dropping the "value function" term γ(∇_x g(x, y^γ)-∇_x g(x, y^g)) that standard PBGD computes by an inner loop. The paper's central claim (Theorem 3) is that if Assumption 2 holds and f(x_t,·) is (δ(x_t), α)-flat at the lower-level solution set for all iterates, with the same α∈(1,1.5] and average δ(x_t) ≤ δ, then choosing γ = O($ε^{{-(2-α)/2}}$) yields (1/T)Σ_{t=0}^{T-1} ||∇F_γ(x_t)||^2 ≤ O($T^{{-1}}$ + $δ^{{2(α-1)/α}}$). Thus the algorithm reaches an ε-stationary neighborhood of the penalized problem in O($ε^{{-1}}$) iterations with no Hessian and no inner loop. Lemma 1 sharpens the approximation gap to ||φ-F_γ|| = O($γ^{{-α/(2-α)}}$ + δ), and Lemma 3 says the penalized stationary points are approximately stationary for the original bilevel problem, so the authors read the theorem as a positive answer to whether skipping the value-function loop is justified when the upper landscape is flat.
Load-bearing premise
The proof needs the flatness constant δ(x) to change only proportionally to how far x moves along the trajectory; if δ(x) can jump sharply when x moves a little, the descent inequality that yields Theorem 3 does not close.
Editorial extensions
If this is right
- With small δ, PBGD-Free solves a nonconvex bilevel problem at O(ε^{-1}) iterations and O(1) per-iteration cost, using only first-order gradients: no Hessian-vector products and no nested lower-level loop.
- The penalty parameter only needs to grow like ε^{-(2-α)/2}, so moderate values such as γ≈10–15 (the range used in the paper's experiments) are theoretically justified, and the tradeoff between the preference-alignment loss and the supervised fine-tuning loss observed when γ grows is a predicted side effect.
- Lemma 3 transfers stationarity from the penalized problem to the original bilevel problem, so the x found by the single-loop method is an approximate stationary point of the original formulation, not merely of the penalty.
- Under the standard Lipschitz assumption, Proposition 2 shows the same algorithm can only reach a $\Theta(l_{f,0}^2)$ neighborhood; flatness is therefore what separates convergence from stalling in the paper's examples.
Reading between the lines
- If flatness is as prevalent in alignment landscapes as the paper's fine-tuning measurements suggest, the same single-loop scheme should transfer to other bilevel fine-tuning settings, e.g., diffusion-model reward tuning or data reweighting; the practical check is to monitor δ(x_t) along the trajectory.
- The theory yields a concrete diagnostic: compute δ(x_t) from (12) during training; when it stays below the target ε, K=1 should be enough, and any accuracy gap to double-loop baselines would indicate the flatness assumption is failing rather than the loop count.
- The hinge is Lemma 2: if δ(x) is not Lipschitz in realistic landscapes, the theorem collapses to the O(l_{f,0}^2) neighborhood of Proposition 2, so an empirical measurement of the variation of δ along trajectories would settle the practical reach of the result.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes PBGD-Free, a fully single-loop first-order penalty-based method for nonconvex bilevel optimization that updates the upper-level variable x using only ∇_x f(x,y), thereby avoiding inner-loop estimation of the lower-level solution and Hessian computation. The authors first show, via Example 1 and Proposition 2, that under the standard Lipschitz assumption on f(x,·) the method converges only to a Θ(l_{f,0}^2) neighborhood of a stationary point. They then introduce a (δ,α)-flatness condition on f(x,·), argue that it is satisfied by a representation-learning LLM PEFT problem, and claim in Theorem 3 that under this condition Algorithm 1 with K=1 reaches an ε-stationary point of the penalized problem in O(ε^{-1}) iterations. The appendix contains proofs of Lemmas 1-3, a stationary-point equivalence result, and experiments on LLM PEFT, fair representation learning, and BiDoRA fine-tuning.
Significance. If the flatness-based analysis were correct, the paper would make a useful contribution: it identifies a concrete computational bottleneck in penalty-based bilevel methods, gives a clean negative result showing that PBGD-Free fails under Lipschitzness, and proposes a landscape condition tailored to representation-learning PEFT. Lemma 1's improved γ-exponent over Proposition 1 is a genuine insight, and the empirical flatness measurements in Figure 5 connect the theory to the application. However, the central convergence result is not established as written: the proof of Lemma 2 silently invokes the Lipschitz-based bound of Proposition 1, which is exactly the assumption the flatness program was meant to replace, and Theorem 3's parameter choice is inconsistent with the proof in Appendix B.6. Because the advertised O(ε^{-1}) single-loop guarantee rests on these two load-bearing points, the positive contribution is currently unsupported.
major comments (4)
- [Appendix B.4, Eqs. (48) and (51)] Lemma 2's proof is not self-consistent with the flatness setting. The bound ∥E_i∥=O(γ^{-1}) at (48) and the distance estimate ∥y_g^*(x)-y_γ^*(x)∥=O(γ^{-1}) at (51)(b) both invoke Proposition 1, which is proved under Assumption 1's Lipschitz continuity. Under the flatness hypothesis alone, Lemma 1 gives only O(γ^{-1/(2-α)}+δ^{1/2}γ^{-1/2}); for δ=Θ(1), or more generally δ>γ^{-1}, the δ^{1/2}γ^{-1/2} term is asymptotically larger than γ^{-1}. Consequently the claimed O(cγ^{-(α-1)}) Lipschitz constant for δ(x) is not established, and the later use of a Lipschitz selection y_γ^*(x) and of the smoothness of v_h in the proof of Theorem 3 is unsupported.
- [Theorem 3 and Appendix B.6, Eq. (64)] The stated choice of γ in Theorem 3 does not match the proof. The theorem says γ=O(ε^{-(2-α)/2}), while the proof sets γ=O(δ^{-(2-α)/α}) before Eq. (64) to obtain the final O(T^{-1}+δ^{2(α-1)/α}) bound. With the stated γ, the δγ term in (60) becomes δ ε^{-(2-α)/2}, which is not O(δ^{2(α-1)/α}). Thus the theorem as stated is not what is proved; the authors need to state the correct relation between ε, δ, and γ, or revise the theorem.
- [Theorem 3 and Lemma 3] The claimed O(ε^{-1}) complexity to a necessary stationary condition of the original problem requires more than the displayed bound. Theorem 3 only bounds the average squared gradient of the penalized objective Fγ by O(T^{-1}+δ^{2(α-1)/α}). To turn this into an ε-stationary guarantee one needs δ ≤ ε^{α/(2(α-1))}, and Lemma 3 separately requires δ≤O(ε^{α/2}) at the limit point. These smallness conditions on δ are not stated in Theorem 3, and the experimental validation in Figure 5 reports absolute values of δ (around 3×10^{-4}) rather than δ relative to a target accuracy ε. Without them, the result establishes convergence only to a δ-dependent neighborhood, not the advertised ε-stationary point of the original bilevel problem.
- [Appendix B.6, Eqs. (60)-(62)] The telescoping argument that closes the descent inequality relies on the smoothness constant l_{vh,1}=l_{h,1}(1+L_y^γ) of the lower-level value function v_h(x)=min_y h(x,y), which in turn requires a Lipschitz selection x↦y_γ^*(x) with constant independent of γ. The manuscript cites [11] for this fact, but [11] works under the Lipschitz assumption Assumption 1. Under flatness alone, Lemma 1's distance bound is not sufficient to produce such a selection, and Lemma 2, which could have supplied part of this regularity, is itself unproven as noted above. This is a structural gap in the proof of the main theorem.
minor comments (4)
- [Section 2.1, Assumption 1] Assumption 1 is stated as a one-sided Lipschitz condition at y_g^*(x), while Definition 2 in Appendix A defines global Lipschitz continuity; the relationship between the two should be clarified.
- [Eq. (12) and Appendix B.4, Eq. (45)] The definition of δ(x) depends on the chosen selections y_g^*(x) and y_γ^*(x); the manuscript should specify how these selections are made so that δ(x) is well-defined. Also, (45) defines δ'(x) without the max with 0, and the sentence 'δ(x) is a ReLU function' is too terse.
- [References] The reference list contains duplicated numbering, e.g., [27] and [69] appear twice; those entries should be cleaned up.
- [Table 1 and Section 5] The claimed O(ε^{-1}) complexity should be qualified in the table and concluding remarks as convergence to an ε-stationary point of the penalized objective under an additional δ-smallness condition, since Theorem 3 as written only gives a δ-dependent neighborhood.
Circularity Check
Lemma 2, the only regularity bridge from per-iterate flatness to the uniform analysis in Theorem 3, is proved using Proposition 1's O(gamma^{-1}) distance bound, which requires the Lipschitz upper-level assumption that flatness was introduced to replace.
-
other
[Appendix B.4, proof of Lemma 2, Eqs. (47)-(51), and main text before Theorem 3]
"By the smoothness of f, the Lipschitzness of ∇2g and by Proposition 1, we know that ∥E1∥,∥E2∥,∥E3∥,∥E4∥ =O(γ−1). ... (b) is from ∥y∗g(x)−y∗γ(x)∥ =O(γ−1), and the 1-Lipschitzness of the norm function"
Lemma 2 is the step that turns the per-iterate flatness bound of Lemma 1 into the uniform Lipschitz regularity of δ(x) needed by Theorem 3. Its proof bounds E1–E4 by O(γ−1) and uses ∥y∗g(x)−y∗γ(x)∥ = O(γ−1), which are exactly the estimates of Proposition 1 under Assumption 1 (l_{f,0}-Lipschitz upper-level). Lemma 1, the flatness-based result, gives only d = O(γ^{−1/(2−α)} + δ^{1/2}γ^{−1/2}). For fixed δ = Θ(1) this is O(γ^{−1/2}) for large γ, not O(γ^{−1}); hence the Lipschitz constant O(cγ^{−(α−1)}) is not established under flatness. Since Theorem 3's uniform telescoping in Appendix B.6, Eqs. (61)-(62), and the smoothness of v_h via L^γ_y rely on this Lipschitz regularity, the O(ε^{−1}) flatness claim imports the very Lipschitz regime it was designed to avoid.
full rationale
I flag one structural circular step. The empirical flatness check in Figure 5 is not circular: the constants c = 0.5 and α = 1.5 are stated, δ(x) is computed from its definition, and no fitted parameter is relabeled as a prediction. Proposition 2's counterexample and lower bound are also self-contained. The circularity is confined to the theoretical bridge: Lemma 2 is advertised as establishing Lipschitz continuity of the flatness constant δ(x) under flatness, but its proof invokes Proposition 1's O(γ^{-1}) distance estimate, which holds only under the l_{f,0}-Lipschitz assumption that flatness was meant to replace. Lemma 1's flatness estimate contains a δ^{1/2}γ^{-1/2} term; with δ not forced small this is much larger than γ^{-1}, so the claimed Lipschitz constant O(cγ^{-(α-1)}) and the telescoping in Appendix B.6 are unsupported. The v_h-smoothness step in Eq. (62) similarly relies on a Lipschitz selection y_γ^*(x) imported from prior work and not re-derived under flatness. Thus the central O(ε^{-1}) complexity is not fully independent of the old Lipschitz-based analysis: at a key lemma it reduces to the very input the flatness condition was intended to relax.
Assumptions & free parameters
free parameters (4)
- penalty constant gamma =
10 in LLM experiments; O(epsilon^-(2-alpha)/2) in theory
- flatness exponent alpha =
1.5 in Figure 5
- flatness modulus c =
0.5 in Figure 5
- flatness offset delta =
reported below 3e-4 in the toy flatness check; assumed small in the theorem
assumptions (3)
- domain assumption For some gamma* > 0, c f(x,y) + g(x,y) is mu-PL in y for all c in [0, 1/gamma*], and f and g have Lipschitz gradients and Hessians (Assumption 2).
- ad hoc to paper Lipschitz continuity of f(x, .) at y_g^*(x), i.e. Assumption 1, is silently re-used in Lemma 2's proof through Proposition 1.
- domain assumption There exist Lipschitz selections y_g^*(x) and y_gamma^*(x) of the solution mappings used in Lemma 2.
Cite this review
Pith. "Pith review of Beyond Value Functions: Single-Loop Bilevel Optimization under Flatness Conditions." pith.science (2026). https://pith.science/paper/XF2EDCLS
@misc{pith2026250720400,
author = {Pith},
title = {Pith review of: Beyond Value Functions: Single-Loop Bilevel Optimization under Flatness Conditions},
year = {2026},
howpublished = {\url{https://pith.science/paper/XF2EDCLS}},
note = {Machine review of arXiv:2507.20400}
}
read the original abstract
Bilevel optimization, a hierarchical optimization paradigm, has gained significant attention in a wide range of practical applications, notably in the fine-tuning of generative models. However, due to the nested problem structure, most existing algorithms require either the Hessian vector calculation or the nested loop updates, which are computationally inefficient in large language model (LLM) fine-tuning. In this paper, building upon the fully first-order penalty-based approach, we propose an efficient value function-free (PBGD-Free) algorithm that eliminates the loop of solving the lower-level problem and admits fully single-loop updates. Inspired by the landscape analysis of representation learning-based LLM fine-tuning problem, we propose a relaxed flatness condition for the upper-level function and prove the convergence of the proposed value-function-free algorithm. We test the performance of the proposed algorithm in various applications and demonstrate its superior computational efficiency over the state-of-the-art bilevel methods.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[11]
On finding small hyper-gradients in bilevel optimization: Hardness results and improved analysis
Lesi Chen, Jing Xu, and Jingzhao Zhang. On finding small hyper-gradients in bilevel optimization: Hardness results and improved analysis. In The Thirty Seventh Annual Conference on Learning Theory, pages 947–980. PMLR, 2024
2024
-
[1]
Sharp-maml: Sharpness-aware model-agnostic meta learning
Momin Abbas, Quan Xiao, Lisha Chen, Pin-Yu Chen, and Tianyi Chen. Sharp-maml: Sharpness-aware model-agnostic meta learning. In International conference on machine learning, pages 10–32, 2022
2022
-
[2]
Muppet: Massive multi-task representations with pre-finetuning
Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, and Sonal Gupta. Muppet: Massive multi-task representations with pre-finetuning. In Proc. of the Conference on Empirical Methods in Natural Language Processing, 2021
2021
-
[3]
Non-convex bilevel games with critical point selection maps
Michael Arbel and Julien Mairal. Non-convex bilevel games with critical point selection maps. In Proc. Advances in Neural Information Processing Systems, New Orleans, LA, 2022
2022
-
[4]
Provable representation learning for imitation learning via bi-level optimization
Sanjeev Arora, Simon Du, Sham Kakade, Yuping Luo, and Nikunj Saunshi. Provable representation learning for imitation learning via bi-level optimization. InInternational Conference on Machine Learning, pages 367–376. PMLR, 2020
2020
-
[5]
On the quality of first-order approximation of functions with h ¨older continuous gradient
Guillaume O Berger, P-A Absil, Rapha¨el M Jungers, and Yurii Nesterov. On the quality of first-order approximation of functions with h ¨older continuous gradient. Journal of Optimization Theory and Applications, 185:17–33, 2020
2020
-
[6]
Pythia: A suite for analyzing large language models across training and scaling
Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, 2023
2023
-
[7]
Automatic differentiation of nonsmooth iterative algorithms
J´erˆome Bolte, Edouard Pauwels, and Samuel Vaiter. Automatic differentiation of nonsmooth iterative algorithms. In Advances in Neural Information Processing Systems, 2022
2022
Show all 106 references
-
[8]
Coresets via bilevel optimization for continual learning and streaming
Zal´an Borsos, Mojmir Mutny, and Andreas Krause. Coresets via bilevel optimization for continual learning and streaming. In Advances in Neural Information Processing Systems, virtual, 2020
2020
-
[9]
Convex Optimization: Algorithms and Complexity
S´ebastien Bubeck. Convex Optimization: Algorithms and Complexity . Foundations and Trends ® in Machine Learning, 2015
2015
-
[10]
On bilevel optimization without lower-level strong convexity
Lesi Chen, Jing Xu, and Jingzhao Zhang. On bilevel optimization without lower-level strong convexity. arXiv preprint arXiv:2301.00712, 2023
2023 arXiv
-
[12]
Learning with limited samples – meta-learning and applications to communication systems
Lisha Chen, Sharu Theresa Jose, Ivana Nikoloska, Sangwoo Park, Tianyi Chen, and Osvaldo Simeone. Learning with limited samples – meta-learning and applications to communication systems. Foundations and Trends in Signal Processing, 1 2023
2023
-
[13]
Efficient first-order optimization on the pareto set for multi-objective learning under preference guidance
Lisha Chen, Quan Xiao, Ellen Hidemi Fukuda, Xinyi Chen, Kun Yuan, and Tianyi Chen. Efficient first-order optimization on the pareto set for multi-objective learning under preference guidance. arXiv preprint arXiv:2504.02854, 2025
2025 arXiv
-
[14]
A single-timescale method for stochastic bilevel optimization
Tianyi Chen, Yuejiao Sun, Quan Xiao, and Wotao Yin. A single-timescale method for stochastic bilevel optimization. In Proc. International Conference on Artificial Intelligence and Statistics, 2022
2022
-
[15]
Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems
Tianyi Chen, Yuejiao Sun, and Wotao Yin. Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems. In Advances in Neural Information Processing Systems, Virtual, 2021
2021
-
[16]
A fast and convergent proximal algorithm for regularized nonconvex and nonsmooth bi-level optimization
Ziyi Chen, Bhavya Kailkhura, and Yi Zhou. A fast and convergent proximal algorithm for regularized nonconvex and nonsmooth bi-level optimization. arXiv preprint arXiv:2203.16615, 2022
2022 arXiv
-
[17]
Directly fine-tuning diffusion models on differentiable rewards
Kevin Clark, Paul Vicol, Kevin Swersky, and David J Fleet. Directly fine-tuning diffusion models on differentiable rewards. In International Conference on Learning Representations, 2024
2024
-
[18]
Optimization and nonsmooth analysis
Frank H Clarke. Optimization and nonsmooth analysis. SIAM, 1990
1990
-
[19]
A framework for bilevel optimization that enables stochastic and global variance reduction algorithms
Mathieu Dagr´eou, Pierre Ablin, Samuel Vaiter, and Thomas Moreau. A framework for bilevel optimization that enables stochastic and global variance reduction algorithms. In Advances in Neural Information Processing Systems, 2022
2022
-
[20]
Automatically constructing a corpus of sentential paraphrases
Bill Dolan and Chris Brockett. Automatically constructing a corpus of sentential paraphrases. In Third international workshop on paraphrasing (IWP2005), 2005
2005
-
[21]
Efficient curvature-aware hypergradient approxi- mation for bilevel optimization
Youran Dong, Junfeng Yang, Wei Yao, and Jin Zhang. Efficient curvature-aware hypergradient approxi- mation for bilevel optimization. arXiv preprint arXiv:2505.02101, 2025. 11
2025 arXiv
-
[22]
Llama-omni: Seamless speech interaction with large language models
Qingkai Fang, Shoutao Guo, Yan Zhou, Zhengrui Ma, Shaolei Zhang, and Yang Feng. Llama-omni: Seamless speech interaction with large language models. arXiv preprint arXiv:2409.06666, 2024
2024 arXiv
-
[23]
qnbo: quasi-newton meets bilevel optimization
Sheng Fang, Yong-Jin Liu, Wei Yao, Chengming Yu, and Jin Zhang. qnbo: quasi-newton meets bilevel optimization. In Proc. International Conference on Learning Representations, 2025
2025
-
[24]
Mitigating forgetting in llm supervised fine-tuning and preference learning
Heshan Fernando, Han Shen, Parikshit Ram, Yi Zhou, Horst Samulowitz, Nathalie Baracaldo, and Tianyi Chen. Mitigating forgetting in llm supervised fine-tuning and preference learning. arXiv preprint arXiv:2410.15483, 2024
2024
-
[25]
Sharpness-aware minimization for efficiently improving generalization
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. In Proc. International Conference on Learning Representations, virtual, 2021
2021
-
[26]
Forward and reverse gradient- based hyperparameter optimization
Luca Franceschi, Michele Donini, Paolo Frasconi, and Massimiliano Pontil. Forward and reverse gradient- based hyperparameter optimization. In International Conference on Machine Learning, pages 1165–1173. PMLR, 2017
2017
-
[27]
Bilevel programming for hyperparameter optimization and meta-learning
Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. InInternational conference on machine learning, pages 1568–1577. PMLR, 2018
2018
-
[28]
Approximation methods for bilevel programming
Saeed Ghadimi and Mengdi Wang. Approximation methods for bilevel programming. arXiv preprint arXiv:1802.02246, 2018
2018 arXiv
-
[29]
An investigation into neural net optimization via hessian eigenvalue density
Behrooz Ghorbani, Shankar Krishnan, and Ying Xiao. An investigation into neural net optimization via hessian eigenvalue density. In Proc. International Conference on Machine Learning, pages 2232–2241, 2019
2019
-
[30]
The llama 3 herd of models
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024
2024 arXiv
-
[31]
On the iteration complexity of hypergradient computation
Riccardo Grazzi, Luca Franceschi, Massimiliano Pontil, and Saverio Salzo. On the iteration complexity of hypergradient computation. In International Conference on Machine Learning, pages 3748–3758, virtual, 2020
2020
-
[32]
Bilevel coreset selection in continual learning: A new formulation and algorithm
Jie Hao, Kaiyi Ji, and Mingrui Liu. Bilevel coreset selection in continual learning: A new formulation and algorithm. In Advances in Neural Information Processing Systems, New Orleans, LA, 2023
2023
-
[33]
A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and application to actor-critic
Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and application to actor-critic. SIAM Journal on Optimization, 33(1):147–180, 2023
2023
-
[34]
Lora: Low-rank adaptation of large language models
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022
2022
-
[35]
Contextual stochastic bilevel optimiza- tion
Yifan Hu, Jie Wang, Yao Xie, Andreas Krause, and Daniel Kuhn. Contextual stochastic bilevel optimiza- tion. In Proc. Advances in Neural Information Processing Systems, 2023
2023
-
[36]
Will bilevel optimizers benefit from loops
Kaiyi Ji, Mingrui Liu, Yingbin Liang, and Lei Ying. Will bilevel optimizers benefit from loops. arXiv preprint arXiv:2205.14224, 2022
2022 arXiv
-
[37]
Bilevel optimization: Convergence analysis and enhanced design
Kaiyi Ji, Junjie Yang, and Yingbin Liang. Bilevel optimization: Convergence analysis and enhanced design. In International conference on machine learning, pages 4882–4892. PMLR, 2021
2021
-
[38]
Provably faster algorithms for bilevel optimization and applications to meta-learning
Kaiyi Ji, Junjie Yang, and Yingbin Liang. Provably faster algorithms for bilevel optimization and applications to meta-learning. In Advances in Neural Information Processing Systems, 2021
2021
-
[39]
A primal-dual-assisted penalty approach to bilevel optimization with coupled constraints
Liuyuan Jiang, Quan Xiao, Victor M Tenorio, Fernando Real-Rojas, Antonio Marques, and Tianyi Chen. A primal-dual-assisted penalty approach to bilevel optimization with coupled constraints. In Advances in Neural Information Processing Systems, 2024
2024
-
[40]
Linear convergence of gradient and proximal-gradient methods under the polyak-łojasiewicz condition
Hamed Karimi, Julie Nutini, and Mark Schmidt. Linear convergence of gradient and proximal-gradient methods under the polyak-łojasiewicz condition. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2016, Riva del Garda, Italy, September 19...
2016
-
[41]
A near-optimal algorithm for stochastic bilevel optimization via double-momentum
Prashant Khanduri, Siliang Zeng, Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. A near-optimal algorithm for stochastic bilevel optimization via double-momentum. In Advances in Neural Information Processing Systems, Virtual, 2021
2021
-
[42]
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint:1412.6980, December 2014. 12
2014 arXiv
-
[43]
A fully first-order method for stochastic bilevel optimization
Jeongyeol Kwon, Dohyun Kwon, Stephen Wright, and Robert D Nowak. A fully first-order method for stochastic bilevel optimization. In International Conference on Machine Learning, pages 18083–18113, 2023
2023
-
[44]
On penalty methods for nonconvex bilevel optimization and first-order stochastic approximation
Jeongyeol Kwon, Dohyun Kwon, Steve Wright, and Robert Nowak. On penalty methods for nonconvex bilevel optimization and first-order stochastic approximation. In International Conference on Learning Representations, 2024
2024
-
[45]
Convergence of adam under relaxed assumptions
Haochuan Li, Alexander Rakhlin, and Ali Jadbabaie. Convergence of adam under relaxed assumptions. In Proc. Advances in Neural Information Processing Systems, New Orleans, LA, 2023
2023
-
[46]
A fully single loop algorithm for bilevel optimization without hessian inverse
Junyi Li, Bin Gu, and Heng Huang. A fully single loop algorithm for bilevel optimization without hessian inverse. In Association for the Advancement of Artificial Intelligence, pages 7426–7434, virtual, 2022
2022
-
[47]
Darts: Differentiable architecture search
Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055, 2018
2018 arXiv
-
[48]
Investigating bilevel optimization for learning and vision from a unified perspective: A survey and beyond
Risheng Liu, Jiaxin Gao, Jin Zhang, Deyu Meng, and Zhouchen Lin. Investigating bilevel optimization for learning and vision from a unified perspective: A survey and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):10045–10067, 2021
2021
-
[49]
A value-function-based interior- point method for non-convex bi-level optimization
Risheng Liu, Xuan Liu, Xiaoming Yuan, Shangzhi Zeng, and Jin Zhang. A value-function-based interior- point method for non-convex bi-level optimization. In International conference on machine learning, pages 6882–6892, 2021
2021
-
[50]
Value-function-based sequential minimization for bi-level optimization
Risheng Liu, Xuan Liu, Shangzhi Zeng, Jin Zhang, and Yixuan Zhang. Value-function-based sequential minimization for bi-level optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
2023
-
[51]
Averaged method of multipliers for bi-level optimization without lower-level strong convexity
Risheng Liu, Yaohua Liu, Wei Yao, Shangzhi Zeng, and Jin Zhang. Averaged method of multipliers for bi-level optimization without lower-level strong convexity. InProc. International Conference on Machine Learning, Honolulu, HI, 2023
2023
-
[52]
Dora: Weight-decomposed low-rank adaptation
Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang- Ting Cheng, and Min-Hung Chen. Dora: Weight-decomposed low-rank adaptation. arXiv preprint arXiv:2402.09353, 2024
2024 arXiv
-
[53]
On the optimization landscape of low rank adaptation methods for large language models
Xu-Hui Liu, Yali Du, Jun Wang, and Yang Yu. On the optimization landscape of low rank adaptation methods for large language models. In Proc. International Conference on Learning Representations, 2025
2025
-
[54]
The flan collection: Designing data and methods for effective instruction tuning
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. The flan collection: Designing data and methods for effective instruction tuning. In Proc. International Conference on Machine Learning, 2023
2023
-
[55]
Boundary matters: A bi-level active finetuning method
Han Lu, Yichen Xie, Xiaokang Yang, and Junchi Yan. Boundary matters: A bi-level active finetuning method. In Advances in Neural Information Processing Systems, 2024
2024
-
[56]
Learning word vectors for sentiment analysis
Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. Learning word vectors for sentiment analysis. InProceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 142–150, 2011
2011
-
[57]
Gradient-based hyperparameter optimization through reversible learning
Dougal Maclaurin, David Duvenaud, and Ryan Adams. Gradient-based hyperparameter optimization through reversible learning. In International Conference on Machine Learning, pages 2113–2122, Lille, France, 2015
2015
-
[58]
Implicit diffusion: Efficient optimization through stochastic sampling
Pierre Marion, Anna Korba, Peter Bartlett, Mathieu Blondel, Valentin De Bortoli, Arnaud Doucet, Felipe Llinares-L´opez, Courtney Paquette, and Quentin Berthet. Implicit diffusion: Efficient optimization through stochastic sampling. In International Conference on Artificial Int...
2025
-
[59]
Introductory lectures on convex optimization: A basic course , volume 87
Yurii Nesterov. Introductory lectures on convex optimization: A basic course , volume 87. Springer Science & Business Media, 2013
2013
-
[60]
On first-order meta-learning algorithms
Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999, 2018
2018 arXiv
-
[61]
Scalebio: Scalable bilevel optimization for llm data reweighting
Rui Pan, Jipeng Zhang, Xingyuan Pan, Renjie Pi, Xiaoyu Wang, and Tong Zhang. Scalebio: Scalable bilevel optimization for llm data reweighting. arXiv preprint arXiv:2406.19976, 2024
2024 arXiv
-
[62]
Bleu: a method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002
2002
-
[63]
Functional bilevel optimization for machine learning
Ieva Petrulionyt˙e, Julien Mairal, and Michael Arbel. Functional bilevel optimization for machine learning. In Proc. Advances in Neural Information Processing Systems, Vancouver, Canada, 2024. 13
2024
-
[64]
Bidora: Bi-level optimization-based weight-decomposed low-rank adaptation
Peijia Qin, Ruiyi Zhang, and Pengtao Xie. Bidora: Bi-level optimization-based weight-decomposed low-rank adaptation. arXiv preprint arXiv:2410.09758, 2024
2024 arXiv
-
[65]
Direct preference optimization: Your language model is secretly a reward model
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023
2023
-
[66]
Shine: Sharing the inverse estimate from the forward pass for bi-level optimization and implicit models
Zaccharie Ramzi, Florian Mannel, Shaojie Bai, Jean-Luc Starck, Philippe Ciuciu, and Thomas Moreau. Shine: Sharing the inverse estimate from the forward pass for bi-level optimization and implicit models. In Proc. International Conference on Learning Representations, 2022
2022
-
[67]
How to prepare your task head for finetuning
Yi Ren, Shangmin Guo, Wonho Bae, and Danica J Sutherland. How to prepare your task head for finetuning. In International Conference on Learning Representations, 2023
2023
-
[68]
Cohort profile: The national longitudinal survey of youth 1979 (nlsy79)
Donna S Rothstein, Deborah Carr, and Elizabeth Cooksey. Cohort profile: The national longitudinal survey of youth 1979 (nlsy79). International journal of epidemiology, 48(1):22–22e, 2019
1979
-
[69]
Equilibrium propagation: Bridging the gap between energy-based models and backpropagation
Benjamin Scellier and Yoshua Bengio. Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in computational neuroscience, 11:24, 2017
2017
-
[70]
Truncated back-propagation for bilevel optimization
Amirreza Shaban, Ching-An Cheng, Nathan Hatch, and Byron Boots. Truncated back-propagation for bilevel optimization. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1723–1732. PMLR, 2019
2019
-
[71]
Seal: Safety-enhanced aligned llm fine-tuning via bilevel data selection
Han Shen, Pin-Yu Chen, Payel Das, and Tianyi Chen. Seal: Safety-enhanced aligned llm fine-tuning via bilevel data selection. In International Conference on Learning Representations, 2025
2025
-
[72]
A single-timescale analysis for stochastic approximation with multiple coupled sequences
Han Shen and Tianyi Chen. A single-timescale analysis for stochastic approximation with multiple coupled sequences. Advances in Neural Information Processing Systems, 35:17415–17429, 2022
2022
-
[73]
On penalty-based bilevel gradient descent method
Han Shen, Quan Xiao, and Tianyi Chen. On penalty-based bilevel gradient descent method. In Interna- tional Conference on Machine Learning, Honolulu, HI, 2023
2023
-
[74]
Principled penalty-based methods for bilevel reinforcement learning and RLHF
Han Shen, Zhuoran Yang, and Tianyi Chen. Principled penalty-based methods for bilevel reinforcement learning and RLHF. In International Conference on Machine Learning, Vienna, Austria, 2024
2024
-
[75]
Fair representation learning through implicit path alignment
Changjian Shui, Qi Chen, Jiaqi Li, Boyu Wang, and Christian Gagn´e. Fair representation learning through implicit path alignment. In International Conference on Machine Learning, pages 20156–20175. PMLR, 2022
2022
-
[76]
A constrained optimization approach to bilevel optimization with multiple inner minima
Daouda Sow, Kaiyi Ji, Ziwei Guan, and Yingbin Liang. A constrained optimization approach to bilevel optimization with multiple inner minima. arXiv preprint arXiv:2203.01123, 2022
2022 arXiv
-
[77]
On the convergence theory for hessian-free bilevel algorithms
Daouda Sow, Kaiyi Ji, and Yingbin Liang. On the convergence theory for hessian-free bilevel algorithms. In Advances in Neural Information Processing Systems, volume 35, pages 4136–4149, 2022
2022
-
[78]
Retrospectives: Who invented instrumental variable regression? Journal of Economic Perspectives, 17(3):177–194, 2003
James H Stock and Francesco Trebbi. Retrospectives: Who invented instrumental variable regression? Journal of Economic Perspectives, 17(3):177–194, 2003
2003
-
[79]
Bi-level optimization of charging scheduling of a battery swap station based on deep reinforcement learning
Mao Tan, Zhuocen Dai, Yongxin Su, Caixue Chen, Ling Wang, and Jie Chen. Bi-level optimization of charging scheduling of a battery swap station based on deep reinforcement learning. Engineering Applications of Artificial Intelligence, 118:105557, 2023
2023
-
[80]
Contextual bilevel reinforcement learning for incentive alignment
Vinzenz Thoma, Barna P´asztor, Andreas Krause, Giorgia Ramponi, and Yifan Hu. Contextual bilevel reinforcement learning for incentive alignment. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024
2024
-
[81]
On implicit bias in overparameterized bilevel optimization
Paul Vicol, Jonathan Lorraine, Fabian Pedregosa, David Duvenaud, and Roger Grosse. On implicit bias in overparameterized bilevel optimization. In Proc. International Conference on Machine Learning, 2022
2022
-
[82]
Fully first-order methods for decentralized bilevel optimization
Xiaoyu Wang, Xuxing Chen, Shiqian Ma, and Tong Zhang. Fully first-order methods for decentralized bilevel optimization. arXiv preprint arXiv:2410.19319, 2024
2024
-
[83]
Convergence of alternating gradient descent for matrix factorization
Rachel Ward and Tamara Kolda. Convergence of alternating gradient descent for matrix factorization. In Proc. Advances in Neural Information Processing Systems, New Orleans, LA, 2023
2023
-
[84]
Perturbation theory for pseudo-inverses
Per- ˚Ake Wedin. Perturbation theory for pseudo-inverses. BIT Numerical Mathematics, 13:217–232, 1973
1973
-
[85]
Unlocking global optimality in bilevel optimization: A pilot study
Quan Xiao and Tianyi Chen. Unlocking global optimality in bilevel optimization: A pilot study. In Proc. International Conference on Learning Representations, 2025
2025
-
[86]
A generalized alternating method for bilevel optimization under the polyak-łojasiewicz condition
Quan Xiao, Songtao Lu, and Tianyi Chen. A generalized alternating method for bilevel optimization under the polyak-łojasiewicz condition. In Advances in Neural Information Processing Systems, New Orleans, LA, 2023. 14
2023
-
[87]
Alternating implicit projected sgd and its effi- cient variants for equality-constrained bilevel optimization
Quan Xiao, Han Shen, Wotao Yin, and Tianyi Chen. Alternating implicit projected sgd and its effi- cient variants for equality-constrained bilevel optimization. In International Conference on Artificial Intelligence and Statistics, 2023
2023
-
[88]
A first- order generative bilevel optimization framework for diffusion models
Quan Xiao, Hui Yuan, AFM Saif, Gaowen Liu, Ramana Kompella, Mengdi Wang, and Tianyi Chen. A first- order generative bilevel optimization framework for diffusion models. arXiv preprint arXiv:2502.08808, 2025
2025 arXiv
-
[89]
All you need is beyond a good init: Exploring better solution for training extremely deep convolutional neural networks with orthonormality and modulation
Di Xie, Jiang Xiong, and Shiliang Pu. All you need is beyond a good init: Exploring better solution for training extremely deep convolutional neural networks with orthonormality and modulation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p...
2017
-
[90]
Deep proxy causal learning and its application to confounded bandit policy evaluation
Liyuan Xu, Heishiro Kanagawa, and Arthur Gretton. Deep proxy causal learning and its application to confounded bandit policy evaluation. Advances in Neural Information Processing Systems, 34:26264– 26275, 2021
2021
-
[91]
Pruning parameterization with bi-level optimization for efficient semantic segmentation on the edge
Changdi Yang, Pu Zhao, Yanyu Li, Wei Niu, Jiexiong Guan, Hao Tang, Minghai Qin, Bin Ren, Xue Lin, and Yanzhi Wang. Pruning parameterization with bi-level optimization for efficient semantic segmentation on the edge. In Proc. of the IEEE/CVF Conference on Computer Vision and Pa...
2023
-
[92]
Provably faster algorithms for bilevel optimization
Junjie Yang, Kaiyi Ji, and Yingbin Liang. Provably faster algorithms for bilevel optimization. arXiv preprint arXiv:2106.04692, June 2021
2021 arXiv
-
[93]
First-order federated bilevel learning
Yifan Yang, Peiyao Xiao, Shiqian Ma, and Kaiyi Ji. First-order federated bilevel learning. In Proc. of the AAAI Conference on Artificial Intelligence, volume 39, pages 22029–22037, 2025
2025
-
[94]
Constrained bi-level optimization: Proximal lagrangian value function approach and hessian-free algorithm
Wei Yao, Chengming Yu, Shangzhi Zeng, and Jin Zhang. Constrained bi-level optimization: Proximal lagrangian value function approach and hessian-free algorithm. arXiv preprint arXiv:2401.16164, 2024
2024 arXiv
-
[95]
Compressible dynamics in deep overparameterized low-rank learning & adaptation
Can Yaras, Peng Wang, Laura Balzano, and Qing Qu. Compressible dynamics in deep overparameterized low-rank learning & adaptation. In Proc. International Conference on Machine Learning, Vienna, Austria, 2024
2024
-
[96]
Jane J. Ye. Constraint qualifications and necessary optimality conditions for optimization problems with variational inequality constraints. SIAM Journal on Optimization, 10(4):943–962, 2000
2000
-
[97]
Difference of convex algorithms for bilevel programs with applications in hyperparameter selection
Jane J Ye, Xiaoming Yuan, Shangzhi Zeng, and Jin Zhang. Difference of convex algorithms for bilevel programs with applications in hyperparameter selection. Mathematical Programming, 198(2):1583–1616, 2023
2023
-
[98]
Exact penalization and necessary optimality conditions for generalized bilevel programming problems
Jane J Ye, Daoli Zhu, and Qiji Jim Zhu. Exact penalization and necessary optimality conditions for generalized bilevel programming problems. SIAM Journal on optimization, 7(2):481–507, 1997
1997
-
[99]
Bome! bilevel optimization made easy: A simple first-order approach
Mao Ye, Bo Liu, Stephen Wright, Peter Stone, and Qiang Liu. Bome! bilevel optimization made easy: A simple first-order approach. In Proc. Advances in Neural Information Processing Systems, New Orleans, LA, 2022
2022
-
[100]
Bi- level actor-critic for multi-agent coordination
Haifeng Zhang, Weizhe Chen, Zeren Huang, Minne Li, Yaodong Yang, Weinan Zhang, and Jun Wang. Bi- level actor-critic for multi-agent coordination. In Proc. of the AAAI Conference on Artificial Intelligence, 2020
2020
-
[101]
Why gradient clipping accelerates training: A theoretical justification for adaptivity
Jingzhao Zhang, Tianxing He, Suvrit Sra, and Ali Jadbabaie. Why gradient clipping accelerates training: A theoretical justification for adaptivity. In Proc. International Conference on Learning Representations, virtual, 2020
2020
-
[102]
Advancing model pruning via bi-level optimization
Yihua Zhang, Yuguang Yao, Parikshit Ram, Pu Zhao, Tianlong Chen, Mingyi Hong, Yanzhi Wang, and Sijia Liu. Advancing model pruning via bi-level optimization. In Advances in Neural Information Processing Systems, 2022
2022
-
[103]
Why transformers need adam: A hessian perspective
Yushun Zhang, Congliang Chen, Tian Ding, Ziniu Li, Ruoyu Sun, and Zhiquan Luo. Why transformers need adam: A hessian perspective. In Proc. Advances in Neural Information Processing Systems , Vancouver, Canada, 2024
2024
-
[104]
Beyond Value Functions: Single-Loop Bilevel Optimization under Flatness Conditions
Nicolas Zucchet and Jo˜ao Sacramento. Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation. Neural Computation, 34(12):2309–2346, 2022. 15 Supplementary Material for “Beyond Value Functions: Single-Loop Bilevel Optimization ...
2022
-
[105]
This shows that the preference backbonex learned by both of them can be adapted to new task by fine-tuning only the linear head to achieve strong SFT performance
demonstrate the ability to preserve strong preference alignment (DPO) while conducting SFT training in S2). This shows that the preference backbonex learned by both of them can be adapted to new task by fine-tuning only the linear head to achieve strong SFT performance. Notabl...
2000
-
[106]
We conduct experiments on Microsoft Research Paraphrase Corpus (MRPC) dataset [20], and Internet Movie Database (IMDb) in Hugging Face by fine-tuning Bert model [56]
proposed BiDoRa, which considers fine-tuning using DoRa [52] by training on a BLO problem min m ll tr(m,v∗(m)) s.t v∗(m) =ls tr(m,v∗(m) +ρR(v) (78) where m is the magnitude and v is the direction matrix for the low-rank incremental direction, ll tr(m,v ) andll tr(m,v ) are res...
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.