Pith. sign in

REVIEW 4 major objections 4 minor 49 references

This paper introduces D-NSVRGDA, a decentralized bilevel optimizer that normalizes gradients instead of clipping them, and proves it drives the original bilevel objective's gradient to zero under heavy-tailed noise at a rate set by the nois

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 16:12 UTC pith:RFMF3DGL

load-bearing objection A plausible first rate for decentralized nonconvex bilevel optimization under heavy-tailed noise, but the key PL assumption on the penalized objective is unproved and the proofs are missing from the current text. the 4 major comments →

arxiv 2509.15543 v2 pith:RFMF3DGL submitted 2025-09-19 cs.LG

Nonconvex Decentralized Stochastic Bilevel Optimization under Heavy-Tailed Noise

Xinwen Zhang , Yihan Zhang , Heng Liang , Hongchang Gao This is my paper
classification cs.LG MSC 90C2690C1568W15
keywords decentralized bilevel optimizationheavy-tailed noisegradient normalizationvariance reductionPolyak-Lojasiewicz conditiongradient trackingstochastic optimizationminimax reformulation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Traditional decentralized bilevel optimization assumes the lower-level problem is strongly convex and that gradient noise has finite variance. Real models, especially language models, violate both. This paper develops D-NSVRGDA, a decentralized algorithm that normalizes variance-reduced gradients instead of clipping them, and proves that on a network of workers it drives the gradient norm of the original bilevel objective to zero even when noise only has a finite s-th moment for s in (1,2]. The guaranteed rate is O(1/T^{(s-1)/(2s+1)}) up to constants depending on the tail, the condition number, and the network, and the paper claims this is the first such guarantee for nonconvex decentralized bilevel problems. If correct, it means heavy-tailed noise can be handled by normalization alone, without tuning a clipping threshold.

Core claim

The central result is Theorem 4.5. Under Lipschitz smoothness of the stochastic gradients, a PL condition on both the lower-level objective and the penalized objective h_delta(x,y)=delta f(x,y)+g(x,y), heavy-tailed noise with finite s-th moment, and a standard mixing condition on the gossip matrix, D-NSVRGDA with the paper's step sizes, momentum, penalty parameter, and initial batch size satisfies (1/T) sum_{t=0}^{T-1} E[||grad Phi(x_bar_t)||] <= O(kappa^4 ell sigma^{(2s-2)/(2s+1)} K^{(s-1)/(2s+1)} T^{-(s-1)/(2s+1)}) plus lower-order terms. In words: the expected norm of the gradient of the original upper-level objective Phi goes to zero at a rate governed by the heaviness of the noise; heav

What carries the argument

The algorithm converts the bilevel problem into a single-level minimax problem via the penalty function h_delta(x,y)=delta f(x,y)+g(x,y), so only first-order gradients are needed. It then combines three ingredients: (i) momentum-based variance-reduced estimators for the gradients of f, g(x,y), and g(x,z); (ii) gradient tracking to keep local gradient estimates close to the network average; and (iii) normalized updates, p_t/||p_t||, which replace clipping. The analysis's load-bearing object is the PL condition on h_delta in y, which lets Lemma 4.2 turn inner-problem gradient norms into objective-gap terms; Lemmas 4.3 and 4.4 bound the variance-reduction errors and consensus errors under only

Load-bearing premise

The load-bearing premise is Assumption 3.1.3, that the penalized objective h_delta(x,y)=delta f(x,y)+g(x,y) satisfies the Polyak-Lojasiewicz condition in y; the paper assumes it outright, and without it the inner minimization cannot be certified and the convergence rate collapses.

What would settle it

On a network with controlled heavy-tailed noise of known exponent s, run D-NSVRGDA and plot the average gradient norm of Phi against iterations on a log-log scale. If the observed slope is systematically slower than -(s-1)/(2s+1), the theorem's rate is wrong; if the rate holds on an instance where h_delta is not PL, then Assumption 3.1.3 is not necessary.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Users of decentralized bilevel methods can replace clipping-threshold tuning with normalization; the paper proves the normalized update converges in expectation.
  • The lower-level loss only needs a PL condition, not strong convexity, expanding applicability to overparameterized networks.
  • Adding workers gives linear speedup in the dominant term: K^{(s-1)/(2s+1)} appears as a favorable factor in the rate.
  • When the noise has finite variance (s=2) and there is one worker, the bound collapses to the known O(T^{-1/5}) single-machine penalty-method rate, so the result is a strict generalization.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The heaviest hidden assumption is unverified: Assumption 3.1.3 requires h_delta to satisfy PL in y, and the paper neither derives it from g's PL nor checks it on the two-layer networks used in experiments. A user applying D-NSVRGDA should verify this on their own model; otherwise the stated rate has no foundation.
  • Because the dominant rate degrades as s approaches 1, the result predicts that normalization mitigates but does not eliminate heavy tails: datasets with thinner tails should converge faster in practice. This is testable by measuring the tail index of the gradient noise.
  • The experiments compare only against D-SVRGDA (the same algorithm without normalization); a natural next step is benchmarking against clipped decentralized methods on real language data, where clipping-threshold sensitivity is the main pain point.
  • The minimax reformulation's approximation error delta enters the bound, so the theorem suggests a trade-off: smaller delta gives a tighter approximation of the original bilevel problem but worsens condition-number constants. The optimal finite-T penalty scaling is delta = O(K^{-(s-1)/(2s+1)} T^{-(s-1)/(2s+1)}), which practitioners could tune.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies decentralized nonconvex bilevel optimization (3.1) under heavy-tailed stochastic gradients. It replaces the usual strong-convexity and bounded-variance assumptions with a PL condition on the lower-level objective and an s-th moment condition (s∈(1,2]) on the gradient noise. A penalty reformulation converts the problem into the minimax problem (3.2), and Algorithm 1 (D-NSVRGDA) solves it with normalized variance-reduced gradient tracking, avoiding clipping. The central result, Theorem 4.5, gives an in-expectation bound on the average gradient norm of the original objective: with specified step sizes, momentums, initial batch size, and penalty parameter, the rate is O(κ^4 ℓ σ^{(2s-2)/(2s+1)} K^{(s-1)/(2s+1)} T^{-(s-1)/(2s+1)}) plus lower-order terms. Experiments compare the algorithm with a non-normalized variant on synthetic and real datasets.

Significance. If correct, this is a meaningful step: it would be the first decentralized bilevel method with a heavy-tailed noise guarantee that avoids clipping, and it extends normalization-based heavy-tailed analysis from single-level to bilevel optimization. The paper gives explicit, non-fitted rates and recovers the finite-variance O(T^{-1/5}) rate when s=2, K=1. The main theorem, however, is currently unverifiable because the proofs for Lemmas 4.1–4.4 and the omitted z-gradient analogue are not in the submission. In addition, Assumption 3.1.3—PL of the penalized objective with the same constant µ as g—is a strong quantitative condition that is neither derived nor verified. These issues are fixable in principle, so the contribution is promising but not yet established.

major comments (4)
  1. [§4.2 (Lemmas 4.1–4.4, Theorem 4.5)] The core theoretical claims are stated without proofs; the derivations are deferred to an appendix that is not part of the submitted text. For example, after Lemma 4.2 the z-gradient bound is said to be deferred to the Appendix, and Section 4.2.2 cites Eq. (A.81), which does not appear. Since Theorem 4.5 aggregates Lemmas 4.1–4.4 and this omitted z-gradient lemma, the central convergence rate cannot be independently checked. The authors must provide the complete proof chain or a complete appendix before the result can be evaluated.
  2. [Assumption 3.1.3 / Lemma 4.2] Assumption 3.1.3 asserts that hδ(x,y)=δ f(x,y)+g(x,y) is µ-PL in y with the same constant µ as g. This is not implied by smoothness/bounded Hessian of f plus PL of g, and no proof or numerical verification is given. Lemma 4.2's bound on E||∇2 hδ|| and the step-size condition η_x ≤ η_y µ/(2(δL_f+L_g)) both rely exactly on this PL constant. If hδ is not PL, or has a much smaller PL constant, the inner y-minimization is not certified, the condition number κ=ℓ/µ is underestimated, and the rate in Theorem 4.5 collapses. The authors should either prove 3.1.3 under explicit restrictions on δ or verify the PL constant on the models used in Section 5.
  3. [§5.1] The synthetic experiments generate noise from a Cauchy distribution. A Cauchy random variable has no finite s-th moment for any s>1, so Assumption 3.2 is violated for every s∈(1,2]; moreover, its mean is undefined, so the stochastic gradients are not unbiased as required by Assumption 3.2. Thus the experiments are outside the theorem's scope and cannot validate the heavy-tailed assumption. Use a distribution with finite s-th moment (e.g., Student-t with ν>1) or state explicitly that the experiment is only illustrative and outside the assumptions.
  4. [Eq. (4.5)] As typeset, the first and fourth terms contain T with positive exponents, T^{1/(2s+1)}. A convergence bound must be nonincreasing in T; if these terms are accurate, the bound diverges with T and cannot be a convergence rate. If the negative signs were lost in typesetting, please correct them; the dominating term discussed in the text has T^{-(s-1)/(2s+1)}. This needs clarification in the revision.
minor comments (4)
  1. [Algorithm 1, Step 5] The divisions by ||p_t||, ||q_t||, and ||r_t|| are undefined when the argument is zero. A convention (e.g., set the normalized direction to zero) should be stated.
  2. [§4.2.1, Lemma 4.1] The terms 'Gradient Errors' and 'Consensus Errors' are only described in prose. Displaying the full definitions as equations would make Lemma 4.1 self-contained and easier to verify.
  3. [§5, Figures 1–3] No repetitions, error bars, or standard deviations are reported, so the statistical significance of the differences between D-NSVRGDA and D-SVRGDA cannot be assessed.
  4. [§6] There is a typo: 'provided ta theoretical' should be 'provided a theoretical'.

Circularity Check

0 steps flagged

No significant circularity: the convergence rate is derived from explicit assumptions; the only self-citations are background references and are not load-bearing.

full rationale

Theorem 4.5 is a direct consequence of Assumptions 3.1–3.3 via the lemmas stated in Section 4.2 and their proofs in the appendix. The parameters (step sizes, momentums, penalty, batch size) are set as functions of the problem constants (κ, ℓ, σ, s, K, T) and are not fitted to any data. The quantity being bounded, (1/T)Σ E[‖∇Φ(x̄_t)‖], is not defined in terms of those parameter choices, so the bound is not true by construction. The penalty reformulation and the approximation properties of Φδ are attributed to external prior work [3, 19], not to the authors' own papers. The self-citations [11, 48] appear only in the literature review to describe existing decentralized bilevel methods and their limitations; neither is used as the basis for the main theorem, a uniqueness claim, or an ansatz. Assumption 3.1.3, requiring hδ to satisfy the μ-PL condition with the same μ as g, is strong and potentially unverified for the experiments, but that is a validity and applicability concern rather than circularity: the theorem is explicitly conditional on it. Likewise, the heavy-tailed Cauchy noise in Section 5.1 may violate Assumption 3.2's s-th moment bound, but that is an experimental mismatch, not a circular derivation. Overall, the derivation is self-contained and the claimed rate follows from the stated assumptions rather than being equivalent to an input.

Axiom & Free-Parameter Ledger

1 free parameters · 5 axioms · 0 invented entities

The rate depends on the standard smoothness, PL, heavy-tailed noise, and network axioms, plus the penalty reformulation from prior work. No new physical or mathematical entities are introduced, and the theorem's hyperparameters are chosen by the proof rather than fitted to data.

free parameters (1)
  • Step sizes eta_x, eta_y, eta_z; momentum gamma_x, gamma_y, gamma_z; penalty delta; initial batch B0 = Set as specific O(.) functions of K, T, sigma, s in Theorem 4.5
    The convergence rate is established only under these specific choices; they are selected in the analysis to balance gradient, consensus, and approximation errors, not estimated from data. The dependence on s and sigma is an implementability caveat because these are usually unknown in practice.
axioms (5)
  • domain assumption Smoothness and Hessian Lipschitz of f^(k) and g^(k), Assumptions 3.1.1-3.1.2
    Standard regularity conditions used throughout the proof, but not verified on real objective functions.
  • domain assumption PL condition for g and for h_delta = delta f + g in y, Assumption 3.1.3
    The strongest regularity assumption and the weakest premise of the paper; it is used in Lemma 4.2 to control the inner gradient and is not implied by PL of g alone.
  • domain assumption Heavy-tailed noise with bounded s-th moment for s in (1,2], Assumption 3.2
    Weaker than finite variance, but still imposes a moment bound on stochastic first and second order gradients; the entire analysis is conditional on this model.
  • domain assumption Doubly stochastic mixing matrix with spectral gap 1 - lambda, Assumption 3.3
    Standard decentralized consensus assumption; the rate depends on the spectral gap through the consensus error bounds.
  • domain assumption Penalty minimax reformulation approximates the original bilevel problem, Section 3.2
    The paper relies on the reformulation from [3,19] and bounds the approximation error between Phi and Phi_delta; this is cited rather than re-derived in the text.

pith-pipeline@v1.3.0-alltime-deepseek · 15689 in / 15742 out tokens · 145740 ms · 2026-08-04T16:12:52.500209+00:00 · methodology

0 comments
read the original abstract

Existing decentralized stochastic optimization methods assume the lower-level loss function is strongly convex and the stochastic gradient noise has finite variance. These strong assumptions typically are not satisfied in real-world machine learning models. For example, learning on language data typically leads to heavy-tailed gradient. To address these limitations, we develop a novel decentralized stochastic bilevel optimization algorithm for the nonconvex bilevel optimization problem under heavy-tailed noise. Specifically, we develop a normalized stochastic variance-reduced bilevel gradient descent algorithm, which does not rely on any clipping operation. Moreover, we establish its convergence rate by innovatively bounding interdependent gradient sequences under heavy-tailed noise for nonconvex decentralized bilevel optimization problems. As far as we know, this is the first decentralized bilevel optimization algorithm with rigorous theoretical guarantees under heavy-tailed noise. The extensive experimental results confirm the effectiveness of our algorithm in handling heavy-tailed noise.

Figures

Figures reproduced from arXiv: 2509.15543 by Heng Liang, Hongchang Gao, Xinwen Zhang, Yihan Zhang.

Figure 1
Figure 1. Figure 1: The upper-level loss function value and test accuracy on different datasets that are generated with [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The upper-level loss function value and test accuracy on the second synthetic dataset. [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The upper-level loss function value and test accuracy on real-world datasets. [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 16 linked inside Pith

  1. [1]

    K. Ahn, X. Cheng, M. Song, C. Yun, A. Jadbabaie, and S. Sra. Linear attention is (maybe) all you need (to understand transformer optimization).arXiv preprint arXiv:2310.01082, 2023

    Pith/arXiv arXiv 2023

  2. [2]

    Battash, L

    B. Battash, L. Wolf, and O. Lindenbaum. Revisiting the noise model of stochastic gradient descent. In International Conference on Artificial Intelligence and Statistics, pages 4780–4788. PMLR, 2024

    2024

  3. [3]

    L. Chen, J. Xu, and J. Zhang. On finding small hyper-gradients in bilevel optimization: Hardness results and improved analysis. InThe Thirty Seventh Annual Conference on Learning Theory, pages 947–980. PMLR, 2024

    2024

  4. [4]

    X. Chen, M. Huang, and S. Ma. Decentralized bilevel optimization.arXiv preprint arXiv:2206.05670, 2022

    Pith/arXiv arXiv 2022

  5. [5]

    X. Chen, M. Huang, S. Ma, and K. Balasubramanian. Decentralized stochastic bilevel optimization with improved per-iteration complexity.arXiv preprint arXiv:2210.12839, 2022

    Pith/arXiv arXiv 2022

  6. [6]

    Cutkosky and H

    A. Cutkosky and H. Mehta. High-probability bounds for non-convex stochastic optimization with heavy tails.Advances in Neural Information Processing Systems, 34:4883–4895, 2021

    2021

  7. [7]

    Cutkosky and F

    A. Cutkosky and F. Orabona. Momentum-based variance reduction in non-convex sgd.Advances in neural information processing systems, 32, 2019

    2019

  8. [8]

    Y. Dong, S. Ma, J. Yang, and C. Yin. A single-loop algorithm for decentralized bilevel optimization. CoRR, abs/2311.08945, 2023

    Pith/arXiv arXiv 2023

  9. [9]

    C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. InInternational conference on machine learning, pages 1126–1135. PMLR, 2017

    2017

  10. [10]

    Franceschi, P

    L. Franceschi, P. Frasconi, S. Salzo, R. Grazzi, and M. Pontil. Bilevel programming for hyperparameter optimization and meta-learning. InInternational Conference on Machine Learning, pages 1568–1577. PMLR, 2018. 11

    2018

  11. [11]

    H. Gao, B. Gu, and M. T. Thai. On the convergence of distributed stochastic bilevel optimization algorithms over a network. InInternational Conference on Artificial Intelligence and Statistics, pages 9238–9281. PMLR, 2023

    2023

  12. [12]

    Gorbunov, A

    E. Gorbunov, A. Sadiev, M. Danilova, S. Horv´ ath, G. Gidel, P. Dvurechensky, A. Gasnikov, and P. Richt´ arik. High-probability convergence for composite and distributed stochastic minimization and variational inequalities with heavy-tailed noise.arXiv preprint arXiv:2310.01860, 2023

    Pith/arXiv arXiv 2023

  13. [13]

    G¨ urb¨ uzbalaban, Y

    M. G¨ urb¨ uzbalaban, Y. Hu, U. S ¸im¸ sekli, K. Yuan, and L. Zhu. Heavy-tail phenomenon in decentralized sgd.IISE Transactions, pages 1–15, 2024

    2024

  14. [14]

    Huang and S

    F. Huang and S. Chen. Near-optimal decentralized momentum method for nonconvex-pl minimax problems.arXiv preprint arXiv:2304.10902, 2023

    Pith/arXiv arXiv 2023

  15. [15]

    H¨ ubler, I

    F. H¨ ubler, I. Fatkhullin, and N. He. From gradient clipping to normalization for heavy tailed sgd.arXiv preprint arXiv:2410.13849, 2024

    Pith/arXiv arXiv 2024

  16. [16]

    Karimi, J

    H. Karimi, J. Nutini, and M. Schmidt. Linear convergence of gradient and proximal-gradient methods under the polyak- lojasiewicz condition. InJoint European conference on machine learning and knowledge discovery in databases, pages 795–811. Springer, 2016

    2016

  17. [17]

    B. Kong, S. Zhu, S. Lu, X. Huang, and K. Yuan. Decentralized bilevel optimization over graphs: Loopless algorithmic update and transient iteration complexity.arXiv preprint arXiv:2402.03167, 2024

    arXiv 2024

  18. [18]

    J. Kwon, D. Kwon, and H. Lyu. On the complexity of first-order methods in stochastic bilevel opti- mization.arXiv preprint arXiv:2402.07101, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    J. Kwon, D. Kwon, S. Wright, and R. Nowak. On penalty methods for nonconvex bilevel optimization and first-order stochastic approximation.arXiv preprint arXiv:2309.01753, 2023

    Pith/arXiv arXiv 2023

  20. [20]

    J. Kwon, D. Kwon, S. Wright, and R. D. Nowak. A fully first-order method for stochastic bilevel optimization. InInternational Conference on Machine Learning, pages 18083–18113. PMLR, 2023

    2023

  21. [21]

    S. H. Lee, M. Zaheer, and T. Li. Efficient distributed optimization under heavy-tailed noise.arXiv preprint arXiv:2502.04164, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    C. Li, S. Zeng, Z. Liao, J. Li, D. Kang, A. Garcia, and M. Hong. Learning reward and policy jointly from demonstration and preference improves alignment. InNeurIPS 2024 Workshop on Fine-Tuning in Modern Machine Learning: Principles and Scalability, 2024

    2024

  23. [23]

    J. Li, S. Zeng, H. T. Wai, C. Li, A. Garcia, and M. Hong. Getting more juice out of the SFT data: Reward learning from human demonstration improves SFT for LLM alignment. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  24. [24]

    L. Liu, Y. Wang, and L. Zhang. High-probability bound for non-smooth non-convex stochastic opti- mization with heavy tails. InForty-first International Conference on Machine Learning, 2024

    2024

  25. [25]

    Z. Liu, J. Zhang, and Z. Zhou. Breaking the lower bound with (little) structure: Acceleration in non- convex stochastic optimization with heavy-tailed noise. InThe Thirty Sixth Annual Conference on Learning Theory, pages 2266–2290. PMLR, 2023

    2023

  26. [26]

    Z. Liu, X. Zhang, P. Khanduri, S. Lu, and J. Liu. Interact: achieving low sample and communication complexities in decentralized bilevel learning over networks. InProceedings of the Twenty-Third In- ternational Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing, pages 61–70, 2022

    2022

  27. [27]

    Z. Liu, X. Zhang, P. Khanduri, S. Lu, and J. Liu. Prometheus: taming sample and communication complexities in constrained decentralized stochastic bilevel learning. InInternational Conference on Machine Learning, pages 22420–22453. PMLR, 2023

    2023

  28. [28]

    Liu and Z

    Z. Liu and Z. Zhou. Nonconvex stochastic optimization under heavy-tailed noises: Optimal convergence without gradient clipping. InThe Thirteenth International Conference on Learning Representations, 2025. 12

    2025

  29. [29]

    S. Lu, X. Cui, M. S. Squillante, B. Kingsbury, and L. Horesh. Decentralized bilevel optimization for personalized client learning. InICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5543–5547. IEEE, 2022

    2022

  30. [30]

    T. D. Nguyen, T. H. Nguyen, A. Ene, and H. Nguyen. Improved convergence in high probability of clipped gradient methods with heavy tailed noise.Advances in Neural Information Processing Systems, 36:24191–24222, 2023

    2023

  31. [31]

    Nouiehed, M

    M. Nouiehed, M. Sanjabi, T. Huang, J. D. Lee, and M. Razaviyayn. Solving a class of non-convex min-max games using iterative first order methods.arXiv preprint arXiv:1902.08297, 2019

    Pith/arXiv arXiv 1902

  32. [32]

    Pedregosa

    F. Pedregosa. Hyperparameter optimization with approximate gradient. InInternational conference on machine learning, pages 737–746. PMLR, 2016

    2016

  33. [33]

    Z. Qin, Z. Liu, S. Lu, Y. Liang, and J. Liu. DUET: Decentralized bilevel optimization without lower-level strong convexity. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  34. [34]

    Sadiev, M

    A. Sadiev, M. Danilova, E. Gorbunov, S. Horv´ ath, G. Gidel, P. Dvurechensky, A. Gasnikov, and P. Richt´ arik. High-probability bounds for stochastic optimization and variational inequalities: the case of unbounded variance. InInternational Conference on Machine Learning, pages 29563–29648. PMLR, 2023

    2023

  35. [35]

    Shen, P.-Y

    H. Shen, P.-Y. Chen, P. Das, and T. Chen. SEAL: Safety-enhanced aligned LLM fine-tuning via bilevel data selection. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  36. [36]

    Shen and T

    H. Shen and T. Chen. On penalty-based bilevel gradient descent method. InInternational Conference on Machine Learning, pages 30992–31015. PMLR, 2023

    2023

  37. [37]

    H. Shen, Z. Yang, and T. Chen. Principled penalty-based methods for bilevel reinforcement learning and rlhf.arXiv preprint arXiv:2402.06886, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    S ¸im¸ sekli, M

    U. S ¸im¸ sekli, M. G¨ urb¨ uzbalaban, T. H. Nguyen, G. Richard, and L. Sagun. On the heavy-tailed theory of stochastic gradient descent for deep neural networks.arXiv preprint arXiv:1912.00018, 2019

    Pith/arXiv arXiv 1912

  39. [39]

    Simsekli, L

    U. Simsekli, L. Sagun, and M. Gurbuzbalaban. A tail-index analysis of stochastic gradient noise in deep neural networks. InInternational Conference on Machine Learning, pages 5827–5837. PMLR, 2019

    2019

  40. [40]

    T. Sun, X. Liu, and K. Yuan. Gradient normalization provably benefits nonconvex sgd under heavy- tailed noise.arXiv preprint arXiv:2410.16561, 2024

    arXiv 2024

  41. [41]

    X. Wang, X. Chen, S. Ma, and T. Zhang. Fully first-order methods for decentralized bilevel optimization. arXiv preprint arXiv:2410.19319, 2024

    arXiv 2024

  42. [42]

    W. Xian, F. Huang, Y. Zhang, and H. Huang. A faster decentralized algorithm for nonconvex minimax problems.Advances in Neural Information Processing Systems, 34:25865–25877, 2021

    2021

  43. [43]

    heavier-tailed

    H. Yang, P. Qiu, and J. Liu. Taming fat-tailed (“heavier-tailed” with potentially infinite variance) noise in federated learning.Advances in Neural Information Processing Systems, 35:17017–17029, 2022

    2022

  44. [44]

    S. Yang, X. Zhang, and M. Wang. Decentralized gossip-based stochastic bilevel optimization over communication networks.arXiv preprint arXiv:2206.10870, 2022

    Pith/arXiv arXiv 2022

  45. [45]

    T. Yang. Algorithmic foundations of empirical x-risk minimization.arXiv preprint arXiv:2206.00439, 2022

    Pith/arXiv arXiv 2022

  46. [46]

    Zhang, S

    J. Zhang, S. P. Karimireddy, A. Veit, S. Kim, S. Reddi, S. Kumar, and S. Sra. Why are adaptive methods good for attention models?Advances in Neural Information Processing Systems, 33:15383–15393, 2020

    2020

  47. [47]

    Zhang, G

    X. Zhang, G. Mancino-Ball, N. S. Aybat, and Y. Xu. Jointly improving the sample and communication complexities in decentralized stochastic minimax optimization. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 20865–20873, 2024. 13

    2024

  48. [48]

    Zhang, M

    Y. Zhang, M. T. Thai, J. Wu, and H. Gao. On the communication complexity of decentralized bilevel optimization.arXiv preprint arXiv:2311.11342, 2023

    Pith/arXiv arXiv 2023

  49. [49]

    S. Zhu, B. Kong, S. Lu, X. Huang, and K. Yuan. Sparkle: a unified single-loop primal-dual framework for decentralized bilevel optimization.Advances in Neural Information Processing Systems, 37:62912– 62987, 2024. 14

    2024