{"total":14,"items":[{"citing_arxiv_id":"2606.23364","ref_index":38,"ref_count":1,"confidence":0.9,"is_internal_anchor":false,"paper_title":"Convergence of Gradient Descent for General Neural Network Architectures Beyond the NTK Regime","primary_cat":"cs.LG","submitted_at":"2026-06-22T14:00:26+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Proves GD convergence to stationary point neighborhoods for general NN architectures beyond NTK via block-level analysis, analyticity, and local smoothness conditions.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.22436","ref_index":5,"ref_count":1,"confidence":0.9,"is_internal_anchor":false,"paper_title":"Distribution-Aware Robust Bilevel Optimization: Quantile-Guided Huber Updates in Two-Timescale Stochastic Approximation","primary_cat":"cs.LG","submitted_at":"2026-06-21T10:57:37+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"RQ-TTSA achieves O(T^{-(p-1)/(3p-2)}) convergence for nonconvex-strongly convex bilevel optimization under heavy-tailed noise (p in (1,2]) via quantile-guided Huber clipping and shows empirical gains on vision, games, and RL tasks.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.21581","ref_index":70,"ref_count":1,"confidence":0.9,"is_internal_anchor":false,"paper_title":"Convergence Analysis of Muon-type Methods with Inexact LMO in the Degenerate Case","primary_cat":"math.OC","submitted_at":"2026-06-19T16:36:53+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Convergence rates are derived for Muon-type methods with inexact LMO in the degenerate case under novel assumptions and layer-wise (L^0, L^1)-smoothness for non-convex and star-convex objectives with weight decay.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.08783","ref_index":48,"ref_count":2,"confidence":0.9,"is_internal_anchor":false,"paper_title":"OptMuon: Closed-Loop Orthogonalized Momentum Methods for Stochastic Optimization with Zero-Noise Optimality","primary_cat":"math.OC","submitted_at":"2026-06-07T18:59:24+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"OptMuon combines orthogonalized momentum with trajectory-dependent AdaGrad-Norm adaptation to obtain expected-stationarity rates of order T^{-1/2} + sigma^{1/2}T^{-1/4} or T^{-1/2} + sigma^{1/3}T^{-1/3} that reduce to near-optimal deterministic first-order rates in the zero-noise regime.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2606.04384","ref_index":45,"ref_count":1,"confidence":0.9,"is_internal_anchor":false,"paper_title":"Revisiting Privacy Amplification by Subsampling in Selective Release DPSGD","primary_cat":"cs.LG","submitted_at":"2026-06-03T03:00:26+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"DPSR-CG corrects the privacy accounting for selective release in DPSGD by addressing sampling probability variation and reports strong empirical results on MNIST, CIFAR-10, IMDB, and FMNIST while claiming strict privacy.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.15522","ref_index":3,"ref_count":1,"confidence":0.9,"is_internal_anchor":false,"paper_title":"Stochastic Non-Smooth Convex Optimization with Unbounded Gradients","primary_cat":"math.OC","submitted_at":"2026-05-15T01:43:22+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Clipped AdamW with exponentially weighted accumulation achieves superior global convergence rates for convex stochastic generalized Lipschitz optimization compared to SGD and AdaGrad.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.15314","ref_index":44,"ref_count":1,"confidence":0.9,"is_internal_anchor":false,"paper_title":"Beyond Bounded Variance: Variance-Reduced Normalized Methods for Nonconvex Optimization under Blum-Gladyshev Noise","primary_cat":"cs.LG","submitted_at":"2026-05-14T18:27:49+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Normalized momentum SGD and variance-reduced STORM achieve O(ε^{-6}) and O(ε^{-4}) oracle complexities respectively under quadratic distance-dependent noise in nonconvex stochastic optimization.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.12666","ref_index":42,"ref_count":1,"confidence":0.9,"is_internal_anchor":false,"paper_title":"Newton methods beyond Hessian Lipschitz continuity: A nonlinear preconditioning approach","primary_cat":"math.OC","submitted_at":"2026-05-12T19:16:13+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"Nonlinear preconditioning extends Newton methods to objectives lacking Hessian Lipschitz continuity by analyzing a transformed mapping under a relaxed smoothness condition, with superlinear convergence and O(ε^{-3/2}) iteration complexity.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2605.11850","ref_index":71,"ref_count":1,"confidence":0.9,"is_internal_anchor":false,"paper_title":"Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives","primary_cat":"math.OC","submitted_at":"2026-05-12T09:36:13+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"Proximal stochastic spectral preconditioning converges for nonconvex constrained objectives under heavy-tailed noise, with a variance-reduced version achieving faster rates and a refined analysis of Muon iterations.","context_count":1,"top_context_role":"background","top_context_polarity":"background","context_text":"More precisely, if D is an estimate of the gradient and D = UΣV ⊤ a reduced SVD, only the productU V ⊤ is considered in the update. This modification of the update direction places spectral gradient methods within a broader class of adaptive algorithms that normalize the gradients before updating the variables. Prominent examples include gradient clipping [71, 28], normalized gradient [17] and sign gradient [9, 6] among others. These methods have been studied extensively not only for the practical benefit of stabilizing the training procedure, but also for their theoretical properties of adapting to generalized forms of smoothness [71, 64] and handling heavy-tailed noise [17, 24]. Despite this growing body of work, existing analyses have predominantly focused on"},{"citing_arxiv_id":"2605.05660","ref_index":27,"ref_count":1,"confidence":0.9,"is_internal_anchor":false,"paper_title":"Distributionally Robust Multi-Objective Optimization","primary_cat":"cs.LG","submitted_at":"2026-05-07T04:24:17+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"DR-MOO adds distributional robustness to multi-objective optimization and gives single-loop MGDA algorithms reaching epsilon-Pareto-stationary points in O(epsilon^{-4}) samples for nonconvex problems.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2604.28020","ref_index":22,"ref_count":1,"confidence":0.9,"is_internal_anchor":false,"paper_title":"Cost-Aware Learning","primary_cat":"cs.LG","submitted_at":"2026-04-30T15:39:09+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"Cost-Aware SGD samples by gradient-norm-to-cost ratio and is instantiated as Cost-Aware GRPO for length-dependent policy gradients, reducing tokens used in LLM RL while matching baseline accuracy.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2604.17560","ref_index":14,"ref_count":1,"confidence":0.9,"is_internal_anchor":false,"paper_title":"The Multi-Block DC Function Class: Theory, Algorithms, and Applications","primary_cat":"math.OC","submitted_at":"2026-04-19T17:59:22+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":7.0,"formal_verification":"none","one_line_summary":"The Multi-Block DC class admits polynomial-size DC decompositions for problems that require exponential size under standard DC programming and supplies explicit constructive formulations for deep ReLU networks together with convergent batch and stochastic algorithms.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2510.16468","ref_index":21,"ref_count":2,"confidence":0.9,"is_internal_anchor":false,"paper_title":"Frank-Wolfe Algorithms for (L0, L1)-smooth functions","primary_cat":"math.OC","submitted_at":"2025-10-18T12:26:28+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":5.0,"formal_verification":"none","one_line_summary":"Proposes (L0, L1)-Frank-Wolfe and adaptive variant claiming superior convergence rates for (L0, L1)-smooth objectives over classical Frank-Wolfe.","context_count":0,"top_context_role":null,"top_context_polarity":null,"context_text":null},{"citing_arxiv_id":"2003.00295","ref_index":44,"ref_count":1,"confidence":0.9,"is_internal_anchor":false,"paper_title":"Adaptive Federated Optimization","primary_cat":"cs.LG","submitted_at":"2020-02-29T16:37:29+00:00","verdict":"UNVERDICTED","verdict_confidence":"LOW","novelty_score":6.0,"formal_verification":"none","one_line_summary":"Proposes federated adaptive optimizers (FedAdagrad, FedAdam, FedYogi) with convergence analysis for non-convex objectives under data heterogeneity and reports empirical gains over FedAvg.","context_count":1,"top_context_role":"background","top_context_polarity":"unclear","context_text":"For FEDADAGRAD , we setβ1 =β2 = 0 (as typical versions of ADAGRAD do not use momentum). For FEDADAM and FEDYOGI we setβ1 = 0.9,β 2 = 0.99. While these parameters are generally 22 Published as a conference paper at ICLR 2021 Algorithm 5 FEDADAGRADFEDADAGRADFEDADAGRAD , FEDYOGIFEDYOGIFEDYOGI , and FEDADAMFEDADAMFEDADAM - Batched data Input:x0,v−1≥τ 2, optionalβ1,β 2∈ [0, 1) for FEDYOGI and FEDADAM fort = 0,··· ,T − 1 do Sample a subsetS of clients xt i =xt for each clienti∈S in parallel do fore = 1,...,E do forb∈B i do xt i =xt i−ηl∇fi(xt i;b) ∆t i =xt i−xt n = ∑ i∈Sni, ∆t = ∑ i∈S ni n ∆t i mt =β1mt−1 + (1−β1)∆t vt =vt−1 + ∆2 t (FEDADAGRAD )(FEDADAGRAD )(FEDADAGRAD ) vt =vt−1− (1−β2)∆2 t sign(vt−1− ∆2 t ) (FEDYOGI)(FEDYOGI)(FEDYOGI) vt =β2vt−1 + (1−β2)∆2 t (FEDADAM)(FEDADAM)(FEDADAM) xt+1 =xt +η mt√vt+τ good choices (Zaheer et al., 2018), we emphasize that better results may be obtainable by tuning these parameters. B.2 SCAFFOLD As discussed in Section 5, we compare all five optimizers above toSCAFFOLD (Karimireddy et al., 2019) on our various tasks. There are a few important notes about the validity of this comparison. 1. In cross-device settings, this is not a fair comparison. In particular, SCAFFOLD does not work in settings where clients cannot maintain state across rounds, as may be the case for federated learning systems on edge devices, such as cell phones. 2. SCAFFOLD has two variants described by Karimireddy et al. (2019). In Option I, the control variate of a client is updated using a full gradient computation. This effectively requires performing an extra pass over each client's dataset, as compared to Algorithm 1. In order to normalize the amount of client work, we instead use Option II, in which the clients' control variates are updated using the difference between the server model and the client's learned model. This requires the same amount of client work as FEDAVG and Algorithm 2. For practical reasons, we implement a version of SCAFFOLD mirro"}],"limit":50,"offset":0}