Pith. sign in

REVIEW 4 major objections 6 minor 43 references

Towards Robust Learning to Optimize with Theoretical Guarantees

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A learning-to-optimize model that is perfectly trained as gradient descent on every in-distribution instance still has no proved convergence guarantee on out-of-distribution instances; the paper bounds the deterioration by the magnitude…

desk verdict A genuinely new virtual-feature OOD analysis for L2O that is internally consistent under a very strong 'trained model equals gradient descent' assumption, but the proposed GO-Math-L2O does not satisfy that assumption, and the upper bounds are over-read as superiority proofs. read the letter →

arxiv 2506.14263 v1 pith:Y3WBFIPS submitted 2025-06-17 cs.LG math.OC

Qingyu Song , Wei Lin , Juncheng Wang , Hong Xu This is my paper
classification cs.LGmath.OC MSC 68T0790C25
keywords learningtooptimizeout-of-distributiongeneralizationconvergenceratevirtualfeaturegradient-onlyfeaturesGO-Math-L2Oconvexoptimizationrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to make robustness to out-of-distribution (OOD) shifts a provable property of learned optimizers instead of an empirical hope. It first identifies a sufficient condition under which a learning-to-optimize (L2O) model improves in every iteration on all in-distribution (InD) instances: the learned update must coincide with gradient descent with step $1/L$. Assuming that condition holds exactly, the paper derives upper bounds showing that OOD per-iteration progress deteriorates from the gradient-descent improvement $-\|\nabla f'\|^2/(2L)$ by positive terms proportional to the magnitude of a 'virtual feature,' the difference between the model's input features in OOD and InD scenarios; the same structure appears for $K$-iteration convergence rates. Because the bounds grow with feature size, the paper proposes an L2O model with gradient-only features, GO-Math-L2O, and reports that it outperforms state-of-the-art baselines in both InD and OOD tests, with up to $10\times$ faster convergence in OOD scenarios. A sympathetic reader should take away a precise statement: input feature construction is not a modeling detail but a first-order determinant of OOD convergence guarantees.

What carries the argument

The load-bearing object is the virtual feature $s' = z' - z$, the difference between L2O input features in OOD and InD scenarios at aligned points, together with the mean-value expansion $d(z')=d(z)+J_d s'$ using a bounded virtual Jacobian $J_d$. This expansion connects each OOD trajectory to an InD trajectory whose convergence behavior is known, and then bounds the gap. The quantitative engine is the per-iteration bound of Theorem 1 and its $\|s'\|$-dependent corollaries, combined with the sufficient condition (Assumption 1) that the trained network equals gradient descent, $N_1=1/(2L)$ and $N_2=\nabla f/(2L)$, on all InD points.

What would settle it

Train a one-dimensional convex quadratic problem with an L2O model constructed so that Assumption 1 holds on a dense InD grid, then run the frozen model on an OOD objective $F'(x)=F(x+t)$ and check whether the per-iteration objective drop ever exceeds the right-hand side of Corollary 2; any violation would refute the bound. Alternatively, evaluate $N_1(z)$ and $N_2(z)$ on held-out InD points and find any point where they differ from $1/(2L)$ and $\nabla f/(2L)$, which would show that the theorem's premise is not satisfied in practice.

Watch

Extended reading notes

Core claim

The paper's central claim is that an L2O model's OOD convergence rate deteriorates by an explicit function of its input features. Under Assumption 1, where after training $N_1(z)=1/(2L)$ and $N_2(z)=\nabla f(x)/(2L)$ on every InD point, Theorem 1 bounds one iteration's OOD improvement by $-\|\nabla f'(x+s)\|^2/(2L)+L\|\mathrm{diag}(J_1s')\nabla f'(x+s)\|^2+L\|(\nabla f'(x+s)-\nabla f(x))/(2L)-J_2s'\|^2$, and Corollary 2 relaxes this to explicit positive terms in $\|s'\|^2$. Theorem 2 and Corollary 3 extend the pattern to $K$ iterations: the gradient-descent telescoping terms remain, but additional sums proportional to $\|s'_{k-1}\|$ prevent any guaranteed global convergence. The paper also argues the same deterioration mechanism holds for non-smooth and longer-horizon settings, and that eliminating variable-related features shrinks $s'$ and therefore tightens every bound. Numerically, the resulting gradient-only GO-Math-L2O converges faster than prior learned and classical baselines in the paper's test problems.

Load-bearing premise

The paper's OOD deterioration equations suppose that after training the learned optimizer is exactly gradient descent with step $1/L$ at every in-distribution point; if a real trained network deviates from this pointwise identity, the stated bounds need not govern its OOD behavior.

Editorial extensions

If this is right

  • An L2O model that behaves exactly as gradient descent on all in-distribution instances still admits no proved convergence guarantee on out-of-distribution instances; the derived upper bounds contain positive terms that can outweigh the negative gradient-descent improvement.
  • The deterioration in the OOD convergence bound scales with the magnitude of the L2O model's input features $\|s'\|$, and this is asserted as a theorem-level statement rather than an empirical trend.
  • Feature designs that eliminate variable-related inputs reduce the dimension and magnitude of $s'$ and thereby tighten the OOD bound for both one iteration and $K$ iterations.
  • For composite objectives, a gradient-only longer-horizon construction preserves the same theoretical structure, and the paper reports that GO-Math-L2O converges up to $10\times$ faster than state-of-the-art baselines in OOD numerical tests.
  • Because the InD model is assumed to be exactly gradient descent, in-distribution convergence can be guaranteed at the rate $L/(2K)\|x_0-x^*\|^2$, while the OOD bound retains extra positive terms that are not guaranteed to vanish.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's Assumption 1 is a pointwise equality, and real networks trained by Adam on finite datasets almost surely violate it; the tightest reading of the theorems applies to an idealized perfectly trained optimizer rather than to every trained checkpoint.
  • If the input-feature dependence is causal as stated, then L2O robustness benchmarks could report measured feature drift $\|s'\|$ alongside objective trajectories; one could test whether models with similar in-distribution accuracy but smaller feature drift exhibit the predicted smaller OOD gap.
  • The same virtual-feature pairing may extend beyond L2O to any learned fixed-point iterator: the bound formalizes a generic price for distribution shift in the input representation, with the Jacobian norm of the learned map as the key constant.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper aims to provide theoretical guarantees for learning-to-optimize (L2O) models in out-of-distribution (OOD) scenarios. It introduces a 'virtual feature' and 'virtual trajectory' to relate an OOD optimization trajectory to an in-distribution (InD) trajectory, and proves several bounds on the per-iteration and multi-iteration convergence of the L2O model in the OOD setting under an assumption that the learned optimizer exactly matches gradient descent on all InD instances. Based on these bounds, the paper claims that larger input feature magnitudes cause greater deterioration of convergence, and proposes a new gradient-only L2O model, GO-Math-L2O, with a history modeling mechanism. Experiments on LASSO and logistic regression, including synthetic and real-world OOD scenarios, show that GO-Math-L2O often converges faster than existing L2O baselines.

Significance. The empirical contribution is solid and the code is provided. The idea of linking OOD behavior to input feature magnitude is interesting and could inspire future work. However, the theoretical foundations are not established: the key assumption is an unverified exact equality, the derived upper bounds do not logically imply deterioration, and the rate statements contain algebraic errors. As a result, the paper's main theoretical claim is not supported, and the theory does not apply to the proposed model.

major comments (4)
  1. [Sec. 4.1, Assumption 1; Sec. 5] Assumption 1 (Sec. 4.1) postulates N1(z_{k-1}) = 1/(2L) I and N2(z_{k-1}) = ∇f(x_{k-1})/(2L) pointwise on all InD inputs. The paper does not show that a network trained with Adam on 32,000 problems satisfies this continuum of equalities, nor does it report any check of the deviation. More importantly, the proposed GO-Math-L2O model (Eqs. (8) and (10)) has no N2 bias term; even in the smooth limit with v=0 and b1=0, its update is x_k = x_{k-1} - R_k ∇f(x_{k-1}) with a learned matrix R_k, so Assumption 1 cannot hold for this architecture. Consequently, Theorems 1, 2 and Corollaries 2, 3, which are presented as the theoretical motivation for gradient-only features, do not apply to the model evaluated in Section 6.
  2. [Sec. 4.1, Theorem 1 and Corollaries 2-3] Theorem 1 bounds F'(x_k+s_k) - F'(x_{k-1}+s_{k-1}) above by -||∇f'||^2/(2L) plus nonnegative residual terms. An upper bound with positive terms does not imply the actual decrease is smaller than gradient descent's; the actual decrease could be larger (more negative). Corollary 2 and 3 similarly bound the objective difference above by quantities that increase with ||s'||, but these are not lower bounds. The paper's statements that 'any OOD will dampen convergence' and that 'Corollaries 2 and 3 theoretically demonstrate the outperformance over the method in [14]' are therefore not logical consequences of the results. A lower bound or a worst-case construction is needed to substantiate the deterioration claim.
  3. [Sec. 4.1, Theorem 2 and Corollary 3] In Theorem 2 and Corollary 3, the first two terms on the right-hand side, L/2 ||x0 - x* + s0 - s*||^2 - L/2 ||xK - x* + sK - s*||^2, are not multiplied by 1/K, while the sum terms are. Since these bounds are derived by summing K per-iteration inequalities and then dividing by K to pass to the minimum, the leading terms must also be divided by K. As stated, the right-hand side is O(1) in K and cannot serve as a convergence rate. This is a load-bearing algebraic error in the paper's main theoretical results.
  4. [Sec. 5, Theorem 3] Theorem 3 claims existence of symmetric positive definite R_k, Q_k, B_k under Conditions FP and GC. In the proof, R_k is defined as the Jacobian block J_{2,k} of a mean-value representation; its positive definiteness is not established from FP and GC. The argument later 'assumes' R_k ≻ 0 to derive the proximal operator form (Eq. (11)). The theorem as stated is therefore not proven; at best it proves existence of bounded matrices with an additional unstated regularity condition.
minor comments (6)
  1. [Throughout] There are numerous typos and grammatical errors in the appendix, including 'inversible', 'contruct', 'Firgure', 'derease', 'entra', and 'objeective'. A careful proofreading pass is needed.
  2. [Sec. 5] The text says the gradient-only feature reduces the input magnitude 'by 1/3', but the change from [x^T, ∇f^T]^T (2n dimensions) to ∇f (n dimensions) is a reduction by 1/2, not 1/3.
  3. [Sec. 8.6 (Appendix)] The sentence 'Note that s := 0 cannot lead to the convergence rate of gradient descent since the third term in equation 34 is non-zero and cannot be eliminated' appears to contradict the immediately preceding derivation, which shows cancellation for s=0. This passage should be reworded for clarity.
  4. [Sec. 2.2] The statement that 'Corollaries 2 and 3 theoretically demonstrate the outperformance over the method in [14]' is an overreach given the upper-bound nature of the results; a larger upper bound does not constitute a proof of worse performance.
  5. [Figures 3, 4, 16, 17] The legends have duplicate entries (e.g., 'GO-Math-L2O, s = 10' appears twice), presumably one curve for +s and one for -s. The captions should clarify which line corresponds to which sign.
  6. [Sec. 6.2] The real-world OOD experiments in Figures 2, 14, and 15 are not clearly defined as OOD; the paper should specify the distribution shift relative to the training distribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity; the OOD-deterioration bounds are conditional on an explicitly stated idealized InD assumption, not equivalent to the paper's inputs or to its own fitted values.

full rationale

The derivation chain is self-contained rather than circular. Corollary 1 starts from the explicit choice N1 = 1/(2L) I and N2 = grad f/(2L), which makes the Math-L2O update exactly gradient descent; this is a construction, not a fitted parameter disguised as a result. Assumption 1 then states this equality as an explicit hypothesis for the InD-trained model, and Theorems 1-2 and Corollaries 2-3 derive OOD bounds from that hypothesis using only L-smoothness, convexity, and the Mean Value Theorem bound on the Jacobians of the network. The conclusion that gradient-only features improve the bound follows algebraically from the definition of the virtual feature s' and the inequality ||s'||^2 >= ||grad f'(x+s) - grad f(x)||^2, i.e. from the paper's own definitions and the triangle inequality, not from the conclusion being assumed. The framework of Math-L2O is imported from Liu et al. [14], whose authors do not overlap with the present paper, so the heavy reliance on [14] is external support, not a self-citation chain. Theorem 3 explicitly conditions on the sufficient Conditions FP and GC and derives a necessary form; it is a conditional mathematical lemma, and the paper even notes that the model does not guarantee FP/GC, so no uniqueness or forced choice is smuggled in. The genuine weaknesses--Assumption 1 requiring pointwise equality of a trained network to 1/L gradient descent, and the fact that the evaluated GO-Math-L2O architecture need not satisfy that assumption--are soundness or applicability concerns, not circularity. No fitted input is renamed as a prediction, and no load-bearing step reduces by construction to its own inputs.

Assumptions & free parameters 1 free parameters · 6 assumptions · 2 invented entities

The theoretical results rest mostly on standard convex analysis plus a strong, unverified post-training assumption (Assumption 1). The empirical GO-Math-L2O performance depends on model parameters fitted by Adam; those parameters do not enter the OOD bounds. The only newly introduced objects are analytical constructs, virtual features and virtual Jacobians, not physical entities.

free parameters (1)
  • R_k, Q_k, B_k (learned parameter matrices in GO-Math-L2O) = Learned by Adam on 32,000 synthetic problems; numeric values not reported
    Empirical claims that GO-Math-L2O converges faster depend on these fitted values. The theory only fixes their structure as bounded, Sigmoid-activated diagonal matrices, not their content.
assumptions (6)
  • standard math f is L-smooth and convex; r is proper, closed, and convex (Sec. 2.1)
    Standard convex optimization assumptions inherited from Math-L2O; they define the problem class.
  • domain assumption L2O map d is differentiable with bounded Jacobian norm ||Jd||_F <= C (Eq. 1)
    Needed so the mean value theorem gives d(z') = d(z) + Jd(z' - z) with a bounded virtual Jacobian.
  • ad hoc to paper Assumption 1: after training, N1 = 1/(2L) I and N2 = grad f/(2L) on all InD points (Sec. 4.1)
    Load-bearing premise for Theorems 1-2 and Corollaries 2-3; no argument or experiment shows trained neural networks satisfy this exact pointwise equality.
  • standard math The line segment between InD feature z and OOD feature z' lies in the domain where d is differentiable (Sec. 3.1, Eq. 3)
    The mean value theorem application requires the segment to stay in the differentiable domain; this is assumed implicitly.
  • ad hoc to paper Conditions FP and GC hold, and the learned operator d_k is invertible enough to recover an implicit subgradient (Sec. 5, Theorem 3)
    Used to derive the structural form of GO-Math-L2O; the paper states the model does not guarantee these conditions and convergence is only promoted by training.
  • ad hoc to paper Assumptions 2 and 3 for composite and long-horizon cases (Appendix Sec. 9 and 11)
    Analogous post-training exactness assumptions used to derive the additional OOD bounds; same status as Assumption 1.
invented entities (2)
  • Virtual feature s' and virtual trajectory {s_k}
    purpose: Bridge OOD L2O outputs to InD outputs so that convergence can be analyzed via feature differences
    An analytical construction, not observable or measurable; its norm enters every theoretical bound but it cannot be verified independently.
  • Virtual Jacobian matrix J_d (and J1, J2, J3)
    purpose: Linearize the L2O network's output difference between OOD and InD features at interior points
    Exists by the mean value theorem but with unknown entries; only the norm bound is used, so no independent falsifiable handle is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Robust Learning to Optimize with Theoretical Guarantees." pith.science (2026). https://pith.science/paper/Y3WBFIPS

@misc{pith2026250614263,
  author       = {Pith},
  title        = {Pith review of: Towards Robust Learning to Optimize with Theoretical Guarantees},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y3WBFIPS}},
  note         = {Machine review of arXiv:2506.14263}
}
abstract

Learning to optimize (L2O) is an emerging technique to solve mathematical optimization problems with learning-based methods. Although with great success in many real-world scenarios such as wireless communications, computer networks, and electronic design, existing L2O works lack theoretical demonstration of their performance and robustness in out-of-distribution (OOD) scenarios. We address this gap by providing comprehensive proofs. First, we prove a sufficient condition for a robust L2O model with homogeneous convergence rates over all In-Distribution (InD) instances. We assume an L2O model achieves robustness for an InD scenario. Based on our proposed methodology of aligning OOD problems to InD problems, we also demonstrate that the L2O model's convergence rate in OOD scenarios will deteriorate by an equation of the L2O model's input features. Moreover, we propose an L2O model with a concise gradient-only feature construction and a novel gradient-based history modeling method. Numerical simulation demonstrates that our proposed model outperforms the state-of-the-art baseline in both InD and OOD scenarios and achieves up to 10 $\times$ convergence speedup. The code of our method can be found from https://github.com/NetX-lab/GoMathL2O-Official.

Figures

Figures reproduced from arXiv: 2506.14263 by the authors.

Figure 1
Figure 1. LASSO Regression: InD. 10 0 10 1 10 2 10 3 Iteration k 10 2 10 2 10 4 10 6 ( F ( x k ) F ( x * ) ) / F ( x * ) ISTA FISTA L2O-DM L2O-RNNprop L2O-PA Adam AdamHD GO-Math-L2O [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. LASSO Regression: Real-World OOD. variable. The OOD proximal operator with t is given by: ( proxr,pk (¯x))i := −t + sign(¯xi) max(0, |x¯i | − λ(pk)i + sign(¯xi)t). 6.1. InD Comparison The trajectories of solving the LASSO Regression problems are shown in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 5
Figure 5. LASSO Regression: Ablation Study on Gradient Map Configurations. [PITH_FULL_IMAGE:figures/full_fig_p059_5.png] view at source ↗
Figures from the paper (12 more)
Figure 6
Figure 6. Figure 6: LASSO Regression: Ablation Study on Training Settings, 20/100 BP Frequency. [PITH_FULL_IMAGE:figures/full_fig_p059_6.png]
Figure 7
Figure 7. Figure 7: LASSO Regression: Ablation Study on Training Settings, 50/100 BP Frequency. [PITH_FULL_IMAGE:figures/full_fig_p060_7.png]
Figure 8
Figure 8. Figure 8: LASSO Regression: Ablation Study on Training Settings, 100/100 BP Frequency. [PITH_FULL_IMAGE:figures/full_fig_p061_8.png]
Figure 9
Figure 9. Figure 9: LASSO Regression: Ablation Study on Training Settings, Best. [PITH_FULL_IMAGE:figures/full_fig_p061_9.png]
Figure 10
Figure 10. Figure 10: LASSO Regression: Ablation Study on Q Settings, InD scenario. 10 0 10 1 10 2 10 3 Iteration k 10 6 10 2 10 4 10 6 10 7 ( F ( x k ) F ( x * ) ) / F ( x * ) Q, s = 10 Q, s = 20 Q, s = 50 Q, s = 100 Q, s = 10 Q, s = 20 Q, s = 50 Q, s = 100 Q/ L, s = 10 Q/ L, s = 20 Q/ L,…
Figure 11
Figure 11. Figure 11: LASSO Regression: Ablation Study on Q Settings, OOD by Trigger 1. 10 0 10 1 10 2 10 3 Iteration k 10 5 10 4 10 3 10 2 10 1 10 0 ( F ( x k ) F ( x * ) ) / F ( x * ) Q, t = 10 Q, t = 20 Q, t = 50 Q, t = 100 Q, t = 10 Q, t = 20 Q, t = 50 Q, t = 100 Q/ L, t = 10 Q/ L, t =…
Figure 12
Figure 12. Figure 12: LASSO Regression: Ablation Study on Q Settings, OOD by Trigger 2. Q/ √ L setting performs similarly to the Q. 12.7. Real-World Evaluation We further evaluate our model on real-world optimization problems. We follow the methodology proposed in [14] to construct the fol…
Figure 13
Figure 13. Figure 13: Logistic Regression: InD. 10 0 10 1 10 2 10 3 Iteration k 10 1 10 2 10 4 10 6 10 7 ( F ( x k ) F ( x * ) ) / F ( x * ) ISTA FISTA L2O-DM L2O-RNNprop L2O-PA Adam AdamHD GO-Math-L2O [PITH_FULL_IMAGE:figures/full_fig_p063_13.png]
Figure 14
Figure 14. Figure 14: Logistic Regression: Real-World Ionoshpere Dataset. [PITH_FULL_IMAGE:figures/full_fig_p063_14.png]
Figure 15
Figure 15. Figure 15: Logistic Regression: Real-World Spambase Dataset. [PITH_FULL_IMAGE:figures/full_fig_p063_15.png]
Figure 16
Figure 16. Figure 16: Logistic Regression: OOD by Trigger 1. 10 0 10 1 10 2 10 3 Iteration k 10 2 10 1 10 0 10 1 10 2 ( F ( x k ) F ( x * ) ) / F ( x * ) GO-Math-L2O, t = 10 GO-Math-L2O, t = 20 GO-Math-L2O, t = 50 GO-Math-L2O, t = 100 GO-Math-L2O, t = 10 GO-Math-L2O, t = 20 GO-Math-L2O, t …
Figure 17
Figure 17. Figure 17: Logistic Regression: OOD by Trigger 2. where s denotes the extent of initial point shifting. Results in t = ±10, ±20 cases demonstrate that our proposed GO-Math￾L2O method converges significantly faster than L2O-PA [14]. For t = ±50, ±100 cases, our model can also con…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 38 canonical work pages

  1. [14]

    Towards Constituting Mathematical Structures for Learning to Optimize

    Jialin Liu, Xiaohan Chen, Zhangyang Wang, Wotao Yin, and HanQin Cai. Towards Constituting Mathematical Structures for Learning to Optimize. In ICML, 2023. 1, 2, 3, 4, 6, 7, 8, 14, 15, 16, 17, 29, 30, 31, 48, 49, 50, 52, 53, 54, 55

    work page 2023

  2. [1]

    Ada-lista: Learned solvers adaptive to varying models

    Aviad Aberdam, Alona Golts, and Michael Elad. Ada-lista: Learned solvers adaptive to varying models. IEEE TPAMI, 44(12):9222–9235, 2021. 7

    work page 2021

  3. [2]

    A generalizable approach to learning optimizers

    Diogo Almeida, Clemens Winter, Jie Tang, and Wojciech Zaremba. A generalizable approach to learning optimizers. arXiv preprint arXiv:2106.00958, 2021. 1

    arXiv 2021

  4. [3]

    Learning to learn by gradient descent by gradient descent

    Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas. Learning to learn by gradient descent by gradient descent. In NeurIPS, 2016. 7

    work page 2016

  5. [4]

    Online learning rate adaptation with hypergradient descent

    Atilim Gunes Baydin, Robert Cornish, David Martinez Ru- bio, Mark Schmidt, and Frank Wood. Online learning rate adaptation with hypergradient descent. arXiv preprint arXiv:1703.04782, 2017. 7

    arXiv 2017

  6. [5]

    A fast iterative shrinkage- thresholding algorithm for linear inverse problems

    Amir Beck and Marc Teboulle. A fast iterative shrinkage- thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences , 2(1):183–202, 2009. 7, 8, 16, 29, 45, 49

    work page 2009

  7. [6]

    A Deep Q-Network Based-Resource Allocation Scheme for Massive MIMO-NOMA

    Yanmei Cao, Guomei Zhang, Guobing Li, and Jia Zhang. A Deep Q-Network Based-Resource Allocation Scheme for Massive MIMO-NOMA. IEEE Communications Letters, 25 (5):1544–1548, 2021. 1

    work page 2021

  8. [7]

    Learning to optimize: A primer and a benchmark

    Tianlong Chen, Xiaohan Chen, Wuyang Chen, Zhangyang Wang, Howard Heaton, Jialin Liu, and Wotao Yin. Learning to optimize: A primer and a benchmark. The Journal of Machine Learning Research, 23(1):8562–8620, 2022. 1

    work page 2022

Show all 43 references
  1. [8]

    Learning Fast Approxima- tions of Sparse Coding

    Karol Gregor and Yann LeCun. Learning Fast Approxima- tions of Sparse Coding. In ICML, pages 399–406, 2010. 1

    2010

  2. [9]

    Safeguarded learned convex optimization

    Howard Heaton, Xiaohan Chen, Zhangyang Wang, and Wotao Yin. Safeguarded learned convex optimization. In AAAI, pages 7848–7855, 2023. 1

    2023

  3. [10]

    Proof of convergence for the proximal point al- gorithm

    Michael (https://math.stackexchange.com/users/155065/michael). Proof of convergence for the proximal point al- gorithm. Mathematics Stack Exchange, 2015. URL:https://math.stackexchange.com/q/1303325 (ver- sion: 2015-05-30). 14, 17

    2015

  4. [11]

    Iterative Algorithm Induced Deep- Unfolding Neural Networks: Precoding Design for Mul- tiuser MIMO Systems

    Qiyu Hu, Yunlong Cai, Qingjiang Shi, Kaidi Xu, Guand- ing Yu, and Zhi Ding. Iterative Algorithm Induced Deep- Unfolding Neural Networks: Precoding Design for Mul- tiuser MIMO Systems. IEEE TWC, 20(2):1394–1410, 2020. 1

    2020

  5. [12]

    Kalman and S.C

    B.L. Kalman and S.C. Kwasny. Why tanh: choosing a sig- moidal function. In IJCNN International Joint Conference on Neural Networks, pages 578–581 vol.4, 1992. 4

    1992

  6. [13]

    A method for stochastic optimization

    D Kinga, Jimmy Ba Adam, et al. A method for stochastic optimization. In ICLR, page 6. San Diego, California;, 2015. 7, 50

    2015

  7. [15]

    Learning gradient descent: Better generalization and longer horizons

    Kaifeng Lv, Shunhua Jiang, and Jian Li. Learning gradient descent: Better generalization and longer horizons. InICML, pages 2247–2255. PMLR, 2017. 7, 29

    2017

  8. [16]

    Learning gradient descent: Better generalization and longer horizons

    Kaifeng Lv, Shunhua Jiang, and Jian Li. Learning gradient descent: Better generalization and longer horizons. InICML, pages 2247–2255. PMLR, 2017. 1

    2017

  9. [17]

    The generalized sigmoid activation func- tion: Competitive supervised learning

    Sridhar Narayan. The generalized sigmoid activation func- tion: Competitive supervised learning. Information Sciences, 99(1):69–82, 1997. 4, 7

    1997

  10. [18]

    Monotone operators and the proximal point algorithm

    R Tyrrell Rockafellar. Monotone operators and the proximal point algorithm. SIAM journal on control and optimization, 14(5):877–898, 1976. 6, 17

    1976

  11. [19]

    An overview of gradient descent optimiza- tion algorithms

    Sebastian Ruder. An overview of gradient descent optimiza- tion algorithms. CoRR, abs/1609.04747, 2016. 1, 3, 6

    2016 arXiv

  12. [20]

    W. Rudin. Principles of Mathematical Analysis . McGraw- Hill, New York, 1976. 3

    1976

  13. [21]

    Coordinated Sum- Rate Maximization in Multicell MU-MIMO With Deep Un- rolling

    Lukas Schynol and Marius Pesavento. Coordinated Sum- Rate Maximization in Multicell MU-MIMO With Deep Un- rolling. IEEE JSAC, 41(4):1120–1134, 2023. 1

    2023

  14. [22]

    Yifei Shen, Yuanming Shi, Jun Zhang, and Khaled B. Letaief. Graph Neural Networks for Scalable Radio Re- source Management: Architecture Design and Theoretical Analysis. IEEE JSAC, 39(1):101–115, 2021. 1

    2021

  15. [23]

    Towards out-of-distribution generalization: A survey

    Z Shen, J Liu, Y He, X Zhang, R Xu, H Yu, and P Cui. Towards out-of-distribution generalization: A survey. arXiv preprint arXiv:2108.13624, 2023. 1

    2023 arXiv

  16. [24]

    Prac- tical bayesian optimization of machine learning algorithms

    Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Prac- tical bayesian optimization of machine learning algorithms

  17. [25]

    Subgradient Methods

    Stephen Boyd, Lin Xiao, and Almir Mutapcic. Subgradient Methods. Stanford EE392o Optimization Projects, 2003. 6

    2003

  18. [26]

    Learning to optimize: Training deep neural networks for interference management

    Haoran Sun, Xiangyi Chen, Qingjiang Shi, Mingyi Hong, Xiao Fu, and Nicholas D Sidiropoulos. Learning to optimize: Training deep neural networks for interference management. IEEE TSP, 66(20):5438–5453, 2018. 1

    2018

  19. [27]

    Lecture 6: September 12

    Ryan Tibshirani. Lecture 6: September 12. CMU 10-725: Optimization, 2013. 2, 3

    2013

  20. [28]

    Vandenberghe

    L. Vandenberghe. Proximal gradient method. ECE236C (Spring 2022), 2022. 6, 17

    2022

  21. [29]

    Learned optimizers that scale and generalize

    Olga Wichrowska, Niru Maheswaranathan, Matthew W Hoffman, Sergio Gomez Colmenarejo, Misha Denil, Nando Freitas, and Jascha Sohl-Dickstein. Learned optimizers that scale and generalize. In ICML, pages 3751–3760. PMLR,

  22. [30]

    Learn- ing to Generalize Provably in Learning to Optimize

    Junjie Yang, Tianlong Chen, Mingkang Zhu, Fengxiang He, Dacheng Tao, Yingbin Liang, and Zhangyang Wang. Learn- ing to Generalize Provably in Learning to Optimize. InInter- national Conference on Artificial Intelligence and Statistics, pages 9807–9825, 2023. 1

    2023

  23. [31]

    Niemegeers, and Sonia M.Heemstra De Groot

    Yu Zhao, Ignas G. Niemegeers, and Sonia M.Heemstra De Groot. Dynamic Power Allocation for Cell-Free Massive MIMO: Deep Reinforcement Learning Methods. IEEE Ac- cess, 9:102953–102965, 2021. 1

    2021

  24. [32]

    On the fenchel duality between strong con- vexity and lipschitz continuous gradient

    Xingyu Zhou. On the fenchel duality between strong con- vexity and lipschitz continuous gradient. arXiv preprint arXiv:1803.06573, 2018. 17

    2018 arXiv

  25. [33]

    Learn- ing to beamform in heterogeneous massive MIMO networks

    Minghe Zhu, Tsung-Hui Chang, and Mingyi Hong. Learn- ing to beamform in heterogeneous massive MIMO networks. IEEE TWC, 2022. 1 9 Towards Robust Learning to Optimize with Theoretical Guarantees Supplementary Material

    2022

  26. [34]

    well-trained

    Proofs 8.1. Preliminary Demonstration of Equation 3 Proof. Based on demonstration for Lemma 1 in [14], sinced ∈ DC(m), the outcome ofd is an n-dimensional vector. Denote the i-th element as di(1 ≤ i ≤ n) and convert d into a matrix form: d(z′) = d1(z′), . . . , dn(z′) ⊤ , d(z)...

  27. [35]

    Similar to the results in the smooth case of main pages, we derive several theorems and corollaries on per iteration and multi-iteration convergence of the L2O model

    Composite Case Results This section introduces several more theoretical findings on the composite case where the smooth and non-smooth parts in objective P are non-degenerated. Similar to the results in the smooth case of main pages, we derive several theorems and corollaries ...

  28. [36]

    Based on the definition, r(x) is proper and convex, where the “proper” means r(x) is trivially solvable for any x

    Non-Smooth Case Results For the non-smooth case, we set the smooth part in the objective of problem P to be zero f (x) := 0 , and the objective becomes: min x r(x), (P) where x ∈ SP and r ∈ FP . Based on the definition, r(x) is proper and convex, where the “proper” means r(x) ...

  29. [37]

    Longer Horizon Case Results In the smooth and composite cases, we have demonstrated convergence analysis per iteration and multi-iteration convergence analysis for L2O. Modern algorithms utilize historical information to accelerate convergence, such as Nesterov momentum in FIS...

  30. [38]

    The gradient-based longer horizon modeling method is more robust in OOD scenarios

    L ∈ [0, 1]. The gradient-based longer horizon modeling method is more robust in OOD scenarios

  31. [39]

    By setting C g 4 ≤ C v 4 /(L2), the gradient-based longer horizon modeling method is more robust in OOD scenarios

    L ∈ (1, ∞]. By setting C g 4 ≤ C v 4 /(L2), the gradient-based longer horizon modeling method is more robust in OOD scenarios. With Subgradient Case We eliminate this case since we assume r(x) is a proper function that can be trivially solved. 11.4. OOD Multi-Iteration Converg...

  32. [40]

    Implementation Details Our implementation is conducted with PyTorch based on the open-source code provided by the official implementation of

    Details of Experiments 12.1. Implementation Details Our implementation is conducted with PyTorch based on the open-source code provided by the official implementation of

  33. [41]

    BP Frequency

    in https://github.com/xhchrn/MS4L2O. We follow the settings in [14] to implement our GO-Math-L2O model. We construct a coordinate-wise model where our model takes gradient features according to a variable as an input and generates the update for that coordinate independently o...

  34. [42]

    1,000 patches are chosen from the BSDS500 dataset

    LASSO Regression. 1,000 patches are chosen from the BSDS500 dataset. A are calculated with K-SVD method and λ is set to be 0.5

  35. [43]

    Ionoshpere dataset contains 4,601 ai, bi ∈ R34 for each sample

    Logistic Regression. Ionoshpere dataset contains 4,601 ai, bi ∈ R34 for each sample. Spambase dataset contains 4,601 ai, bi ∈ R57 for each sample. 53 100 101 102 103 Iteration k 102 10 2 10 4 10 6 10 8 (F(xk) F(x * ))/F(x * ) ISTA FISTA AdaLISTA L2O-DM L2O-RNNprop L2O-PA Adam ...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.