Pith. sign in

REVIEW 4 major objections 5 minor 58 references

Learnable Loss Geometries with Mirror Descent for Scalable and Convergent Meta-Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Meta-learning can learn a nonlinear loss geometry, not just a quadratic preconditioner, and still converge at the same rate as gradient-descent meta-learning.

desk verdict Worth a serious look: fresh parameterization and strong empirical gains, but the central convergence claim leans on an unverified smoothness assumption and the batch-size corollary looks internally inconsistent. read the letter →

arxiv 2509.02418 v1 pith:IS6LUFRV submitted 2025-09-02 cs.LG

Yilang Zhang , Bingcong Li , Georgios B. Giannakis This is my paper
classification cs.LG
keywords meta-learningmirrordescentdistance-generatingfunctionBregmandivergencebileveloptimizationfew-shotlearninglossgeometryconvexLipschitz-smoothneuralnetwork
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the per-task optimization inside meta-learning should be viewed as mirror descent driven by a learned distance-generating function, rather than as gradient descent with a fixed or learned quadratic preconditioner. It constructs a neural network h* that is provably convex and Lipschitz-smooth, so its Fenchel conjugate h is a valid strongly convex distance generator and the induced mirror map is nonlinear. The authors prove that meta-learning over this learned geometry reaches an epsilon-stationary point in O(epsilon^-2) iterations, matching GD-based meta-learning, and that the framework covers all preconditioned GD methods as special cases. If true, this would let a meta-learned prior capture non-quadratic loss geometry, adapt in as few as one step, and scale to larger models with less per-task unrolling.

What carries the argument

The central object is the distance-generating function h and its Fenchel conjugate h*, whose Bregman divergence D_h defines the task-level geometry. MetaMiDA learns h*(z; theta_h) as a convex, Lipschitz-smooth neural network and updates the dual variable z via z^{k+1} = z^k - alpha grad(ell_t^trn)(grad h*(z^k)), with the primal recovered as phi = grad h*(z). Theorem 1 certifies that the network parameterization is a valid conjugate, Proposition 3 bounds the meta-loss smoothness, and the estimator in (12) makes the adaptive step sizes tractable.

What would settle it

Train MetaMiDA on a few-shot benchmark and measure, for pairs of nearby meta-parameters, the empirical Lipschitz constant of the meta-loss gradient after K=1,2,5,10 unrolled steps. If that constant grows geometrically with K rather than staying bounded, the key bound in Proposition 3 and the resulting O(epsilon^-2) rate would fail for the actual parameterization.

Watch

Extended reading notes

Core claim

The central claim is that the right object to meta-learn is a Bregman distance h, represented through its convex conjugate h*. The paper parameterizes h* as a neural network with convex, non-decreasing, Lipschitz-smooth activations, bounded non-negative weights, and skip connections, and proves this network is convex and Lipschitz-smooth, hence h is strongly convex and the mirror-descent update is well defined. Under Assumptions 1–3, the paper proves that Algorithm 1 (MetaMiDA) converges to an epsilon-stationary point of the meta-objective in O(epsilon^-2) iterations. The proof handles the unbounded smoothness of the meta-loss by bounding it in Proposition 3, constructing an unbiased smoothn

Load-bearing premise

The proof assumes the gradient of the validation loss composed with the learned mirror map stays Lipschitz in the meta-parameters with a constant that does not blow up with the number of adaptation steps; this is assumed rather than derived from the network parameterization, and if the constant grows with K, the advertised O(epsilon^-2) rate and the unbiased estimator are no longer guaranteed.

Editorial extensions

If this is right

  • Because quadratic norms are a special case of the learned distance-generating function, every PGD-based meta-learning method satisfying Assumptions 1–3 inherits a convergence guarantee at O(epsilon^-2).
  • MetaMiDA retains competitive accuracy with K=1 adaptation step, reducing the time and memory cost that grows linearly with K.
  • The unbiased smoothness estimator avoids the extra train-validation gradient closeness assumption used by earlier meta-learning convergence analyses.
  • Cross-domain few-shot results suggest the learned loss geometry transfers across related domains when the shared model structure stays fixed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the transfer result holds generally, the same learned-conjugate construction could be applied to other bilevel optimization problems, such as hyperparameter optimization or learned optimizers, wherever the inner loop is currently a fixed number of gradient steps.
  • Theorem 1 suggests a broader recipe for making learned optimizers provably convergent: enforce convexity and smoothness of the potential h* rather than constraining the update map itself; this could be tested with alternative input-convex network architectures.
  • A testable consequence is that on losses with strongly non-quadratic curvature, MetaMiDA should outperform full-matrix preconditioners by an amount that grows with the deviation from quadratic curvature; synthetic losses with known Hessian variation would quantify this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MetaMiDA, a meta-learning algorithm whose task-level inner loop uses mirror descent driven by a learned distance-generating function (DGF). The conjugate h* of the DGF is parameterized by an input-convex neural network with skip connections and bounded, non-negative weights, which is shown in Theorem 1 and Corollary 2 to be convex and Lipschitz-smooth in the dual variable. The outer loop meta-learns the dual initialization and the DGF parameters via mini-batch SGD. The main theoretical result (Theorem 6 and Corollary 7) is an O(ε^{-2}) iteration complexity for reaching an ε-stationary point, claimed to match GD-based meta-learning while covering preconditioned GD (PGD) methods as a special case. Experiments on miniImageNet, tieredImageNet, CUB, and Cars report accuracy improvements over PGD baselines, including cross-domain settings and reduced adaptation steps.

Significance. If the convergence theorem holds for the proposed neural parameterization, the paper would make a solid contribution: it moves beyond quadratic preconditioners to nonlinear mirror maps, unifies the analysis of PGD-based meta-learning within a mirror-descent framework, and provides detailed proofs and extensive experiments. The empirical results are promising, with clear gains in few-shot classification, and the paper includes useful ablations on adaptation steps and complexity. The main risk is that the advertised rate rests on strong global smoothness assumptions that are not verified for the NN construction; the paper's own proof of the central smoothness bound (Proposition 3) relies critically on Assumption 1.iv, so the theorem may not apply to the very model proposed.

major comments (4)
  1. [§3.3, Assumption 1.iv; Proposition 3 proof, Eqs. (22) and (31)] Assumption 1.iv is load-bearing: it is used to bound ∥g_K^t − g̃_K^t∥ and ∥h_K^t − h̃_K^t∥ in the proof of Proposition 3, and Proposition 3 is used in Theorem 6. However, Theorem 1 and Corollary 2 only prove Lipschitz-smoothness of h* with respect to the dual variable z, with constants uniform in θh; they do not control derivatives of ∇1h* with respect to θh. For the proposed parameterization, ∇1h* includes the term P(θP)z when the optional quadratic term is used, and the composition ℓval_t ∘ ∇1h* may have unbounded gradient as θz or θh grows. Algorithm 1 has no projection or trust region, so Assumption 1.iv must hold globally on R^D, yet no verification is supplied. The authors should either prove Assumption 1.iv for the NN parameterization (with explicit bounds on all parameters, including biases) or revise the theorem to a setting where the assumption is actually verified, e.g., by pr
  2. [§3.3, Assumption 3.ii; Proposition 3 proof, Eqs. (20) and (26)] Assumption 3.ii requires the mixed Jacobian (Hessian in the dual variable) of h* to be Lipschitz in θ, jointly in θz and θh. This third-order condition is used in the bound of ∥G_k^t − G̃_k^t∥ and is not implied by Theorem 1 or Corollary 2. The claim in Remark 2 that Assumptions 1-3 are 'mild' is therefore not supported by the analysis in the paper. Since the proposed NN has parameters θh that include all weights and biases, this is another unverified global condition on which the advertised convergence rate depends.
  3. [§4.2, Table 1 and Algorithm 1] The experiments implement MetaMiDA with a constant meta-learning rate and explicitly do not use the per-step estimator \(G\widehat{G}^r_{L,j}\) from Eq. (12). However, Theorem 6's proof relies on the adaptive rate β_r = 1/(Cβ \widehat{G}^r_{L,j}) and on the moment bounds in Proposition 4. The text asserts that a constant rate 'essentially assumes ℓtrn_t is Lipschitz-continuous', but no convergence theorem is given for the constant-rate variant. Thus the reported empirical validation concerns an algorithm that is not covered by the formal convergence analysis. Please state a theorem for the constant-rate version or clearly present the experiments as a heuristic variant.
  4. [§1 and Remark 4] The paper repeatedly claims that the analysis relies on 'fewer assumptions' than GD-based meta-learning [28]-[30]. The comparison is not substantiated: the paper replaces the train/validation gradient similarity assumption with Assumptions 1.iv and 3.ii, which are global conditions on the learned mirror map and are not verified. A quantitative comparison of the assumption sets, or at least a precise statement of which assumptions are dropped and which are added, is needed before this claim can be accepted.
minor comments (5)
  1. [§3.3, Assumption 1.iv] The notation ∇(ℓval_t ∘ ∇1h*)(θ) is ambiguous: it could mean the total gradient with respect to θ or the partial gradient with respect to the first argument of the composition. The proof of Proposition 3 uses the latter interpretation in (22). Please clarify the notation consistently.
  2. [Figure 2 caption] Typo: 'on5-class' should be 'on 5-class'.
  3. [Appendix G] The hyperparameters β1 and β2 in the numerical setups collide with the meta-learning rates β1^r, β2^r in Algorithm 1. Consider different notation to avoid confusion.
  4. [Appendix E] Typo: 'distrbutes' should be 'distributes'.
  5. [Table 5] The MAML baseline row uses 'baseline' for time but does not give the numeric value; please make the relative normalization explicit.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; convergence is a derivation from stated assumptions, self-citations are not load-bearing, and benchmarks are external.

full rationale

Walking the claimed derivation chain: Theorem 6 and Corollary 7 are established from Assumptions 1–3 via Propositions 3–5. Proposition 3 derives Lipschitz smoothness of the meta-loss from Assumption 1.iv and Assumption 3; this is a standard composition bound, not a restatement of the convergence theorem. Assumption 1.iv is an explicit hypothesis, not a conclusion derived from the NN parameterization; whether it is actually guaranteed by Theorem 1 is a soundness/validity question, not a circularity. The smoothness estimator (12) is an unbiased estimator of the constant GL,j defined in Proposition 3, not a fitted parameter renamed as a prediction. The learned DGF is optimized on validation tasks and evaluated on held-out test tasks; no test-set quantity is used as an input. Self-citations [31], [17], [18] are contextual (extension of prior conference work, related algorithms) and are not load-bearing in the proof of Theorem 6; the proof relies on external results such as [55], [56], and [29]. The grid-search treatment of Lipschitz constants in Remark 4 is a practical implementation gap, but it does not make the theoretical derivation circular. Therefore no circular step can be exhibited.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper does not postulate new physical entities. Its central claim rests on three strong Lipschitz-smoothness assumptions (Assumptions 1-3) that are not verified for the proposed NN parameterization, and on grid-searched constants used to instantiate the theory in practice.

free parameters (3)
  • Lipschitz/smoothness constants (G_l, G_lh, G_h, H_l, H_h) = grid-searched
    Used in the smoothness estimator (12) and in convergence-rate constants; Remark 4 states they are hyperparameters obtained through grid search.
  • Task and meta step sizes (alpha, beta_1, beta_2) = alpha=1e-2 or 2; beta_1=1e-3 or 1e-4; beta_2=1e-4 or 1e-5
    Chosen by greedy grid search on validation tasks (Appendix G); the convergence proof requires specific schedules, but experiments use constant rates.
  • NN architecture hyperparameters (layers I, hidden neurons, Kronecker-factor sizes) = I=1, 2, or 3 depending on dataset
    Chosen per dataset; affect the expressive capacity of h* and hence the learned mirror map.
assumptions (4)
  • domain assumption Assumption 1: losses have Lipschitz-continuous gradient and Hessian, and the composition nabla(ell_val o nabla_1 h^*) is Lipschitz wrt theta
    Central to Proposition 3 and Theorem 6; Assumption 1.iv is strong and not shown to hold for the neural h* in general.
  • domain assumption Assumption 2: bounded variance of validation gradients
    Standard for SGD-based analysis; used in the stochastic convergence arguments.
  • domain assumption Assumption 3: h* has Lipschitz-continuous partial gradient and mixed Jacobian
    Used throughout the proof (Lemmas 9-13); not guaranteed by Theorem 1 alone.
  • standard math Theorem 1 conditions: activations convex, Lipschitz, non-decreasing and Lipschitz-smooth; Wi non-negative bounded; Mi bounded
    Parameterization constraints that ensure h* is convex and Lipschitz-smooth; satisfied by ELU/Softplus with sigmoid/tanh weights.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learnable Loss Geometries with Mirror Descent for Scalable and Convergent Meta-Learning." pith.science (2026). https://pith.science/paper/IS6LUFRV

@misc{pith2026250902418,
  author       = {Pith},
  title        = {Pith review of: Learnable Loss Geometries with Mirror Descent for Scalable and Convergent Meta-Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IS6LUFRV}},
  note         = {Machine review of arXiv:2509.02418}
}
abstract

Utilizing task-invariant knowledge acquired from related tasks as prior information, meta-learning offers a principled approach to learning a new task with limited data records. Sample-efficient adaptation of this prior information is a major challenge facing meta-learning, and plays an important role because it facilitates training the sought task-specific model with just a few optimization steps. Past works deal with this challenge through preconditioning that speeds up convergence of the per-task training. Though effective in representing locally quadratic loss curvatures, simple linear preconditioning can be hardly potent with complex loss geometries. Instead of relying on a quadratic distance metric, the present contribution copes with complex loss metrics by learning a versatile distance-generating function, which induces a nonlinear mirror map to effectively capture and optimize a wide range of loss geometries. With suitable parameterization, this generating function is effected by an expressive neural network that is provably a valid distance. Analytical results establish convergence of not only the proposed method, but also all meta-learning approaches based on preconditioning. To attain gradient norm less than $\epsilon$, the convergence rate of $\mathcal{O}(\epsilon^{-2})$ is on par with standard gradient-based meta-learning methods. Numerical tests on few-shot learning datasets demonstrate the superior empirical performance of the novel algorithm, as well as its rapid per-task convergence, which markedly reduces the number of adaptation steps, hence also accommodating large-scale meta-learning models.

Figures

Figures reproduced from arXiv: 2509.02418 by the authors.

Figure 1
Figure 1. Illustration of (a) (P)GD, and (b) DGF. The blue dashed line and [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Change of meta-testing accuracy on 5-class 1-shot miniImageNet with step k. All models are meta-trained and meta-tested with K = 5. can be still captured by the learned loss geometries prior, thus facilitating the learning of cross-domain tasks. 4.4 Enhanced performance via accelerated adaptation Having confirmed the empirical performance gain of MetaMiDA on popular few-shot classification datasets, the next test an… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

58 extracted references · 53 canonical work pages

  1. [21]

    Meta mirror descent: Optimiser learning for fast convergence,

    B. Gao, H. Gouk, H. B. Lee, and T. M. Hospedales, “Meta mirror descent: Optimiser learning for fast convergence,” arXiv preprint arXiv:2203.02711, 2022

    arXiv 2022

  2. [28]

    On the convergence the- ory of gradient-based model-agnostic meta-learning algorithms,

    A. Fallah, A. Mokhtari, and A. Ozdaglar, “On the convergence the- ory of gradient-based model-agnostic meta-learning algorithms,” in Proc. Int. Conf. Artif. Intel. and Stats. , vol. 108, 26–28 Aug 2020, pp. 1082–1092

    work page 2020

  3. [30]

    Convergence of meta- learning with task-specific adaptation over partial parameters,

    K. Ji, J. D. Lee, Y. Liang, and H. V . Poor, “Convergence of meta- learning with task-specific adaptation over partial parameters,” in Proc. Adv. Neural Info. Processing Syst. , vol. 33, 2020, pp. 11 490– 11 500

    work page 2020

  4. [1]

    Imagenet classi- fication with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classi- fication with deep convolutional neural networks,” in Proc. Adv. Neural Info. Processing Syst., vol. 25, 2012

    work page 2012

  5. [2]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. Adv. Neural Info. Processing Syst., vol. 30, 2017

    work page 2017

  6. [3]

    Deep residual learn. for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learn. for image recognition,” in Proc. Conf. Computer Vis. and Pat. Recog., June 2016

    work page 2016

  7. [4]

    Dataset of breast ultrasound images,

    W. Al-Dhabyani, M. Gomaa, H. Khaled, and A. Fahmy, “Dataset of breast ultrasound images,” Data in Brief, vol. 28, p. 104863, 2020

    work page 2020

  8. [5]

    Optimization as a model for few-shot learning,

    S. Ravi and H. Larochelle, “Optimization as a model for few-shot learning,” in Proc. Int. Conf. Learn. Repr., 2017

    work page 2017

Show all 58 references
  1. [6]

    Low data drug discovery with one-shot learning,

    H. Altae-Tran, B. Ramsundar, A. S. Pappu, and V . Pande, “Low data drug discovery with one-shot learning,” ACS Central Science, vol. 3, no. 4, pp. 283–293, 2017

    2017

  2. [7]

    Meta-learning for low-resource neural machine translation,

    J. Gu, Y. Wang, Y. Chen, K. Cho, and V . O. Li, “Meta-learning for low-resource neural machine translation,” arXiv preprint arXiv:1808.08437, 2018

    2018 arXiv

  3. [8]

    Targeted meta-learning for critical incident detection in weather data,

    M. M. Kamani, S. Farhang, M. Mahdavi, and J. Z. Wang, “Targeted meta-learning for critical incident detection in weather data,” in ICML 2019 Workshop on Climate Change: How Can AI Help? , 2019

    2019

  4. [9]

    Model-agnostic meta-learning for fast adaptation of deep networks,

    C. Finn, P . Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in Proc. Int. Conf. Machine Learn., vol. 70, 2017, pp. 1126–1135

    2017

  5. [10]

    Meta-learning with memory-augmented neural networks,

    A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, and T. Lillicrap, “Meta-learning with memory-augmented neural networks,” in Proc. Int. Conf. Machine Learn. , vol. 48, 20–22 Jun 2016, pp. 1842– 1850

    2016

  6. [11]

    A simple neural attentive meta-learner,

    N. Mishra, M. Rohaninejad, X. Chen, and P . Abbeel, “A simple neural attentive meta-learner,” in Proc. Int. Conf. Learn. Repr., 2018

    2018

  7. [12]

    Recasting gradient-based meta-learning as hierarchical Bayes,

    E. Grant, C. Finn, S. Levine, T. Darrell, and T. Griffiths, “Recasting gradient-based meta-learning as hierarchical Bayes,” in Proc. Int. Conf. Learn. Repr., 2018

    2018

  8. [13]

    Meta- learning with implicit gradients,

    A. Rajeswaran, C. Finn, S. M. Kakade, and S. Levine, “Meta- learning with implicit gradients,” in Proc. Adv. Neural Info. Pro- cessing Syst., vol. 32, 2019

    2019

  9. [14]

    Meta- learning with differentiable closed-form solvers,

    L. Bertinetto, J. F. Henriques, P . Torr, and A. Vedaldi, “Meta- learning with differentiable closed-form solvers,” inProc. Int. Conf. Learn. Repr., 2019

    2019

  10. [15]

    Meta-learning with differentiable convex optimization,

    K. Lee, S. Maji, A. Ravichandran, and S. Soatto, “Meta-learning with differentiable convex optimization,” in Proc. Conf. Computer Vis. and Pat. Recog., 2019

    2019

  11. [16]

    Bayesian model-agnostic meta-learning,

    J. Yoon, T. Kim, O. Dia, S. Kim, Y. Bengio, and S. Ahn, “Bayesian model-agnostic meta-learning,” in Proc. Adv. Neural Info. Processing Syst., vol. 31, 2018

    2018

  12. [17]

    Meta-learning priors using un- rolled proximal networks,

    Y. Zhang and G. B. Giannakis, “Meta-learning priors using un- rolled proximal networks,” in Proc. Int. Conf. Learn. Repr., 2024

    2024

  13. [18]

    Scalable bayesian meta-learning through generalized implicit gradients,

    Y. Zhang, B. Li, S. Gao, and G. B. Giannakis, “Scalable bayesian meta-learning through generalized implicit gradients,” in Proc. AAAI Conf. Artif. Intel., vol. 37(9), 2023, pp. 11 298–11 306

    2023

  14. [19]

    Nesterov, Introductory lectures on convex optimization: A basic course

    Y. Nesterov, Introductory lectures on convex optimization: A basic course. Springer Science & Business Media, 2013, vol. 87

    2013

  15. [20]

    Meta-sgd: Learning to learn quickly for few-shot learning,

    Z. Li, F. Zhou, F. Chen, and H. Li, “Meta-sgd: Learning to learn quickly for few-shot learning,” arXiv preprint arXiv:1707.09835 , 2017

    2017 arXiv

  16. [22]

    Meta-curvature,

    E. Park and J. B. Oliva, “Meta-curvature,” in Proc. Adv. Neural Info. Processing Syst., vol. 32, 2019

    2019

  17. [23]

    Meta-learning with warped gradient descent,

    S. Flennerhag, A. A. Rusu, R. Pascanu, F. Visin, H. Yin, and R. Hadsell, “Meta-learning with warped gradient descent,” in Proc. Int. Conf. Learn. Repr., 2020

    2020

  18. [24]

    Learning where to learn: Gradient sparsity in meta and continual learning,

    J. von Oswald, D. Zhao, S. Kobayashi, S. Schug, M. Caccia, N. Zuc- chet, and J. a. Sacramento, “Learning where to learn: Gradient sparsity in meta and continual learning,” in Proc. Adv. Neural Info. Processing Syst., vol. 34, 2021, pp. 5250–5263

    2021

  19. [25]

    When maml can adapt fast and how to assist when it cannot,

    S. M. R. Arnold, S. Iqbal, and F. Sha, “When maml can adapt fast and how to assist when it cannot,” in Proc. Int. Conf. Artif. Intel. and Stats., vol. 130, 2021, pp. 244–252

    2021

  20. [26]

    Meta-learning with a geometry-adaptive preconditioner,

    S. Kang, D. Hwang, M. Eo, T. Kim, and W. Rhee, “Meta-learning with a geometry-adaptive preconditioner,” in Proc. Conf. Computer Vis. and Pat. Recog., June 2023, pp. 16 080–16 090

    2023

  21. [27]

    A. S. Nemirovskij and D. B. Yudin, Problem complexity and method efficiency in optimization. Wiley-Interscience, 1983

    1983

  22. [29]

    Theoretical convergence of multi-step model-agnostic meta-learning,

    K. Ji, J. Yang, and Y. Liang, “Theoretical convergence of multi-step model-agnostic meta-learning,” J. Mach. Learn. Res., vol. 23, no. 29, pp. 1–41, 2022

    2022

  23. [31]

    Meta-learning with versatile loss geometries for fast adaptation using mirror descent,

    Y. Zhang, B. Li, and G. B. Giannakis, “Meta-learning with versatile loss geometries for fast adaptation using mirror descent,” in Proc. IEEE Int. Conf. Acoust., Speech, Sig. Process. , 2024, pp. 5220–5224

    2024

  24. [32]

    Approximation by superpositions of a sigmoidal function,

    G. Cybenko, “Approximation by superpositions of a sigmoidal function,” Mathematics of control, signals and systems , vol. 2, no. 4, pp. 303–314, 1989

    1989

  25. [33]

    Amortized bayesian meta-learning,

    S. Ravi and A. Beatson, “Amortized bayesian meta-learning,” in Proc. Int. Conf. Learn. Repr., 2019

    2019

  26. [34]

    Meta-learning with network pruning,

    H. Tian, B. Liu, X.-T. Yuan, and Q. Liu, “Meta-learning with network pruning,” in Proc. European Conf. Computer Vis., 2020, pp. 675–700

    2020

  27. [35]

    Nonlinear programming,

    D. P . Bertsekas, “Nonlinear programming,” J. Operational Research Society, vol. 48, no. 3, pp. 334–334, 1997

    1997

  28. [36]

    Gradient-based meta-learning with learned layerwise metric and subspace,

    Y. Lee and S. Choi, “Gradient-based meta-learning with learned layerwise metric and subspace,” in Proc. Int. Conf. Machine Learn. , vol. 80, 2018, pp. 2927–2936

    2018

  29. [37]

    On modulating the gradient for meta-learning,

    C. Simon, P . Koniusz, R. Nock, and M. Harandi, “On modulating the gradient for meta-learning,” in Proc. European Conf. Computer Vis., 2020, pp. 556–572

    2020

  30. [38]

    Meta- learning the learning trends shared across tasks,

    J. Rajasegaran, S. Khan, M. Hayat, F. Khan, and M. Shah, “Meta- learning the learning trends shared across tasks,” in 32nd British Machine Vis. Conf., 2021

    2021

  31. [39]

    Mirror descent and nonlinear pro- jected subgradient methods for convex optimization,

    A. Beck and M. Teboulle, “Mirror descent and nonlinear pro- jected subgradient methods for convex optimization,” Operations Research Letters, vol. 31, no. 3, pp. 167–175, 2003

    2003

  32. [40]

    Fast and accurate deep network learning by exponential linear units (elus),

    D.-A. Clevert, T. Unterthiner, and S. Hochreiter, “Fast and accurate deep network learning by exponential linear units (elus),” arXiv preprint arXiv:1511.07289, 2015

    2015 arXiv

  33. [41]

    Bilevel optimization: Convergence analysis and enhanced design,

    K. Ji, J. Yang, and Y. Liang, “Bilevel optimization: Convergence analysis and enhanced design,” in Proc. Int. Conf. Machine Learn. , vol. 139, 18–24 Jul 2021, pp. 4882–4892

    2021

  34. [42]

    Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems,

    T. Chen, Y. Sun, and W. Yin, “Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems,” in Proc. Adv. Neural Info. Processing Syst. , vol. 34, 2021, pp. 25 294– 25 307

    2021

  35. [43]

    A two-timescale stochastic algorithm framework for bilevel optimization: Com- plexity analysis and application to actor-critic,

    M. Hong, H.-T. Wai, Z. Wang, and Z. Yang, “A two-timescale stochastic algorithm framework for bilevel optimization: Com- plexity analysis and application to actor-critic,” SIAM Journal on Optimization, vol. 33, no. 1, pp. 147–180, 2023

    2023

  36. [44]

    Matching networks for one shot learning,

    O. Vinyals, C. Blundell, T. Lillicrap, k. kavukcuoglu, and D. Wier- stra, “Matching networks for one shot learning,” in Proc. Adv. Neural Info. Processing Syst., vol. 29, 2016

    2016

  37. [45]

    ImageNet Large Scale Visual Recognition Challenge,

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,” Int. J. Computer Vis., vol. 115, no. 3, pp. 211–252, 2015. IEEE TRANSACTIONS ON PATTE...

    2015

  38. [46]

    Meta-learning for semi- supervised few-shot classification,

    M. Ren, S. Ravi, E. Triantafillou, J. Snell, K. Swersky, J. B. Tenen- baum, H. Larochelle, and R. S. Zemel, “Meta-learning for semi- supervised few-shot classification,” in Proc. Int. Conf. Learn. Repr. , 2018

    2018

  39. [47]

    The Caltech-UCSD birds-200-2011 dataset,

    C. Wah, S. Branson, P . Welinder, P . Perona, and S. Belongie, “The Caltech-UCSD birds-200-2011 dataset,” California Institute of Technology, Tech. Rep. CNS-TR-2011-001, 2011

    2011

  40. [48]

    A closer look at few-shot classification,

    W.-Y. Chen, Y.-C. Liu, Z. Kira, Y.-C. F. Wang, and J.-B. Huang, “A closer look at few-shot classification,” in Proc. Int. Conf. Learn. Repr., 2019

    2019

  41. [49]

    3d object representa- tions for fine-grained categorization,

    J. Krause, M. Stark, J. Deng, and L. Fei-Fei, “3d object representa- tions for fine-grained categorization,” in Proc. Int. Conf. Computer Vis. Workshops, June 2013

    2013

  42. [50]

    Cross- domain few-shot classification via learned feature-wise transfor- mation,

    H.-Y. Tseng, H.-Y. Lee, J.-B. Huang, and M.-H. Yang, “Cross- domain few-shot classification via learned feature-wise transfor- mation,” in Proc. Int. Conf. Learn. Repr., 2020

    2020

  43. [51]

    Meta-learning with latent embedding opti- mization,

    A. A. Rusu, D. Rao, J. Sygnowski, O. Vinyals, R. Pascanu, S. Osin- dero, and R. Hadsell, “Meta-learning with latent embedding opti- mization,” in Proc. Int. Conf. Learn. Repr., 2019

    2019

  44. [52]

    Rapid learn- ing or feature reuse? towards understanding the effectiveness of MAML,

    A. Raghu, M. Raghu, S. Bengio, and O. Vinyals, “Rapid learn- ing or feature reuse? towards understanding the effectiveness of MAML,” in Proc. Int. Conf. Learn. Repr., 2020

    2020

  45. [53]

    Boil: Towards representation change for few-shot learning,

    J. Oh, H. Yoo, C. Kim, and S.-Y. Yun, “Boil: Towards representation change for few-shot learning,” in Proc. Int. Conf. Learn. Repr., 2021

    2021

  46. [54]

    On enforcing better conditioned meta-learning for rapid few-shot adaptation,

    M. Hiller, M. Harandi, and T. Drummond, “On enforcing better conditioned meta-learning for rapid few-shot adaptation,” in Proc. Adv. Neural Info. Processing Syst., vol. 35, 2022, pp. 4059–4071

    2022

  47. [55]

    Input convex neural networks,

    B. Amos, L. Xu, and J. Z. Kolter, “Input convex neural networks,” in Proc. Int. Conf. Machine Learn., vol. 70, 06–11 Aug 2017, pp. 146– 155

    2017

  48. [56]

    Bounds on reciprocal moments with applications and developments in stein estimation and post-stratification,

    D. A. Wooff, “Bounds on reciprocal moments with applications and developments in stein estimation and post-stratification,” Journal of the Royal Statistical Society: Series B , vol. 47, no. 2, pp. 362–371, 1985

    1985

  49. [57]

    Adam: A method for stochastic optimiza- tion,

    D. P . Kingma and J. Ba, “Adam: A method for stochastic optimiza- tion,” in Proc. Int. Conf. Learn. Repr., 2015. BIOGRAPHY SECTION Yilang Zhang received the B.S. degree in elec- trical engineering (data science and technology track) from Fudan University, Shanghai, China, in 2...

    2015

  50. [58]

    Taking expectation wrt θr leads to E η1∥∇1L(θr)∥2 2 ∥∇1L(θr)∥2 + η2 + E η3∥∇2L(θr)∥2 2 ∥∇1L(θr)∥2 + η4 ≤ E[L(θr) − L(θr+1)] + η5 B

    Next, applying the upper and lower bounds of Gr L,j and the lower bound of Gr L,1 Gr L,2 to (42) render EBr, ˆBr [L(θr+1)|θr] ≤ L(θr) − 2−γK Cβ CG,1 (1 − CB,2+B CB,1B )∥∇1L(θr)∥2 2 ∥∇1L(θr)∥2 + CL,2 + γ2K(2 − γK) Gℓ h CG,1 − 2−γK Cβ CG,2 (1 − B+2 CB,1B )∥∇2L(θr)∥2 2 ∥∇1L(θr)∥2...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.