Pith. sign in

REVIEW 3 major objections 5 minor 55 references

Learning to Learn Weight Generation via Local Consistency Diffusion

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A diffusion denoiser trained with a local consistency loss generates task weights without gradients, hitting intermediate optimizer checkpoints on schedule and ending at the global optimum, with higher accuracy and lower latency.

desk verdict A promising empirical recipe for diffusion-based weight generation whose central consistency theorem, as stated, isn't proved by the appendix — worth engaging but not ready as-is. read the letter →

arxiv 2502.01117 v3 pith:RCLTN7CH submitted 2025-02-03 cs.LG cs.AIcs.CV

Yunchuan Guan , Yu Liu , Ke Zhou , Zhiqi Shen , Jenq-Neng Hwang , Lei Li This is my paper · ORCID
classification cs.LGcs.AIcs.CV
keywords diffusionmodelsweightgenerationmeta-learninglocalconsistencyfew-shotlearningtransferdomaingeneralizationLoRAadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Diffusion models that generate neural-network weights are attractive because they replace gradient fine-tuning with a single sampling run, but existing methods generalize poorly across tasks and only model the final optimal weights. This paper argues that the fix is to treat the optimizer's intermediate checkpoints as local targets and train the denoiser with a local consistency loss. The proposed method, Mc-Di, wraps this diffusion model in a bi-level meta-learning update so it learns across tasks instead of per task. The key theoretical claim is that one inference chain can reach every local target at evenly spaced intervals and still finish at the global optimum, which is what makes the generated weights accurate and the inference fast. If correct, Mc-Di gives a gradient-free recipe for transfer learning, few-shot learning, domain generalization, and large language model adaptation.

What carries the argument

The load-bearing object is the local consistency loss $L^{\mathrm{loc}} = \mathbb{E}_{i\in(0,k]} L_i^{\mathrm{loc}}$, with $$$L_i^{{\mathrm{loc}}$} = \mathbb{E}_{t\in[0,iT/k)} \big\|\sqrt{1-\bar\alpha_t^i}\,\epsilon_\$\varphi$(x_t,t) - \sqrt{1-\bar\alpha_t}\,\epsilon\big\|^2, \quad x_t = \sqrt{\bar\alpha_t^i}\,\theta_{i\times d} + \sqrt{1-\bar\alpha_t^i}\,\epsilon,$$ where $\bar\alpha_t^i = \prod_{j=t}^{iT/k-1}\alpha_j$ and $\bar\alpha_t = \prod_{j=t}^{T-1}\alpha_j$. The loss makes the denoiser predict, within each segment, the noise that maps the next state back to the local target, and Theorem 1 says the shared chain then hits $\theta_{i\times d}$ at step $iT/k$. Supporting machinery: a weight-preparation stage that runs Adam and samples $\{\theta_d,\ldots,\theta_{kd}\}$ at uniform spacing $d=M/k$; a bi-level meta-learning inner loop that conditions the denoiser on a task embedding; and sharpness-aware minimization plus data augmentation moved into weight preparation so they improve the Theorem 2 bound without extra meta-training time.

What would settle it

Run the trained Mc-Di sampler with the deployed inference rule $x_{t+1} = \frac{1}{\sqrt{\bar\alpha_{t+1}}}(x_t - \sqrt{1-\bar\alpha_{t+1}}\epsilon_\phi(x_t,t))$ and record $\|x_{iT/k} - \theta_{i\times d}\|^2$ for $i=1,\ldots,k$. Theorem 1 predicts these distances go to zero at every checkpoint; if the final weight is accurate but the checkpoints do not track $\theta_{i\times d}$, the local-consistency mechanism is not what produces the reported gains. An algebraic check of whether this recursion equals the proof's update $x_{t+1} = \frac{1}{\sqrt{\alpha_t}}x_t - \frac{1-\alpha_t}{\sqrt{1-\bar\alpha_t}\sqrt{\alpha_t}}\epsilon_\phi$ would settle the mismatch without new experiments.

Watch

Extended reading notes

Core claim

The central claim is that a denoiser $\epsilon_\phi$ trained with the local consistency loss $L^{\mathrm{loc}}$ from Eq. (3) recovers the target sequence $\theta_d,\theta_{2d},\ldots,\theta_{M=kd}$ from standard Gaussian noise under the vanilla diffusion inference rule of Eq. (2), reaching $\theta_{i\times d}$ after exactly $iT/k$ steps and $\theta_M$ after $T$ steps. The proof derives the loss from the DDPM denoising matching objective, using Bayes' rule and the reparameterization trick to compare the posterior expectation with the denoiser's prediction. Because the intermediate targets are placed on the same inference chain rather than trained as separate generation tasks, the local targets reinforce the global optimum instead of conflicting with it. The paper also proves a cumulative-error bound, $L_D(\hat\theta)-L_D(\theta^*) \le \frac{\lambda}{2}(c + \frac{2\psi}{\mu}(1-\mu/l)^M)$, and moves sharpness-aware minimization into weight preparation to shrink $\lambda$. At $k=1$ the method reduces to vanilla diffusion, so Mc-Di is presented as the generalization of prior diffusion weight-generation methods.

Load-bearing premise

The entire local-consistency argument depends on the update rule used in the proof being algebraically the same as the update rule the model runs at inference; the paper writes them differently, so if they are not equivalent, the promised on-schedule hitting of local targets is not proved.

Editorial extensions

If this is right

  • Bi-level meta-learning makes diffusion weight generation transferable across tasks: after meta-training, a single denoiser can generate weights for unseen tasks without any gradient updates, which is what distinguishes Mc-Di from single-level diffusion methods like OCD and D2NWG.
  • Local targets add supervision without breaking global consistency, so the same inference chain reaches intermediate checkpoints and the final global optimum; the ablation attributes the accuracy gain to this consistency rather than to the extra targets alone.
  • Decomposing a $T$-step generation problem into $k$ shorter subproblems reduces the practical time to reach a given reconstruction error, so Mc-Di keeps accuracy with fewer diffusion steps.
  • Moving SAM and data augmentation into weight preparation improves the convergence bound through a smaller Hessian curvature term without adding time during meta-training.
  • Generated LoRA matrices reproduce gradient-based fine-tuning accuracy at roughly one quarter of the wall-clock latency, so Mc-Di offers a multi-task fine-tuning mode for LLMs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same local-consistency objective could be tested on other optimizer trajectories (e.g., SGD with momentum or AdamW) and on non-uniform target spacings, which would show whether the gains come from the trajectory itself or from the particular checkpoints chosen.
  • A direct comparison with a curriculum that trains each segment independently and stitches the chain at inference would isolate the value of the shared-chain consistency term; the paper's theory predicts the shared version should win at equal compute.
  • The limitation section points toward arbitrary sub-trajectories $\theta_i \to \theta_j$ and names Flow Matching and trajectory-balance objectives as the next step; if the local-consistency idea carries over, weight generation could eventually start from any previous weight rather than from Gaussian noise.
  • If the guarantee holds on larger models, weight generation for foundation-model adaptation could become a single forward pass per task, changing how multi-task serving is scheduled.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Mc-Di, a meta-learning framework that uses a conditional diffusion model to generate neural-network weights for downstream tasks. The method combines REPTILE-style bi-level optimization with a diffusion denoiser, and its main novelty is 'local consistency diffusion': the denoiser is trained with a local consistency loss that supervises intermediate weight targets sampled uniformly from an optimization trajectory, with the aim of recovering the sequence θ_d, θ_2d, ..., θ_M at evenly spaced intervals of T/k diffusion steps. The paper gives Theorem 1 to justify this property, Theorem 2 to analyze convergence and motivate a SAM-based improvement, and experiments on transfer learning, few-shot learning, domain generalization, and LLM fine-tuning. The empirical results are positive and the paper is ambitious, but the theoretical guarantee as stated is not established because the proof in Appendix A uses a different denoising update than the one declared in the theorem and in Section 2.3.

Significance. If the local-consistency property were rigorously established, the paper would make a useful contribution to gradient-free weight generation: exploiting intermediate optimizer states could improve both generation accuracy and inference latency, and the meta-learning wrapper addresses cross-task transfer. The empirical study is broad, covering four downstream scenarios with consistent gains over several baselines, and the paper also attempts a convergence analysis. However, the central theoretical claim is load-bearing and currently unsupported: the inference rule stated in Eq. (2) is not the update analyzed in the proof, and the concatenation of local segments is asserted rather than proven. The SAM-improvement claim also rests on an unverified assumption about the Hessian spectrum. These issues do not necessarily invalidate the empirical observations, but they do mean the paper's main intellectual contribution is not yet demonstrated.

major comments (3)
  1. [Section 2.3 and Appendix A] Theorem 1 is not proved for the inference rule it states. Main-text Eq. (2) and Appendix A Eq. (1) define the update as x_{t+1} = (1/sqrt(alphabar_{t+1}))(x_t - sqrt(1-alphabar_{t+1}) eps_phi), but the proof after Appendix A Eq. (10) substitutes the standard DDPM update x_{t+1} = (1/sqrt(alpha_t))x_t - ((1-alpha_t)/(sqrt(1-alphabar_t)sqrt(alpha_t))) eps_phi, with alphabar_t = product_{j=t}^{T-1} alpha_j. These are not algebraically equivalent. Moreover, the stated update is not a valid reverse process for the forward model q(x_t|x_T) = N(sqrt(alphabar_t)x_T, (1-alphabar_t)I): substituting a perfect denoiser eps_phi = eps and x_t = sqrt(alphabar_t)theta + sqrt(1-alphabar_t)eps does not yield sqrt(alphabar_{t+1})theta + sqrt(1-alphabar_{t+1})eps, and at t = T-1 (where alphabar_T = 1) the update gives x_T = x_{T-1}, so the final step performs no denoising. Thus the guarantee that local targets are reached on schedule is unproven for the deployment rule described in the paper. The authors should either prove the theorem for the update actually used in implementation (the one in Appendix D.1) or correct the stated inference rule, and re-derive the local consistency loss accordingly.
  2. [Theorem 1 proof, Appendix A] The proof analyzes a single segment i in isolation and then asserts that training with the sum L_loc = sum_i L_loc^i makes the different segments share one inference chain. This concatenation is not demonstrated. For any t < T/k, the loss receives contributions from every i, each with a different schedule alphabar_t^i and a different target theta_{i*d}; the same network must satisfy all these conditions simultaneously at the states visited by the shared chain. The proof does not rule out interference between segments, nor does it show that the fixed-point of the combined objective yields the claimed 'pass through theta_d at T/k, then theta_2d at 2T/k, ...' behavior. A rigorous argument for the concatenation property is required for the central claim of the paper.
  3. [Theorem 2 and Section 3.2.3] The claimed convergence benefit of SAM is not established. Theorem 2 assumes an l-smooth, mu-strongly convex loss with Hessian eigenvalues bounded by lambda, and the bound involves lambda; the paper then states that SAM improves convergence by reducing lambda. However, no theorem or measurement in the paper shows that SAM reduces lambda in the setting of interest, and Figure 7 plots accuracy versus GPU hours rather than Hessian spectra. In addition, Lemma 1 analyzes gradient descent with step size 1/l, while the weight preparation stage in Algorithm 1 uses Adam. As written, the statement in Section 3.2.3 that 'the convergence of the weight generation paradigm can be improved by introducing SAM' is an empirical observation, not a consequence of the presented analysis. The authors should either provide a proof that SAM reduces the relevant lambda, or expressly label the SAM motivation as empirical.
minor comments (5)
  1. [Theorem 1 statement, Eq. (3)] The loss L_loc in Eq. (3) drops the positive coefficient (1-alpha_t)^2 / (alpha_t (1-alphabar_t^i)(1-alphabar_t)) that appears in the KL derivation in Eq. (11) of Appendix A. The proof only establishes proportionality, not equality; if the theorem's guarantee depends on the exact objective, this discrepancy should be addressed or the loss definition should be revised.
  2. [Section 2.3 and Appendix D.1] The inference update in Eq. (2) is described as 'aligns with the vanilla diffusion algorithm', but it does not match the vanilla update given in Appendix D.1. These two formulas should be reconciled, and the indexing convention for alphabar_t (including the empty product at t = T) should be stated explicitly.
  3. [Appendix C, proof of Theorem 2] The Taylor expansion in Eq. (14) evaluates the Hessian at a point xi on the segment between hat-theta and theta*, while the assumption bounds eigenvalues only 'around the optimum theta*'; the bound in Eq. (15) therefore needs an additional uniformity assumption on the Hessian along that segment.
  4. [References] Several references are incomplete or malformed, e.g., '[23] et al. Jang', '[37] et al. Madan', '[41] et al. Pan', and '[51] et al. Zhang'; these should be completed with full author lists.
  5. [Figures 4 and 5] The axis labels in Figures 4 and 5 appear garbled in the submitted PDF (the text is rendered as unicode fragments). Please ensure that all figures use readable, standard labels.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor definitional circularity: Theorem 1's schedule guarantee restates the construction of L_loc; no fitted-input circularity and no load-bearing self-citation.

  1. self definitional [Appendix A, Theorem 1 (Eqs. 1-2); main text Section 3.1, Eq. (3)]
    "Then, the denoiser ϵϕ can recover the target sequence {θd, ..., θk∗d=M } from standard Gaussian noise x0 with evenly T /ksteps intervals, when ϵϕ is trained by local consistency loss Lloc as follows: Lloc = E i∈(0,k] Lloc_i, Lloc_i = E t∈[0,i∗T /k) || q 1 − ¯αi_tϵϕ(xt, t) − √ 1 − ¯αtϵ||^2, xt = q ¯αi_tθi∗d + q 1 − ¯αi_tϵ."

    The loss Lloc_i is defined by taking a forward noising process that is anchored at θi∗d and runs for exactly i∗T /k steps. The theorem's conclusion is that the denoiser recovers θi∗d after i∗T /k steps, which is exactly the training objective's construction, not an independent derivation. Any denoiser that minimizes the denoising matching term for that anchored forward process recovers the anchored target by construction. The claimed T /k spacing between consecutive targets likewise follows from placing successive targets at successive endpoints i∗T /k of the loss definitions.

full rationale

The paper's main theoretical claim, Theorem 1, is a derivation of a training objective: L_loc_i is the denoising matching loss for a forward process anchored at each local target θi∗d at time i∗T /k, so the statement that the perfect denoiser reaches θi∗d after i∗T /k steps is true by construction and is not an externally validated prediction. This is a minor self-definitional element, but it is not a fitted-input-called-prediction: no model parameter is fit to a subset of the target data and then reported as a prediction of that same data. The hyperparameters k and T are tuned in sensitivity experiments on validation data, but they do not enter the theoretical claims as fitted constants. The paper also contains no load-bearing self-citation: the authors' own prior work [17] is used only for an experimental setup, and no uniqueness or consistency theorem is imported from the authors' earlier papers. The empirical contributions are benchmarked against independent baselines on transfer learning, few-shot learning, domain generalization, and LLM adaptation, so the central performance claims are not circular. Separately, I note a non-circularity correctness concern: the Appendix A proof uses the standard DDPM update xt+1 = (1/sqrt(alpha_t))xt - ((1-alpha_t)/(sqrt(1-alphabar_t)sqrt(alpha_t)))epsilon_phi, while the stated Theorem 1 and Section 2.3 Eq. (2) use xt+1 = (1/sqrt(alphabar_{t+1}))(xt - sqrt(1-alphabar_{t+1})epsilon_phi); these updates are not algebraically equivalent. That is a proof gap affecting whether the theorem covers the deployed inference rule, but it does not make the derivation circular, so it does not raise the circularity score.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

All five free parameters affect the local consistency loss or the meta-training dynamics. k and T directly define the schedule of local targets; d sets which checkpoints are used as targets; rho and the learning rates shape the weight preparation and inner-loop updates. None is derived from first principles; all are chosen by experiment. No physical constants are involved.

free parameters (5)
  • k (segment number) = 3
    Chosen as default after the MSE vs. segment-number trade-off in Figure 4. The paper says: 'Considering the stable trade-off across multiple datasets, we set k = 3 as the default'. The value controls how many local targets are modeled.
  • T (diffusion steps) = 20
    Set to 20 for all experiments; Appendix F.1 shows accuracy is stable across values, but the choice is a design decision, not derived from first principles.
  • d (local-target sampling interval) = M/k
    Local targets are uniformly sampled from the optimizer trajectory at interval d = M/k; M is set by an unspecified automatic early-stopping rule, so d is data-dependent.
  • SAM perturbation radius rho = not reported
    Algorithm 1 requires rho for the Sharpness-Aware Minimization step, but the value is never given, hampering reproduction.
  • Inner-loop and meta-learning rates = eta=0.005, zeta=0.001
    Used in Algorithm 2 and stated in Section 4.2.1 without a documented tuning procedure.
assumptions (5)
  • standard math The denoising matching term is equivalent to maximizing the ELBO for diffusion models
    Invoked in Appendix A to derive L_loc from the KL divergence; standard DDPM result [20,34].
  • domain assumption The downstream task loss is l-smooth and mu-strongly convex around the optimum theta*
    Assumed in Theorem 2 and Lemma 1 (Appendix C); unrealistic for deep networks. The paper acknowledges the assumption but does not test its validity.
  • domain assumption The generative model reconstruction error is bounded by c and the downstream loss is bounded by psi
    Assumed in Theorem 2; no empirical estimate of c or psi is provided.
  • domain assumption Uniformly sampled intermediate checkpoints theta_{i*d} are informative diffusion targets
    The whole local-consistency design assumes the optimizer trajectory carries transferable policy information; the paper supports this with Figure 1 but does not prove or quantify it.
  • ad hoc to paper SAM applied in weight preparation reduces the maximum Hessian eigenvalue lambda of the downstream loss
    The only support is the empirical accuracy-vs-time curve in Figure 7; no theorem connects SAM to lambda in this weight-generation setting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning to Learn Weight Generation via Local Consistency Diffusion." pith.science (2026). https://pith.science/paper/RCLTN7CH

@misc{pith2026250201117,
  author       = {Pith},
  title        = {Pith review of: Learning to Learn Weight Generation via Local Consistency Diffusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RCLTN7CH}},
  note         = {Machine review of arXiv:2502.01117}
}
read the original abstract

Diffusion-based algorithms have emerged as promising techniques for weight generation. However, existing solutions are limited by two challenges: generalizability and local target assignment. The former arises from the inherent lack of cross-task transferability in existing single-level optimization methods, limiting the model's performance on new tasks. The latter lies in existing research modeling only global optimal weights, neglecting the supervision signals in local target weights. Moreover, naively assigning local target weights causes local-global inconsistency. To address these issues, we propose Mc-Di, which integrates the diffusion algorithm with meta-learning for better generalizability. Furthermore, we extend the vanilla diffusion into a local consistency diffusion algorithm. Our theory and experiments demonstrate that it can learn from local targets while maintaining consistency with the global optima. We validate Mc-Di's superior accuracy and inference efficiency in tasks that require frequent weight updates, including transfer learning, few-shot learning, domain generalization, and large language model adaptation.

Figures

Figures reproduced from arXiv: 2502.01117 by the authors.

Figure 1
Figure 1. Visualization of inference chains in Om [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Workflow of Mc-Di. In the weight-preparation stage, a real-world optimizer (Adam) is used to [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Naively assigning local targets creates inconsistency. Directly using vanilla diffusion loss in Equation 1 for meta-training creates inconsistency between local and global targets. The top of [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Segment Num￾ber vs. MSE trade-off on 5-way 1-shot and 5-shot tasks. 0 1000 2000 3000 4000 5000 Epoch 0.0 0.1 0.2 MSE L loc 3 L loc 2 L loc 1 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 7
Figure 7. Figure 7: Ablation of functional components. Accu [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Sensitive study on Omniglot Trajectory Length 1 2 3 4 5 6 7 Diffusion Step 15 16 171819202122232425 Accuracy 62 63 64 65 66 67 68 69 [PITH_FULL_IMAGE:figures/full_fig_p027_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 47 canonical work pages

  1. [1]

    Flow network based generative models for non-iterative diverse candidate generation

    Yoshua Bengio et al. Flow network based generative models for non-iterative diverse candidate generation. In Advances in Neural Information Processing Systems 34 (NeurIPS 2021) , 2021

    work page 2021

  2. [2]

    A closer look at the training strategy for modern meta-learning

    Jiaxin Chen, Xiao-Ming Wu, Yanke Li, Qimai Li, Li-Ming Zhan, and Fu-Lai Chung. A closer look at the training strategy for modern meta-learning. In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing S...

    work page 2020

  3. [3]

    Generalization bounds for meta-learning: An information-theoretic analysis

    Qi Chen, Changjian Shui, and Mario Marchand. Generalization bounds for meta-learning: An information-theoretic analysis. In Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors,Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurI...

    work page 2021

  4. [4]

    Meta-baseline: Exploring simple meta-learning for few-shot learning

    Yinbo Chen, Zhuang Liu, Huijuan Xu, Trevor Darrell, and Xiaolong Wang. Meta-baseline: Exploring simple meta-learning for few-shot learning. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021 , pages 9042–9051. IEEE, 2021

    work page 2021

  5. [5]

    Sophia Koepke, Ole Winther, and Zeynep Akata

    Anders Christensen, Massimiliano Mancini, A. Sophia Koepke, Ole Winther, and Zeynep Akata. Image-free classifier injection for zero-shot classification. InIEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023 , pages 19026–19035. IEEE, 2023

    work page 2023

  6. [6]

    An analysis of single-layer networks in unsupervised feature learning

    Adam Coates, Andrew Y Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS) , pages 215–223, 2011

    work page 2011

  7. [7]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 248–255. IEEE, 2009

    work page 2009

  8. [8]

    On stability and generalization of bilevel optimization problems

    Meng Ding, Mingxi Lei, Yunwen Lei, Di Wang, and Jinhui Xu. On stability and generalization of bilevel optimization problems. 2022

    work page 2022

Show all 55 references
  1. [9]

    Decaf: A deep convolutional activation feature for generic visual recognition

    Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. In Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, Chin...

    2014

  2. [10]

    Yingjun Du, Zehao Xiao, Shengcai Liao, and Cees G. M. Snoek. Protodiff: Learning to learn prototypical networks by task-guided diffusion. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023) , 2023

    2023

  3. [11]

    Ozdaglar

    Alireza Fallah, Aryan Mokhtari, and Asuman E. Ozdaglar. On the convergence theory of gradient-based model-agnostic meta-learning algorithms. In Silvia Chiappa and Roberto Calandra, editors, The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 20...

    2020

  4. [12]

    Model-agnostic meta-learning for fast adaptation of deep networks

    Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017 , volume 70, pages 1126–1135. PMLR, 2017

    2017

  5. [13]

    Bootstrapped meta-learning

    Sebastian Flennerhag, Yannick Schroecker, Tom Zahavy, Hado van Hasselt, David Silver, and Satinder Singh. Bootstrapped meta-learning. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview.net, 2022

    2022

  6. [14]

    Sharpness-aware mini- mization for efficiently improving generalization

    Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware mini- mization for efficiently improving generalization. In Proceedings of the International Conference on Learning Representations (ICLR) , 2022

    2022

  7. [15]

    Garrido-Merch´ an

    Roberto Gozalo-Brizuela and Eduardo C. Garrido-Merch´ an. A survey of generative AI applica- tions. CoRR, abs/2306.02781, 2023

    2023 arXiv

  8. [16]

    Task relatedness-based generalization bounds for meta learning

    Jiechao Guan and Zhiwu Lu. Task relatedness-based generalization bounds for meta learning. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview.net, 2022

    2022

  9. [17]

    Hierarchical meta-learning with hyper-tasks for few-shot learning

    Yunchuan Guan, Yu Liu, Ke Zhou, and Junyuan Huang. Hierarchical meta-learning with hyper-tasks for few-shot learning. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, CIKM 2023, Birmingham, United Kingdom, October 21-25, 2023 , p...

    2023

  10. [18]

    David Ha, Andrew Dai, and Quoc V. Le. Hypernetworks. In International Conference on Learning Representations (ICLR), 2017

    2017

  11. [19]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778. IEEE, 2016

    2016

  12. [20]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Inf...

    2020

  13. [21]

    Meta-learning in neural networks: A survey

    Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence , 44(9):5149–5169, 2021

    2021

  14. [22]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In Proceedings of the International Conference on Learning Representations (ICLR) , 2022

    2022

  15. [23]

    et al. Jang. Learning energy decompositions for partial inference of gflownets. In International Conference on Learning Representations (ICLR 2024) , 2024

    2024

  16. [24]

    An information-theoretic analysis of the impact of task similarity on meta-learning

    Sharu Theresa Jose and Osvaldo Simeone. An information-theoretic analysis of the impact of task similarity on meta-learning. In IEEE International Symposium on Information Theory, ISIT 2021, Melbourne, Australia, July 12-20, 2021 , pages 1534–1539. IEEE, 2021. 13

    2021

  17. [25]

    Information-theoretic generalization bounds for meta-learning and applications

    Sharu Theresa Jose and Osvaldo Simeone. Information-theoretic generalization bounds for meta-learning and applications. Entropy, 23(1):126, 2021

    2021

  18. [26]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR) , 2015

    2015

  19. [27]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations (ICLR), 2014. Presented at ICLR 2014 as a conference paper

    2014

  20. [28]

    Taylor, and Adriana Romero-Soriano

    Boris Knyazev, Michal Drozdzal, Graham W. Taylor, and Adriana Romero-Soriano. Parameter prediction for unseen deep architectures. In Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors,Advances in Neural Information Proc...

    2021

  21. [29]

    Boris Knyazev, Doha Hwang, and Simon Lacoste-Julien. Can we scale transformers to predict parameters of diverse imagenet models? In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, International Conference on Mach...

    2023

  22. [30]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009

    2009

  23. [31]

    Lake, Ruslan Salakhutdinov, Jason Gross, and Joshua B

    Brenden M. Lake, Ruslan Salakhutdinov, Jason Gross, and Joshua B. Tenenbaum. One shot learning of simple visual concepts. In Proceedings of the 33th Annual Meeting of the Cognitive Science Society, CogSci 2011, Boston, Massachusetts, USA, July 20-23, 2011 . cognitivescience- s...

    2011

  24. [32]

    Miller, and Mirek Riedewald

    Aristotelis Leventidis, Laura Di Rocco, Wolfgang Gatterbauer, Ren´ ee J. Miller, and Mirek Riedewald. Domainnet: Homograph detection for data lake disambiguation. In Proceedings of the 24th International Conference on Extending Database Technology, EDBT 2021, Nicosia, Cyprus, ...

    2021

  25. [33]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In International Conference on Learning Representations , 2023

    2023

  26. [34]

    Understanding diffusion models: A unified perspective

    Calvin Luo. Understanding diffusion models: A unified perspective. CoRR, abs/2208.11970, 2022

    2022 arXiv

  27. [35]

    OCD: learning to overfit with conditional diffusion models

    Shahar Lutati and Lior Wolf. OCD: learning to overfit with conditional diffusion models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, International Conference on Machine Learning, ICML 2023, 23-29 July 2023...

    2023

  28. [36]

    Fourier frequency tuning for parameter-efficient fine-tuning

    Shikun Ma, Chunyuan Zhou, Saining Xie, Xiangru Chen, Jingjing Liu, and Jianfeng Gao. Fourier frequency tuning for parameter-efficient fine-tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 3456–3467, 2023. 14

    2023

  29. [37]

    et al. Madan. Learning gflownets from partial episodes for improved convergence and stability. In Proceedings of the 40th International Conference on Machine Learning (ICML 2023) , 2023

    2023

  30. [38]

    Fine- grained visual classification of aircraft

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine- grained visual classification of aircraft. In arXiv preprint arXiv:1306.5151 , 2013

    2013 arXiv

  31. [39]

    On first-order meta-learning algorithms

    Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. volume abs/1803.02999, 2018

    2018 arXiv

  32. [40]

    Hyperseg: Patch-wise hypernetwork for real-time semantic segmentation

    Yuval Nirkin, Lior Wolf, and Tal Hassner. Hyperseg: Patch-wise hypernetwork for real-time semantic segmentation. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021 , pages 4061–4070. Computer Vision Foundation / IEEE, 2021

    2021

  33. [41]

    et al. Pan. Better training of gflownets with local credit and incomplete trajectories. In Proceedings of the 40th International Conference on Machine Learning (ICML 2023) , 2023

    2023

  34. [42]

    Cats and dogs

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. IEEE Conference on Computer Vision and Pattern Recognition , 2012

    2012

  35. [43]

    Early stopping-but when? In Neural Networks: Tricks of the trade , pages 55–69

    Lutz Prechelt. Early stopping-but when? In Neural Networks: Tricks of the trade , pages 55–69. Springer, 2002

    2002

  36. [44]

    Rapid learning or feature reuse? towards understanding the effectiveness of MAML

    Aniruddh Raghu, Maithra Raghu, Samy Bengio, and Oriol Vinyals. Rapid learning or feature reuse? towards understanding the effectiveness of MAML. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 . OpenRe- view.net, 2020

    2020

  37. [45]

    Tenenbaum, Hugo Larochelle, and Richard S

    Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B. Tenenbaum, Hugo Larochelle, and Richard S. Zemel. Meta-learning for semi-supervised few-shot classification. In International Conference on Learning Representations (ICLR) , 2018

    2018

  38. [46]

    Hyper- representations as generative models: Sampling unseen neural network weights

    Konstantin Sch¨ urholt, Boris Knyazev, Xavier Gir´ o-i Nieto, and Damian Borth. Hyper- representations as generative models: Sampling unseen neural network weights. In Advances in Neural Information Processing Systems (NeurIPS) , 2022

    2022

  39. [47]

    Diffusion-based neural network weights generation

    Bedionita Soro, Bruno Andreis, Hayeon Lee, Wonyong Jeong, Song Chong, Frank Hutter, and Sung Ju Hwang. Diffusion-based neural network weights generation. arXiv preprint arXiv:2402.18153, 2024

    2024 arXiv

  40. [48]

    Meta- dataset: A dataset of datasets for learning to learn from few examples

    Eleni Triantafillou, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Utku Evci, Kelvin Xu, Ross Goroshin, Carles Gelada, Kevin Swersky, Pierre-Antoine Manzagol, and Hugo Larochelle. Meta- dataset: A dataset of datasets for learning to learn from few examples. In 8th International...

    2020

  41. [49]

    Glue: A multi-task benchmark and analysis platform for natural language understanding

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. Proceedings of the International Conference on Learning Representations (ICLR) , 2019

    2019

  42. [50]

    Metadiff: Meta-learning with conditional diffusion for few-shot learning

    Baoquan Zhang, Chuyao Luo, Demin Yu, Xutao Li, Huiwei Lin, Yunming Ye, and Bowen Zhang. Metadiff: Meta-learning with conditional diffusion for few-shot learning. In Thirty-Eighth 15 AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Ap...

    2024

  43. [51]

    et al. Zhang. Unifying generative models with gflownets and beyond. In Advances in Neural Information Processing Systems 35 (NeurIPS 2022) , 2022

    2022

  44. [52]

    AdaLoRA: Towards efficient adaptive low-rank adaptation for large language models

    Renrui Zhang, Juntao Li, Yifan Xie, Xun Liu, Yiyang Zhao, Qiang Wang, and Ziwei Liu. AdaLoRA: Towards efficient adaptive low-rank adaptation for large language models. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS) , pages 5678–5689, 2023

    2023

  45. [53]

    Meta-learning via hypernetworks

    Dominic Zhao, Seijin Kobayashi, Jo˜ ao Sacramento, and Johannes von Oswald. Meta-learning via hypernetworks. In 4th Workshop on Meta-Learning at NeurIPS 2020 (MetaLearn 2020) . NeurIPS, 2020

    2020

  46. [54]

    DyLoRA: Parameter-efficient tuning of pre-trained models via dynamic low-rank adaptation

    Xinyue Zhuang, Yu Cheng, Zhe Gan, Jingjing Liu, Lichao Liu, Guangsen Chen, Haoxuan Zhang, Meng Wang, Shuohang Liu, and Jianfeng Gao. DyLoRA: Parameter-efficient tuning of pre-trained models via dynamic low-rank adaptation. In Proceedings of the Association for Computational Li...

    2023

  47. [55]

    Unraveling model-agnostic meta-learning via the adaptation learning rate

    Yingtian Zou, Fusheng Liu, and Qianxiao Li. Unraveling model-agnostic meta-learning via the adaptation learning rate. In International Conference on Learning Representations (ICLR) . OpenReview.net, 2022. 16 A Theorem 1 and Proof Readers can refer to the derivation process of ...

    2022

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.