HyperDn is a configuration-conditioned predictor that transfers oracle supervision across denoising paradigms to achieve near-oracle hyperparameter prediction with few or zero target labels.
hub
Reptile: a scalable metalearning algorithm
25 Pith papers cite this work. Polarity classification is still indexing.
25
Pith papers citing it
abstract
This paper considers meta-learning problems, where there is a distribution of tasks, and we would like to obtain an agent that performs well (i.e., learns quickly) when presented with a previously unseen task sampled from this distribution. We analyze a family of algorithms for learning a parameter initialization that can be fine-tuned quickly on a new task, using only first-order derivatives for the meta-learning updates. This family includes and generalizes first-order MAML, an approximation to MAML obtained by ignoring second-order derivatives. It also includes Reptile, a new algorithm that we introduce here, which works by repeatedly sampling a task, training on it, and moving the initialization towards the trained weights on that task. We expand on the results from Finn et al. showing that first-order meta-learning algorithms perform well on some well-established benchmarks for few-shot classification, and we provide theoretical analysis aimed at understanding why these algorithms work.
hub tools
citation-role summary
background 2
method 1
citation-polarity summary
representative citing papers
ConceptSeg-R1 uses Meta-GRPO meta-RL to learn transferable rules from visual demonstrations and apply them via concept translation for generalized concept segmentation across CI, CD, and CR levels.
Lang2MLIP is an LLM multi-agent framework that automates end-to-end development of machine learning interatomic potentials from natural language input for heterogeneous materials systems.
ClawForge is a generator framework that creates reproducible executable benchmarks for command-line agents under state conflict, with ClawForge-Bench showing frontier models reach at most 45.3% strict accuracy and that state inspection drives most performance gaps.
MemDLM embeds a simulated denoising trajectory into DLM training via bi-level optimization, creating a parametric memory that improves convergence and long-context performance even when the memory is dropped at test time.
Scion is a new stochastic LMO-based optimizer family that unifies existing methods, supports unconstrained problems, and delivers hyperparameter transferability plus speedups on nanoGPT training.
Classical RNNs trained on small instances provide parameter initializations for QAOA and VQE that reduce total optimization iterations and generalize across problem sizes.
Meta-IFWI pretrains a SIREN implicit neural network via meta-learning across velocity models to achieve faster convergence, higher accuracy, and better generalization than standard implicit full waveform inversion.
AHC applies meta-learned hierarchical compression with dual memory banks to enable continual object detection on MCUs under a 100KB budget, backed by a forgetting bound of O(ε√T + 1/√M) and competitive results on CORe50, TiROD, and PASCAL VOC.
MePo refines pretrained backbones via meta-learning on constructed pseudo tasks and initializes a meta covariance matrix to enable robust second-order alignment, yielding 12-15% gains on CIFAR-100, ImageNet-R and CUB-200 in rehearsal-free GCL settings.
MCRMO-Attack raises universal targeted attack success rates on unseen images by 23.7% on GPT-4o and 19.9% on Gemini-2.0 over prior universal baselines through stabilized supervision and meta-optimization.
SpidR-Adapt uses meta-learning with a first-order bi-level optimization heuristic to adapt speech representations to new languages with less than 1 hour of data, achieving 100x better efficiency than standard training.
A penalty-based bi-level optimization framework for machine unlearning that decorrelates forget and retention gradients via inner maximization and restores utility via outer minimization, with convergence guarantees and improved trade-offs on vision and language benchmarks.
Titans combine attention for current context with a learnable neural memory for long-term history, achieving better performance and scaling to over 2M-token contexts on language, reasoning, genomics, and time-series tasks.
FocuSFT uses an inner optimization loop to adapt fast-weight parameters into a parametric memory that sharpens attention on relevant content, then conditions outer-loop supervised fine-tuning on this representation, yielding gains on long-context benchmarks.
HARMONY mitigates representation skew in heterogeneous hybrid split federated learning via meta-learning to simulate diverse extractors and server-side contrastive learning to align features, delivering up to 43% accuracy gains.
RFPrompt adapts the Large Wireless Model via deep prompt tokens to improve out-of-distribution robustness in modulation classification while training only a small number of parameters.
BinomMAML uses a binomial expansion to estimate meta-gradients more accurately than prior approximations, with error bounds that improve on existing methods and decay super-exponentially under mild conditions.
A meta-optimized in-context learning approach enables training-free cross-subject semantic visual decoding from fMRI by inferring individual neural encoding patterns via hierarchical inference on a few examples.
Modulation-based meta-learning in a Hamiltonian framework enables accurate few-shot adaptation and generalization across parameter space for structure-preserving dynamics without explicit system parameterization.
A single multilingual NMT model for 103 languages trained on 25B examples demonstrates transfer learning benefits for low-resource languages.
LGD reaches Bayes optimality at optimal hyperparameters and admits an O(dh) pseudo-dimension bound for meta-learning hyperparameters on convex regression tasks.
Proposes Artificial Adaptive Intelligence as the regime between narrow and general AI, defined by elimination of human-specified hyperparameters, and introduces an adaptivity index plus parametric minimality principle grounded in minimum description length.
Placing one Hebbian fast-weight module after the final stage of Swin-Tiny achieves 96.2% accuracy on 5-way 1-shot Omniglot classification, outperforming the non-Hebbian baseline by 0.3 points.
citing papers explorer
-
Oracle Supervision Transfers for Hyperparameter Prediction in Model-Based Image Denoising
HyperDn is a configuration-conditioned predictor that transfers oracle supervision across denoising paradigms to achieve near-oracle hyperparameter prediction with few or zero target labels.
-
ConceptSeg-R1: Segment Any Concept via Meta-Reinforcement Learning
ConceptSeg-R1 uses Meta-GRPO meta-RL to learn transferable rules from visual demonstrations and apply them via concept translation for generalized concept segmentation across CI, CD, and CR levels.
-
Lang2MLIP: End-to-End Language-to-Machine Learning Interatomic Potential Development with Autonomous Agentic Workflows
Lang2MLIP is an LLM multi-agent framework that automates end-to-end development of machine learning interatomic potentials from natural language input for heterogeneous materials systems.
-
ClawForge: Generating Executable Interactive Benchmarks for Command-Line Agents
ClawForge is a generator framework that creates reproducible executable benchmarks for command-line agents under state conflict, with ClawForge-Bench showing frontier models reach at most 45.3% strict accuracy and that state inspection drives most performance gaps.
-
MemDLM: Memory-Enhanced DLM Training
MemDLM embeds a simulated denoising trajectory into DLM training via bi-level optimization, creating a parametric memory that improves convergence and long-context performance even when the memory is dropped at test time.
-
Training Deep Learning Models with Norm-Constrained LMOs
Scion is a new stochastic LMO-based optimizer family that unifies existing methods, supports unconstrained problems, and delivers hyperparameter transferability plus speedups on nanoGPT training.
-
Learning to learn with quantum neural networks via classical neural networks
Classical RNNs trained on small instances provide parameter initializations for QAOA and VQE that reduce total optimization iterations and generalize across problem sizes.
-
Meta-learning-enhanced implicit full waveform inversion
Meta-IFWI pretrains a SIREN implicit neural network via meta-learning across velocity models to achieve faster convergence, higher accuracy, and better generalization than standard implicit full waveform inversion.
-
AHC: Meta-Learned Adaptive Compression for Continual Object Detection on Memory-Constrained Microcontrollers
AHC applies meta-learned hierarchical compression with dual memory banks to enable continual object detection on MCUs under a 100KB budget, backed by a forgetting bound of O(ε√T + 1/√M) and competitive results on CORe50, TiROD, and PASCAL VOC.
-
MePo: Meta Post-Refinement for Rehearsal-Free General Continual Learning
MePo refines pretrained backbones via meta-learning on constructed pseudo tasks and initializes a meta covariance matrix to enable robust second-order alignment, yielding 12-15% gains on CIFAR-100, ImageNet-R and CUB-200 in rehearsal-free GCL settings.
-
Universal Adversarial Attacks against Closed-Source MLLMs via Target-View Routed Meta Optimization
MCRMO-Attack raises universal targeted attack success rates on unseen images by 23.7% on GPT-4o and 19.9% on Gemini-2.0 over prior universal baselines through stabilized supervision and meta-optimization.
-
SpidR-Adapt: A Universal Speech Representation Model for Few-Shot Adaptation
SpidR-Adapt uses meta-learning with a first-order bi-level optimization heuristic to adapt speech representations to new languages with less than 1 hour of data, achieving 100x better efficiency than standard training.
-
OFMU: Optimization-Driven Framework for Machine Unlearning
A penalty-based bi-level optimization framework for machine unlearning that decorrelates forget and retention gradients via inner maximization and restores utility via outer minimization, with convergence guarantees and improved trade-offs on vision and language benchmarks.
-
Titans: Learning to Memorize at Test Time
Titans combine attention for current context with a learnable neural memory for long-term history, achieving better performance and scaling to over 2M-token contexts on language, reasoning, genomics, and time-series tasks.
-
FocuSFT: Bilevel Optimization for Dilution-Aware Long-Context Fine-Tuning
FocuSFT uses an inner optimization loop to adapt fast-weight parameters into a parametric memory that sharpens attention on relevant content, then conditions outer-loop supervised fine-tuning on this representation, yielding gains on long-context benchmarks.
-
HARMONY: Bridging the Personalization-Generalization Gap by Mitigating Representation Skew in Heterogeneous Split Federated Learning
HARMONY mitigates representation skew in heterogeneous hybrid split federated learning via meta-learning to simulate diverse extractors and server-side contrastive learning to align features, delivering up to 43% accuracy gains.
-
RFPrompt: Prompt-Based Expert Adaptation of the Large Wireless Model for Modulation Classification
RFPrompt adapts the Large Wireless Model via deep prompt tokens to improve out-of-distribution robustness in modulation classification while training only a small number of parameters.
-
Binomial Gradient-Based Meta-Learning for Enhanced Meta-Gradient Estimation
BinomMAML uses a binomial expansion to estimate meta-gradients more accurately than prior approximations, with error bounds that improve on existing methods and decay super-exponentially under mild conditions.
-
Meta-learning In-Context Enables Training-Free Cross Subject Brain Decoding
A meta-optimized in-context learning approach enables training-free cross-subject semantic visual decoding from fMRI by inferring individual neural encoding patterns via hierarchical inference on a few examples.
-
Meta-learning Structure-Preserving Dynamics
Modulation-based meta-learning in a Hamiltonian framework enables accurate few-shot adaptation and generalization across parameter space for structure-preserving dynamics without explicit system parameterization.
-
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges
A single multilingual NMT model for 103 languages trained on 25B examples demonstrates transfer learning benefits for low-resource languages.
-
Generalization Guarantees on Data-Driven Tuning of Gradient Descent with Langevin Updates
LGD reaches Bayes optimality at optimal hyperparameters and admits an O(dh) pseudo-dimension bound for meta-learning hyperparameters on convex regression tasks.
-
Artificial Adaptive Intelligence: The Missing Stage Between Narrow and General Intelligence
Proposes Artificial Adaptive Intelligence as the regime between narrow and general AI, defined by elimination of human-specified hyperparameters, and introduces an adaptivity index plus parametric minimality principle grounded in minimum description length.
-
Where to Bind Matters: Hebbian Fast Weights in Vision Transformers for Few-Shot Character Recognition
Placing one Hebbian fast-weight module after the final stage of Swin-Tiny achieves 96.2% accuracy on 5-way 1-shot Omniglot classification, outperforming the non-Hebbian baseline by 0.3 points.
- ForeSplat: Optimization-Aware Foresight for Feed-Forward 3D Gaussian Splatting