HyperDn is a configuration-conditioned predictor that transfers oracle supervision across denoising paradigms to achieve near-oracle hyperparameter prediction with few or zero target labels.
hub
On First-Order Meta-Learning Algorithms
27 Pith papers cite this work. Polarity classification is still indexing.
27
Pith papers citing it
abstract
This paper considers meta-learning problems, where there is a distribution of tasks, and we would like to obtain an agent that performs well (i.e., learns quickly) when presented with a previously unseen task sampled from this distribution. We analyze a family of algorithms for learning a parameter initialization that can be fine-tuned quickly on a new task, using only first-order derivatives for the meta-learning updates. This family includes and generalizes first-order MAML, an approximation to MAML obtained by ignoring second-order derivatives. It also includes Reptile, a new algorithm that we introduce here, which works by repeatedly sampling a task, training on it, and moving the initialization towards the trained weights on that task. We expand on the results from Finn et al. showing that first-order meta-learning algorithms perform well on some well-established benchmarks for few-shot classification, and we provide theoretical analysis aimed at understanding why these algorithms work.
hub tools
citation-role summary
background 2
method 1
citation-polarity summary
representative citing papers
ConceptSeg-R1 uses Meta-GRPO meta-RL to learn transferable rules from visual demonstrations and apply them via concept translation for generalized concept segmentation across CI, CD, and CR levels.
Lang2MLIP is an LLM multi-agent framework that automates end-to-end development of machine learning interatomic potentials from natural language input for heterogeneous materials systems.
ClawForge is a generator framework that creates reproducible executable benchmarks for command-line agents under state conflict, with ClawForge-Bench showing frontier models reach at most 45.3% strict accuracy and that state inspection drives most performance gaps.
MemDLM embeds a simulated denoising trajectory into DLM training via bi-level optimization, creating a parametric memory that improves convergence and long-context performance even when the memory is dropped at test time.
Scion is a new stochastic LMO-based optimizer family that unifies existing methods, supports unconstrained problems, and delivers hyperparameter transferability plus speedups on nanoGPT training.
Classical RNNs trained on small instances provide parameter initializations for QAOA and VQE that reduce total optimization iterations and generalize across problem sizes.
Meta-IFWI pretrains a SIREN implicit neural network via meta-learning across velocity models to achieve faster convergence, higher accuracy, and better generalization than standard implicit full waveform inversion.
ForeSplat introduces MetaGrad, a meta-gradient training rule that makes feed-forward 3DGS predictions optimization-aware so they converge faster and to higher quality upon refinement.
AHC applies meta-learned hierarchical compression with dual memory banks to enable continual object detection on MCUs under a 100KB budget, backed by a forgetting bound of O(ε√T + 1/√M) and competitive results on CORe50, TiROD, and PASCAL VOC.
MePo refines pretrained backbones via meta-learning on constructed pseudo tasks and initializes a meta covariance matrix to enable robust second-order alignment, yielding 12-15% gains on CIFAR-100, ImageNet-R and CUB-200 in rehearsal-free GCL settings.
MCRMO-Attack raises universal targeted attack success rates on unseen images by 23.7% on GPT-4o and 19.9% on Gemini-2.0 over prior universal baselines through stabilized supervision and meta-optimization.
SpidR-Adapt uses meta-learning with a first-order bi-level optimization heuristic to adapt speech representations to new languages with less than 1 hour of data, achieving 100x better efficiency than standard training.
A penalty-based bi-level optimization framework for machine unlearning that decorrelates forget and retention gradients via inner maximization and restores utility via outer minimization, with convergence guarantees and improved trade-offs on vision and language benchmarks.
Titans combine attention for current context with a learnable neural memory for long-term history, achieving better performance and scaling to over 2M-token contexts on language, reasoning, genomics, and time-series tasks.
TAM aligns query video frames to novel class examples, averages per-frame distances along the path, and uses continuous relaxation for end-to-end few-shot optimization, yielding gains on Kinetics and Something-Something-V2.
FocuSFT uses an inner optimization loop to adapt fast-weight parameters into a parametric memory that sharpens attention on relevant content, then conditions outer-loop supervised fine-tuning on this representation, yielding gains on long-context benchmarks.
HARMONY mitigates representation skew in heterogeneous hybrid split federated learning via meta-learning to simulate diverse extractors and server-side contrastive learning to align features, delivering up to 43% accuracy gains.
RFPrompt adapts the Large Wireless Model via deep prompt tokens to improve out-of-distribution robustness in modulation classification while training only a small number of parameters.
BinomMAML uses a binomial expansion to estimate meta-gradients more accurately than prior approximations, with error bounds that improve on existing methods and decay super-exponentially under mild conditions.
A meta-optimized in-context learning approach enables training-free cross-subject semantic visual decoding from fMRI by inferring individual neural encoding patterns via hierarchical inference on a few examples.
Modulation-based meta-learning in a Hamiltonian framework enables accurate few-shot adaptation and generalization across parameter space for structure-preserving dynamics without explicit system parameterization.
A single multilingual NMT model for 103 languages trained on 25B examples demonstrates transfer learning benefits for low-resource languages.
LGD reaches Bayes optimality at optimal hyperparameters and admits an O(dh) pseudo-dimension bound for meta-learning hyperparameters on convex regression tasks.
citing papers explorer
-
FocuSFT: Bilevel Optimization for Dilution-Aware Long-Context Fine-Tuning
FocuSFT uses an inner optimization loop to adapt fast-weight parameters into a parametric memory that sharpens attention on relevant content, then conditions outer-loop supervised fine-tuning on this representation, yielding gains on long-context benchmarks.