REVIEW 13 cited by
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
read the original abstract
An important research direction in machine learning has centered around developing meta-learning algorithms to tackle few-shot learning. An especially successful algorithm has been Model Agnostic Meta-Learning (MAML), a method that consists of two optimization loops, with the outer loop finding a meta-initialization, from which the inner loop can efficiently learn new tasks. Despite MAML's popularity, a fundamental open question remains -- is the effectiveness of MAML due to the meta-initialization being primed for rapid learning (large, efficient changes in the representations) or due to feature reuse, with the meta initialization already containing high quality features? We investigate this question, via ablation studies and analysis of the latent representations, finding that feature reuse is the dominant factor. This leads to the ANIL (Almost No Inner Loop) algorithm, a simplification of MAML where we remove the inner loop for all but the (task-specific) head of a MAML-trained network. ANIL matches MAML's performance on benchmark few-shot image classification and RL and offers computational improvements over MAML. We further study the precise contributions of the head and body of the network, showing that performance on the test tasks is entirely determined by the quality of the learned features, and we can remove even the head of the network (the NIL algorithm). We conclude with a discussion of the rapid learning vs feature reuse question for meta-learning algorithms more broadly.
Forward citations
Cited by 13 Pith papers
-
SpurAudio: A Benchmark for Studying Shortcut Learning in Few-Shot Audio Classification
SpurAudio benchmark shows state-of-the-art few-shot audio classifiers suffer large performance drops when background correlations are disrupted, even in large pretrained models.
-
Second-Order Bilevel Optimization with Accelerated Convergence Rates
Second-order bilevel methods achieve Õ(ε^{-1.5}) iteration complexity for second-order stationary points, faster than first-order approaches, with a lazy variant improving computational efficiency by √d.
-
Graph-Conditioned Meta-Optimizer for QAOA Parameter Generation on Multiple Problem Classes
A graph-conditioned meta-optimizer learns QAOA parameter trajectories from one problem class and transfers them to others, yielding better initializations than standard methods in an empirical study of 64 settings.
-
Binomial Gradient-Based Meta-Learning for Enhanced Meta-Gradient Estimation
BinomMAML uses a binomial expansion to estimate meta-gradients more accurately than prior approximations, with error bounds that improve on existing methods and decay super-exponentially under mild conditions.
-
Continual Few-shot Adaptation for Synthetic Fingerprint Detection
A continual few-shot adaptation method combining binary cross-entropy and supervised contrastive losses with replay achieves a good trade-off between fast adaptation to unseen synthetic fingerprint styles and retentio...
-
DUPLE: An Intelligent Cross-Deployment Recognition Framework for Fiber-Optic Perimeter Security under Scarce Target Labels
DUPLE, a dual-domain multi-prototype meta-learning framework with statistical guidance, improves cross-deployment fiber-optic event recognition on two private few-shot benchmarks.
-
From Trajectories to Instructions: Language-Conditioned Meta-Reinforcement Learning
A language-instruction embedding can replace the gradient-based inner loop of MAML, yielding competitive BabyAI performance with lower per-iteration wall-clock time.
-
Few-Shot Left Atrial Wall Segmentation in 3D LGE MRI via Meta-Learning
MAML with auxiliary cavity tasks and boundary-aware loss achieves better few-shot 3D left atrial wall segmentation than standard fine-tuning, reaching DSC 0.64 at 5 shots versus 0.52.
-
DUPLE: An Intelligent Cross-Deployment Recognition Framework for Fiber-Optic Perimeter Security under Scarce Target Labels
DUPLE adapts multi-prototype class representations using dual-domain features and sample-specific statistical guidance for more accurate cross-deployment DFOS activity recognition under label scarcity.
-
Few-Shot Left Atrial Wall Segmentation in 3D LGE MRI via Meta-Learning
MAML with auxiliary cavity tasks and boundary loss improves 5-shot LA wall segmentation over standard fine-tuning (DSC 0.54 vs 0.48) and nears fully supervised performance at 20 shots.
-
CCoMAML: Efficient Cattle Identification Using Cooperative Model-Agnostic Meta-Learning
CCoMAML, a Cooperative MAML variant with a CNN co-learner, reports strong few-shot cattle identification from muzzle images, but its test-set-tuned hyperparameters and best-split reporting weaken the result.
-
Unsupervised Multi-Attention Meta Transformer for Rotating Machinery Fault Diagnosis
MMT-FD combines time-frequency self-supervised alignment, multi-head attention, a Transformer encoder, and MAML meta-learning to reach around 93-99% fault-diagnosis accuracy with 1-10% labeled data in reported experiments.
-
Meta-Learning and Meta-Reinforcement Learning -- Tracing the Path towards DeepMind's Adaptive Agent
A survey provides a task-based formalization of meta-learning and meta-RL while chronicling algorithms that lead to DeepMind's Adaptive Agent.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.