IA-VAE augments amortized variational inference with hypernetwork-generated instance-adaptive modulations, strictly containing the standard variational family and improving held-out ELBO on synthetic and image data.
Continual learning with hypernet- works
8 Pith papers cite this work. Polarity classification is still indexing.
8
Pith papers citing it
abstract
Artificial neural networks suffer from catastrophic forgetting when they are sequentially trained on multiple tasks. To overcome this problem, we present a novel approach based on task-conditioned hypernetworks, i.e., networks that generate the weights of a target model based on task identity. Continual learning (CL) is less difficult for this class of models thanks to a simple key feature: instead of recalling the input-output relations of all previously seen data, task-conditioned hypernetworks only require rehearsing task-specific weight realizations, which can be maintained in memory using a simple regularizer. Besides achieving state-of-the-art performance on standard CL benchmarks, additional experiments on long task sequences reveal that task-conditioned hypernetworks display a very large capacity to retain previous memories. Notably, such long memory lifetimes are achieved in a compressive regime, when the number of trainable hypernetwork weights is comparable or smaller than target network size. We provide insight into the structure of low-dimensional task embedding spaces (the input space of the hypernetwork) and show that task-conditioned hypernetworks demonstrate transfer learning. Finally, forward information transfer is further supported by empirical results on a challenging CL benchmark based on the CIFAR-10/100 image datasets.
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
A learned local rule (Weight Transformer) iteratively self-organizes task-network weights from local graph neighborhoods and generalizes across unseen MLP, CNN, and ResNet architectures up to ~2M parameters.
Proposes feature splitting and a closed-form bound on extrapolation range to enable zero-shot topological out-of-domain generalization in dynamical systems reconstruction across tipping points.
RareCP improves interval efficiency for time series conformal prediction by retrieving and weighting regime-specific calibration examples while adapting to drift and maintaining coverage.
A frozen LSTM backbone with FiLM-based few-shot context adaptation estimates tip-level contact forces for deformable swabbing tools across nine surface-tool regimes using only wrist-mounted proprioception.
SOR-SNN employs Self-Organizing Regulation networks to reorganize a single SNN into sparse pathways, achieving better performance, energy efficiency, memory use, backward transfer, and self-repair on continual learning tasks including CIFAR100 and ImageNet.
Proposes MMOT, an optimal transport-based online mixture model with dynamic preservation strategy, for improved handling of multimodal data in online incremental learning.
citing papers explorer
-
Instance-Adaptive Parametrization for Amortized Variational Inference
IA-VAE augments amortized variational inference with hypernetwork-generated instance-adaptive modulations, strictly containing the standard variational family and improving held-out ELBO on synthetic and image data.
-
Architecture Generalization with MetaNCA
A learned local rule (Weight Transformer) iteratively self-organizes task-network weights from local graph neighborhoods and generalizes across unseen MLP, CNN, and ResNet architectures up to ~2M parameters.
-
Topological Out-of-Domain Generalization in Dynamical Systems Reconstruction
Proposes feature splitting and a closed-form bound on extrapolation range to enable zero-shot topological out-of-domain generalization in dynamical systems reconstruction across tipping points.
-
RareCP: Regime-Aware Retrieval for Efficient Conformal Prediction
RareCP improves interval efficiency for time series conformal prediction by retrieving and weighting regime-specific calibration examples while adapting to drift and maintaining coverage.
-
Context-Aware Force Estimation for Deformable Tool Manipulation in Robotic Environmental Swabbing via Few-Shot Continual Adaptation
A frozen LSTM backbone with FiLM-based few-shot context adaptation estimates tip-level contact forces for deformable swabbing tools across nine surface-tool regimes using only wrist-mounted proprioception.
-
Adaptive Reorganization of Neural Pathways for Continual Learning with Spiking Neural Networks
SOR-SNN employs Self-Organizing Regulation networks to reorganize a single SNN into sparse pathways, achieving better performance, energy efficiency, memory use, backward transfer, and self-repair on continual learning tasks including CIFAR100 and ImageNet.
-
An Optimal Transport-driven Approach for Cultivating Latent Space in Online Incremental Learning
Proposes MMOT, an optimal transport-based online mixture model with dynamic preservation strategy, for improved handling of multimodal data in online incremental learning.
- CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning