REVIEW 31 cited by
Overcoming catastrophic forgetting in neural networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence. Neural networks are not, in general, capable of this and it has been widely thought that catastrophic forgetting is an inevitable feature of connectionist models. We show that it is possible to overcome this limitation and train networks that can maintain expertise on tasks which they have not experienced for a long time. Our approach remembers old tasks by selectively slowing down learning on the weights important for those tasks. We demonstrate our approach is scalable and effective by solving a set of classification tasks based on the MNIST hand written digit dataset and by learning several Atari 2600 games sequentially.
Forward citations
Cited by 31 Pith papers
-
Online Continual Learning with Maximally Interfered Retrieval
Selecting replay samples by estimated loss increase after a virtual update improves online continual learning performance over random replay.
-
An Agentic Generative Large Language Model for Treatment Planning of Colorectal Cancer
A domain-adapted 8B LLM with agentic guideline retrieval matched UF Health oncologists' ratings on correctness, currency, and safety for colorectal cancer treatment plans in a 79-case blinded evaluation.
-
In-Context Collapse in Vision-Language Models and How to Mitigate it?
Many-shot in-context learning in vision-language models can collapse accuracy as demonstrations accumulate, and the failure is causally localized to the vision-language integration pathway, where a small adapter repairs it.
-
TOOD: Task-Aware Out-of-Distribution Score Calibration for Continual Learners
OOD detection in continual learning degrades through task-dependent logit-scale drift and feature-space crowding; a post-hoc per-task energy calibration recovers most of that loss for energy-based detectors.
-
LoRA-Loop: Closing the Synthetic Replay Cycle for Continual VLM Learning
Adapting a text-to-image generator with task-specific LoRA adapters and filtering samples by the model's own confidence improves synthetic replay in continual vision-language learning.
-
Temporal Information Retrieval via Time-Specifier Model Merging
TSM trains one retriever per time specifier and merges them by parameter averaging, improving temporal retrieval while maintaining non-temporal retrieval.
-
Universal Music Representations? Evaluating Foundation Models on World Music Corpora
Five audio foundation models are evaluated across six Western and non-Western music corpora, showing a consistent Western-centric bias and only limited generalization to culturally distant traditions.
-
Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Unlearning Completeness
IAM interpolates between an original model and a shadow model to score each sample's unlearning completeness, achieving top AUC for exact unlearning and top correlation for approximate unlearning, and exposing under- ...
-
OOD Detection with immature Models
Partially trained GLOW models match or outperform fully trained models for out-of-distribution image detection when scored by layer-wise gradient norms.
-
Chained Tuning Leads to Biased Forgetting
Fine-tuning a safety-tuned LLM on a capability task erases safety behavior more than the reverse order, and this forgetting is worse for specific groups such as Muslim people in the authors' tests.
-
TinySubNets: An efficient and low capacity continual learning strategy
TinySubNets combines per-layer pruning, adaptive quantization, and KL-gated weight sharing to run continual image classification with much lower memory than PackNet, WSN, and Ada-QPacknet.
-
MyTimeMachine: Personalized Facial Age Transformation
A personalized facial age transformation method that uses an adapter network on top of the SAM global aging model, trained with 10 to 50 photos of one person, to produce re-aged images that resemble that person's actu...
-
Improved Cardinality Estimation by Learning Queries Containment Rates
Learned containment rates between query pairs, combined with a queries pool of known cardinalities, substantially improve cardinality estimates on multi-join queries.
-
Interleaved Multitask Learning for Audio Source Separation with Independent Databases
An interleaved multitask training procedure for a shared-encoder source separation network enables training on independent per-source databases and yields SIR improvements over simultaneous multitask training.
-
Developing LLM-based Multi-Agent Systems in Software Engineering: A Mixed-Method Experience Report
A mixed-method experience report on LLM-based multi-agent frameworks for software engineering: broad feature coverage, weak monitoring support, and no clear quality winner on a README-summarization task, with incomple...
-
Memoir: Should a Model Write to Its Memory While It Thinks?
Writing to fast memory during pondering slows associative-recall learning at a fixed budget, but does not reduce final performance once training is long enough.
-
Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0
In a two-phase Terminal-Bench evaluation, only regression-aware RELAI-VCL compounded optimization gains, reaching the highest pass rate at every stage.
-
Preserving Fairness and Safety in Quantized LLMs Through Critical Weight Protection
Quantization tends to degrade LLM fairness and safety—more in non-English tasks—and preserving top sensitivity-ranked weights in FP16 mostly mitigates the loss.
-
When Less is More: 8-bit Quantization Improves Continual Learning in Large Language Models
Quantized (INT8/INT4) LLMs can outperform FP16 in later-task forward accuracy and retention during continual learning, though single-seed runs leave the effect unquantified.
-
Remember Past, Anticipate Future: Learning Continual Multimodal Misinformation Detectors
A continual multimodal misinformation detector that uses Dirichlet process-based expert expansion to curb forgetting and a neural-ODE dynamics model to anticipate evolving fake-news distributions.
-
CultureMERT: Continual Pre-Training for Cross-Cultural Music Representation Learning
CultureMERT adapts the MERT-95M music model to Greek, Turkish and Indian traditions via two-stage continual pre-training, improving non-Western auto-tagging by 4.9% on average while keeping Western performance intact.
-
What is the role of memorization in Continual Learning?
High-memorization training examples are forgotten fastest in class-incremental learning, and a cheap proxy based on learning iteration can guide buffer policies, favoring typical samples for small buffers and memorize...
-
Chain-of-Model Learning for Language Model
A nested Transformer with causally ordered hidden chains offers multiple sub-model sizes, chain-based expansion, and KV-cache sharing for faster prefilling.
-
Online Reward-Weighted Fine-Tuning of Flow Matching with Wasserstein Regularization
A reward-weighted flow matching loss, regularized by a Wasserstein-2 vector-field penalty, fine-tunes flow generators online with a controllable reward-diversity trade-off.
-
Scaling Sequential Recommendation Models with Transformers
Transformer-based sequential recommenders exhibit power-law and saturating NDCG scaling with model size and training interactions, enabling compute-aware model selection and effective pre-train/fine-tune transfer.
-
MTSpark: Enabling Multi-Task Learning with Spiking Neural Networks for Generalist Agents
MTSpark combines active dendrites and dueling in a deep spiking Q-network, reporting strong multi-task RL and classification scores, but the experimental setup may not fairly test the claimed continual-learning benefit.
-
Targeting Negative Flips in Active Learning using Validation Sets
RoSE, a validation-set-based filter that restricts active learning acquisition functions to estimated negative flips, improves accuracy and/or reduces negative flip rates on several image benchmarks.
-
The Future of Continual Learning in the Era of Foundation Models: Three Key Directions
Continual learning should pivot from weight-update-based methods to continual compositionality and orchestration of foundation models and agents.
-
Brain-Inspired Quantum Neural Architectures for Pattern Recognition: Integrating QSNN and QLSTM
A hybrid QSNN and QLSTM model, trained in three phases, is claimed to beat classical and quantum baselines on imbalanced credit card fraud data, but the benchmark lacks matched controls and statistical rigor.
-
Mobile Robots through Task-Based Human Instructions using Incremental Curriculum Learning
A simulated mobile robot learns multi-step household instructions better when training is staged from short sub-goals to full instructions, but the supporting experiments lack quantitative comparison.
-
Energy-Aware Deep Learning on Resource-Constrained Hardware
A survey of energy-aware deep learning methods for resource-constrained devices, covering energy-aware design, adaptive inference, on-device training, and scheduling on energy-harvesting systems.
Discussion (0). Continue with ORCID to comment.