User facts are internalized as surgical local edits to a hash-keyed Engram memory table with reasoning skill held in a shared adapter, claimed to match LoRA recall, improve indirect reasoning 5.6x on average, and compose across users with 33,000x smaller footprint than per-user adapters.
Continual learning of large language models: A comprehensive survey
10 Pith papers cite this work. Polarity classification is still indexing.
10
Pith papers citing it
representative citing papers
In the NTK regime, new-task training induces old-task prediction drift through the cross-task kernel, yielding an exact closed-form forgetting predictor under frozen linear heads and a low-rank concentration result.
Later-domain RL training harms earlier domains via second-order damage concentrated in a low-dimensional shared conflict subspace; brief domain refresh contracts this component to enable selective recovery.
Seven stance-making linguistic features strengthen pro-animal-welfare preference in fine-tuned Llama-3.2-1B and Mistral-7B; hedging and concreteness dilute it; first-person is null.
SEAT preserves epistemic abstention in LLMs during knowledge adaptation via sparse tuning and entity-perturbed KL regularization, yielding 18-101% better abstention on unknown queries while retaining near-perfect knowledge acquisition.
HippoRAG 2 improves on standard RAG and prior HippoRAG by adding deeper passage integration and more effective LLM use in Personalized PageRank, delivering superior performance on factual, sense-making, and associative memory tasks including a 7% gain in associative memory over state-of-the-art.
Scientific amnesia is observable in production-like continual DPO pipelines, with most tested strategy proposers degrading in peak performance and results depending sharply on chain regime, evaluator, and seed coverage.
MADS selects a 15% core set from the 52K Alpaca-GPT4 dataset via activations in Llama-3.2-3B-Instruct, yielding 2.5% average gains on 7B-13B models across six benchmarks versus full-data training.
LightEdit enables scalable lifelong knowledge editing in LLMs via selective knowledge retrieval and probability suppression during decoding, outperforming prior methods on ZSRE, Counterfact, and RIPE while reducing training costs.
Position paper claims multimodal LLMs can significantly advance scientific reasoning and proposes a four-stage roadmap plus challenges and suggestions.
citing papers explorer
-
User as Engram: Internalizing Per-User Memory as Local Parametric Edits
User facts are internalized as surgical local edits to a hash-keyed Engram memory table with reasoning skill held in a shared adapter, claimed to match LoRA recall, improve indirect reasoning 5.6x on average, and compose across users with 33,000x smaller footprint than per-user adapters.
-
Catastrophic Forgetting is Low-Rank: A Function-Space Theory for Continual Adaptation
In the NTK regime, new-task training induces old-task prediction drift through the cross-task kernel, yielding an exact closed-form forgetting predictor under frozen linear heads and a low-rank concentration result.
-
A Local Perturbation Theory for Cross-Domain Interference and Recovery in Multi-Domain RL
Later-domain RL training harms earlier domains via second-order damage concentrated in a low-dimensional shared conflict subspace; brief domain refresh contracts this component to enable selective recovery.
-
Assert, don't describe: Linguistic features that shift LLM reasoning about animal welfare
Seven stance-making linguistic features strengthen pro-animal-welfare preference in fine-tuned Llama-3.2-1B and Mistral-7B; hedging and concreteness dilute it; first-person is null.
-
SEAT: Sparse Entity-Aware Tuning for Knowledge Adaptation while Preserving Epistemic Abstention
SEAT preserves epistemic abstention in LLMs during knowledge adaptation via sparse tuning and entity-perturbed KL regularization, yielding 18-101% better abstention on unknown queries while retaining near-perfect knowledge acquisition.
-
From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
HippoRAG 2 improves on standard RAG and prior HippoRAG by adding deeper passage integration and more effective LLM use in Personalized PageRank, delivering superior performance on factual, sense-making, and associative memory tasks including a 7% gain in associative memory over state-of-the-art.
-
Repeated post-training is not Self-improving: Diagnosing Scientific Amnesia in Continual DPO Pipelines
Scientific amnesia is observable in production-like continual DPO pipelines, with most tested strategy proposers degrading in peak performance and results depending sharply on chain regime, evaluator, and seed coverage.
-
MADS: Model-Aware Diverse Core Set Selection for Instruction Tuning
MADS selects a 15% core set from the 52K Alpaca-GPT4 dataset via activations in Llama-3.2-3B-Instruct, yielding 2.5% average gains on 7B-13B models across six benchmarks versus full-data training.
-
Towards Scalable Lifelong Knowledge Editing with Selective Knowledge Suppression
LightEdit enables scalable lifelong knowledge editing in LLMs via selective knowledge retrieval and probability suppression during decoding, outperforming prior methods on ZSRE, Counterfact, and RIPE while reducing training costs.
-
Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning
Position paper claims multimodal LLMs can significantly advance scientific reasoning and proposes a four-stage roadmap plus challenges and suggestions.