AdvCL repurposes adversarial perturbations into geometric control signals for continual learning using Intra-Smooth, Proto-Clip, and Inter-Align modules, reporting gains in performance, robustness, lower forgetting, and stronger transfer.
Orthogonal Subspace Learning for Language Model Continual Learning
5 Pith papers cite this work, alongside 44 external citations. Polarity classification is still indexing.
5
Pith papers citing it
44
external citations · OpenAlex
years
2026 5representative citing papers
PROXYMIX learns a dynamic replay controller on a small proxy model and transfers it to a large target model, improving accuracy by 3.4 points and reducing forgetting by 3.5 points on LLaMA-3-8B continual tuning sequences.
OP-Mix is an on-policy data mixing method that uses low-rank adapter interpolation to find near-optimal data mixtures throughout language model training with reduced compute.
SLICE applies gradient surgery via projection and truncated SVD to initialize LoRA adapters, yielding better stability-plasticity trade-offs on continual learning benchmarks including adversarial task sequences.
TailLoR applies low-rank updates to the singular value matrix of pre-trained weights while using a soft spectral penalty to protect dominant singular directions during continual learning.
citing papers explorer
-
Repurposing Adversarial Perturbations for Continual Learning: From Defense to Active Alignment
AdvCL repurposes adversarial perturbations into geometric control signals for continual learning using Intra-Smooth, Proto-Clip, and Inter-Align modules, reporting gains in performance, robustness, lower forgetting, and stronger transfer.
-
Dynamic Proxy-Mixing: Transferring Replay Controllers from Small to Large Models for Continual Instruction Tuning
PROXYMIX learns a dynamic replay controller on a small proxy model and transfers it to a large target model, improving accuracy by 3.4 points and reducing forgetting by 3.5 points on LLaMA-3-8B continual tuning sequences.
-
Always Learning, Always Mixing: Efficient and Simple Data Mixing All The Time
OP-Mix is an on-policy data mixing method that uses low-rank adapter interpolation to find near-optimal data mixtures throughout language model training with reduced compute.
-
Low-Rank Adapters Initialization via Gradient Surgery for Continual Learning
SLICE applies gradient surgery via projection and truncated SVD to initialize LoRA adapters, yielding better stability-plasticity trade-offs on continual learning benchmarks including adversarial task sequences.
-
TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning
TailLoR applies low-rank updates to the singular value matrix of pre-trained weights while using a soft spectral penalty to protect dominant singular directions during continual learning.