A fixed 1,000-sample replay set from pretraining text, trained with a threshold-based margin loss on last-layer hidden states, reduces forgetting across 15 sequential finetuning tasks in Llama-3.1-8B.
Emergence of simple-cell recep- tive field properties by learning a sparse code for natural images,
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
GeRe: Towards Efficient Anti-Forgetting in Continual Learning of LLM via General Samples Replay
A fixed 1,000-sample replay set from pretraining text, trained with a threshold-based margin loss on last-layer hidden states, reduces forgetting across 15 sequential finetuning tasks in Llama-3.1-8B.