Plasticity loss in GPT-style transformers on multilingual tasks persists from 5M to 314M parameters, follows a sublinear scaling law with model size, and occurs in both continual and stationary settings.
arXiv preprint arXiv:2402.02342 , year=
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
years
2026 2verdicts
UNVERDICTED 2representative citing papers
A generalization of differential TD extends it to episodic settings while preserving policy ordering, inheriting linear TD guarantees, and improving sample efficiency.
citing papers explorer
-
Can Scale Save Us From Plasticity Loss in Large Language Models?
Plasticity loss in GPT-style transformers on multilingual tasks persists from 5M to 314M parameters, follows a sublinear scaling law with model size, and occurs in both continual and stationary settings.
-
Extending Differential Temporal Difference Methods for Episodic Problems
A generalization of differential TD extends it to episodic settings while preserving policy ordering, inheriting linear TD guarantees, and improving sample efficiency.