High data redundancy and over-training decelerate LLM performance gains, and the authors fit a sub-optimal scaling law with logistic correction terms to predict the slowdown.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Sub-Scaling Laws: On the Role of Data Density and Training Strategies in LLMs
High data redundancy and over-training decelerate LLM performance gains, and the authors fit a sub-optimal scaling law with logistic correction terms to predict the slowdown.