The paper claims pre-trained models do best when token frequency distributions align with Zipf's law, and proposes the R2 of a log-log fit as the vocabulary-size selection criterion.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
REJECT 1representative citing papers
citing papers explorer
-
Pre-trained Models Perform the Best When Token Distributions Follow Zipf's Law
The paper claims pre-trained models do best when token frequency distributions align with Zipf's law, and proposes the R2 of a log-log fit as the vocabulary-size selection criterion.