Overfitting pre-trained LLMs to near-zero training loss on a tiny dataset sharply improves open-ended greedy text generation, beating nucleus sampling on diversity and human preference.
Generative pretraining from pixels
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
The Hyperfitting Phenomenon: Sharpening and Stabilizing LLMs for Open-Ended Text Generation
Overfitting pre-trained LLMs to near-zero training loss on a tiny dataset sharply improves open-ended greedy text generation, beating nucleus sampling on diversity and human preference.