Training tiny language models on GPT-4-simplified, low-complexity text improves their scores on simplified GLUE tasks and instruction-following, compared with training on the original larger corpus.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
TinyHelen's First Curriculum: Training and Evaluating Tiny Language Models in a Simpler Language Environment
Training tiny language models on GPT-4-simplified, low-complexity text improves their scores on simplified GLUE tasks and instruction-following, compared with training on the original larger corpus.