A hierarchical synthetic question-answer pipeline extended a Llama-3.1-8B model to a one-million-token context with strong RULER and InfiniteBench scores and small general-task regression.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Scaling Instruction-Tuned LLMs to Million-Token Contexts via Hierarchical Synthetic Data Generation
A hierarchical synthetic question-answer pipeline extended a Llama-3.1-8B model to a one-million-token context with strong RULER and InfiniteBench scores and small general-task regression.