The authors synthesize long-context training data from short chunks and fine-tune Llama-3 models to reach 1M-token contexts with top open-source RULER scores.
Flashattention-2: Faster attention with better parallelism and work partitioning
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Bootstrap Your Own Context Length
The authors synthesize long-context training data from short chunks and fine-tune Llama-3 models to reach 1M-token contexts with top open-source RULER scores.