A long-short misalignment metric quantifies output distribution drift across context lengths, correlates with long-context performance, and a regularizer based on it improves fine-tuned LLMs.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Long-Short Alignment for Effective Long-Context Modeling in LLMs
A long-short misalignment metric quantifies output distribution drift across context lengths, correlates with long-context performance, and a regularizer based on it improves fine-tuned LLMs.