Faster query-key learning relative to output-value learning sharpens attention onto task-relevant tokens at comparable prediction performance, derived from gradient-flow dynamics and shown on synthetic and real tasks.
Birth of a transformer: a memory viewpoint
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Faster Query-Key Learning Sharpens Attention in Self-Attention Models
Faster query-key learning relative to output-value learning sharpens attention onto task-relevant tokens at comparable prediction performance, derived from gradient-flow dynamics and shown on synthetic and real tasks.