Massive activations in hybrid linear attention LLMs form pre-attention spikes before full attention layers and inter-spike plateaus that grow with full attention density.
Summer is warm. Winter is cold
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus
Massive activations in hybrid linear attention LLMs form pre-attention spikes before full attention layers and inter-spike plateaus that grow with full attention density.