The first survey on Attention Sink in Transformers structures the literature around fundamental utilization, mechanistic interpretation, and strategic mitigation.
arXiv preprint arXiv:2507.16018 (2025)
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
citation-role summary
background 3
citation-polarity summary
years
2026 3roles
background 3polarities
background 3representative citing papers
A V-sink/L-sink taxonomy plus a frozen-backbone, NTP-trained layer-wise key gate (LSG) improves LLaVA-1.5-7B by up to +1.55pp on MMStar and +3.08pp on CVBench.
Sink-Token-aware Pruning (SToP) uses a sink score to suppress attention-sink tokens during visual token pruning, improving fine-grained video understanding in Video LLMs at high pruning rates.
citing papers explorer
-
Attention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigation
The first survey on Attention Sink in Transformers structures the literature around fundamental utilization, mechanistic interpretation, and strategic mitigation.
-
When Sinks Help or Hurt: Unified Framework for Attention Sink in Large Vision-Language Models
A V-sink/L-sink taxonomy plus a frozen-backbone, NTP-trained layer-wise key gate (LSG) improves LLaVA-1.5-7B by up to +1.55pp on MMStar and +3.08pp on CVBench.
-
Sink-Token-Aware Pruning for Fine-Grained Video Understanding in Efficient Video LLMs
Sink-Token-aware Pruning (SToP) uses a sink score to suppress attention-sink tokens during visual token pruning, improving fine-grained video understanding in Video LLMs at high pruning rates.