Learned pooling of speech KV caches from an intermediate LLM layer compresses speech to text-level length while matching or exceeding the uncompressed baseline on ASR and entity recognition, with 1.49–2× decoding speedup.
Efficient streaming language models with attention sinks,
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
eess.AS 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Compress the Cache, Not the Speech Embedding: KV Compression for Efficient Speech LLMs
Learned pooling of speech KV caches from an intermediate LLM layer compresses speech to text-level length while matching or exceeding the uncompressed baseline on ASR and entity recognition, with 1.49–2× decoding speedup.