Pith. sign in

REVIEW 3 major objections 7 minor 36 references

VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference

T0 review · 3 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read VoxZip keeps 91% of audio-LLM accuracy while cutting KV cache 20x

desk verdict VoxZip is a genuinely new compression idea for speech LLMs, but its headline retention numbers don't yet isolate compression fidelity because the Full KV baseline never receives the ASR text that VoxZip injects. read the letter →

arxiv 2608.08569 v1 pith:I24L2MWN submitted 2026-08-09 cs.AI cs.SD

Wenxu Jia , Dongjie Fu , Xize Cheng , Fangming Feng , Linjun Li , Wenshi Chen , Yingming Li , Zhou Zhao
show 1 more author
Tao Jin
This is my paper · ORCID
classification cs.AIcs.SD
keywords KVcachecompressionspeechlargelanguagemodelssemanticanchorsASR-guidedattention-basedtokenevictiontemporaldecaylong-contextaudioinferencetrain-free
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

VoxZip is a training-free method that attacks the memory bottleneck of long-context speech language models by compressing the KV cache in two stages. Its core bet is that ASR transcripts are reliable semantic anchors: they provide timestamps that slice the audio, allowing redundant speech frames to be averaged down to the text-token level and fused with text embeddings, so the same meaning is kept at far higher information density. A second stage then prunes cached tokens using accumulated attention with temporal decay, counteracting the bias that favors early tokens. On the Qwen3-Omni backbone, the paper reports over 90% performance retention at a 20x compression ratio in long-context benchmarks, and at 4x compression it reports a 1.9x throughput gain with a 3.3x peak memory reduction. If true, aggressive audio KV cache compression can be achieved without retraining and without sacrificing paralinguistic cues such as emotion, speaker traits, or environmental sound.

What carries the argument

The central mechanism is the semantic anchor: an ASR text token fused with the temporally aligned acoustic segment it transcribes. The fusion is element-wise addition of a text embedding with the average-pooled audio features of the corresponding interval, so the anchor carries both lexical meaning and paralinguistic acoustics in one token. The second mechanism is a time-decayed accumulated attention score, $A^{(t)}=\gamma [A^{(t-1)},0]+\mathrm{softmax}(q_t K_t^\top/\sqrt{d})$, which scores each cached token by its historical contribution while discounting stale tokens, countering the early-token bias of plain attention accumulation. Together these mechanisms make attention maps denser and give the eviction policy a reliable salience signal.

What would settle it

Take a long recording with known timestamps, apply a controlled time-stretch so the linear mapping is off by a known factor, then run VoxZip; if a 10-20% temporal distortion drops the reported performance retention by more than a few points, the linear-alignment premise is disproved. Conversely, if retention is unchanged, the method is insensitive to exact alignment and the claim is safer than the paper argues.

Watch

Extended reading notes

Core claim

The paper claims that the KV cache of a speech LLM can be aggressively compressed without retraining by using ASR transcriptions as explicit semantic anchors. In the first stage, transcribed segments above a confidence threshold are temporally aligned to audio token indices via a linear mapping between timestamps and token positions; each speech interval is uniformly pooled into as many groups as the aligned text has tokens, then element-wise added to the text embeddings, while untranscribed background audio is kept intact. The resulting sequence is shorter (semantic intervals drop to about a quarter of their original length) and denser. The second stage continuously evicts low-information tokens during decoding using a per-layer, temporally decayed accumulated attention score, unconditionally keeping attention-sink and recent tokens. Across six benchmarks, VoxZip reports 104% performance retention at a 25% cache budget on long-context tasks, 96.24% at 10%, and 91.41% even at a 5% budget, while on general audio QA it stays at or above the uncompressed baseline through a 10% budget.

Load-bearing premise

The load-bearing premise is that ASR word timestamps map linearly onto audio token indices (Eq. 3); if the encoder's frame rate is not uniform or the timestamps are biased by noise or accent, text embeddings are added to the wrong acoustic content and the anchor is corrupted.

Editorial extensions

If this is right

  • At a 25% cache budget VoxZip can exceed the uncompressed baseline in average accuracy on long-context benchmarks, implying the compressed representation is not merely lossless but redistributes attention away from redundant audio.
  • At 20x compression (5% budget), long-context performance retention stays at 91.41%, so extreme pruning remains viable when semantic anchors guide eviction.
  • General audio QA remains at or above full-cache accuracy down to a 10% budget, indicating paralinguistic cues survive text-guided fusion.
  • At 4x compression, end-to-end throughput increases 1.93x and peak memory drops 3.34x on a 64K context, making long multi-turn dialogue feasible on 80GB GPUs.
  • The method is train-free, so it can be applied to an already deployed model as a plug-in without fine-tuning the backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the linear timestamp-to-index mapping in Eq. (3) is inaccurate for a different audio encoder, the semantic anchor would bind text to the wrong acoustic window; a calibration step or attention-based alignment would make VoxZip portable to other speech LLMs.
  • The reported gains over the uncompressed baseline suggest that for long, repetitive audio, a compressed, anchor-densified sequence may act as a form of denoising; one can test this by checking whether VoxZip's advantage grows with audio redundancy.
  • A learnable fusion replacing element-wise addition could capture cross-modal dynamics the paper acknowledges are simplistic; a natural extension is to compare pooling operators and additive versus gated fusion on paralinguistic-heavy subsets.
  • The temporal decay coefficient $\gamma=0.95$ is identified as optimal in the paper; making $\gamma$ adaptive to utterance length or token age instead of fixed could improve the extreme-budget regime.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. VoxZip proposes a training-free, two-stage KV cache compression method for speech LLMs. In Stage 1, an external ASR model provides transcripts whose timestamps are linearly mapped to audio token indices (Eq. 3); the audio embeddings for each transcript segment are average-pooled to the text-token length and element-wise added to the text embeddings (Eq. 4), with untranscribed background audio preserved. In Stage 2, a temporally decayed accumulated attention score (Eq. 6) is used to evict low-importance tokens while retaining attention sinks and recent tokens. The method is evaluated on Qwen3-Omni-30B (Instruct and Thinking) on three long-context benchmarks (Vox-Infinity, AudioMarathon, SPIRAL) and three general audio QA benchmarks (MMSU, MMAU, MMAR), reporting performance retention ratios (PRR) relative to a Full KV baseline and an efficiency analysis at a 25% cache budget.

Significance. If the reported results are valid, VoxZip would be a practical, training-free approach to reduce KV cache memory for long-context audio inference, with a 1.9x throughput gain and a 3.3x peak memory reduction at 4x compression while preserving accuracy. The two-stage design and the explicit use of ASR text as anchors are well-motivated and thoroughly ablated, and the authors are careful to include ASR overhead in the efficiency measurements. However, the central accuracy claim is confounded by the fact that the Full KV baseline does not receive the injected ASR text; the reported PRR values above 100% indicate information augmentation rather than compression fidelity. The empirical basis is also limited to one model family, point estimates without error bars, and one hyperparameter (gamma) selected on a headline benchmark. These issues are fixable but require a revised baseline definition and additional experiments to support the headline claims.

major comments (3)
  1. [§4.2, Tables 1 and 2] The performance retention ratio (PRR) is computed against a Full KV baseline that receives raw audio only, while VoxZip injects ASR transcriptions into the input in Stage 1. Since §4.2 attributes the gains to 'introducing ASR transcriptions as explicit semantic anchors,' the reported PRR values (including 104.0% at a 25% budget in Table 1 and 102.64% in Table 2) conflate compression fidelity with added textual information. The abstract's claim that VoxZip 'sustains over 90% of the uncompressed baseline performance' is therefore not established as a compression-retention result. Please report PRR relative to an uncompressed ASR-augmented baseline (e.g., the same fused embeddings without token reduction) for the headline rows, and separate the accuracy benefit of ASR injection from the accuracy loss due to compression. Table 4's T+A Full vs. Audio-only comparison is a step in this direction but is only shown on a subset of tasks and is not used for the headline numbers.
  2. [§3.2.1, Eq. (3) and §5 Limitations] The temporal alignment in Eq. (3) assumes a fixed linear mapping between ASR word timestamps and audio token indices (idx = floor(L_a * tau / tau_total)). This requires a uniform audio frame rate and unbiased timestamps. The Limitations section acknowledges vulnerability to noise and accents, but the Introduction claims 'Extended evaluations further confirm robustness against acoustic disturbances' without presenting any such evaluation in the paper. Please either add the robustness experiments or remove the claim, and provide some validation or calibration of the linear mapping (e.g., alignment error analysis, or a comparison against forced alignment) to rule out systematic misalignment that would cause Eq. (4) to bind text embeddings to incorrect acoustic segments.
  3. [§4.4.3, Tables 5 and 1] The temporal decay factor gamma=0.95 is selected by sweeping on the Vox-Infinity benchmark at a 5% cache budget (Table 5) and then used for all other results, including the headline 5%-budget Vox-Infinity rows in Table 1. Because Vox-Infinity is one of the headline evaluation benchmarks, this constitutes test-set hyperparameter tuning for the main claim. Please either tune gamma on a held-out development split, use a fixed default value without per-benchmark selection, or explicitly disclose the selection and report results across a range of gamma values for the other benchmarks to assess sensitivity.
minor comments (7)
  1. [§4.1.2] The baseline label 'SLLM [29]' in the Baselines paragraph is a typo; the citation refers to StreamingLLM, and the name should be written out.
  2. [§3.2.1] The phrase 'approximately0 .25×' has a spacing typo; also the factor 0.25 is a rough claim since the compression ratio depends on the ratio of text-token length to audio-token length, which varies across utterances.
  3. [Introduction] The statement that 'Extended evaluations further confirm robustness against acoustic disturbances' is not supported by any experiment in the paper; add such evaluations or remove the claim.
  4. [Table 4 caption] The caption does not specify the KV cache budget at which the configurations (3) and (4) are evaluated, nor whether (2) Audio-only is uncompressed; please add this information for interpretability.
  5. [Tables 1 and 2] The 10% and 5% budget rows are reported only for VoxZip, not for the compressed baselines; please either include the baselines at these budgets or state in the text why they were omitted.
  6. [§4.4.2, Table 4] The 'Acoustic Gain' row is useful, but the per-task column labels (e.g., 'Sign.', 'Perc.', 'Cult.') are ambiguous without cross-referencing Table 2; please provide a legend or expand the abbreviations in the caption.
  7. [General] The paper does not report standard deviations or numbers of runs; adding these would strengthen the reliability of the single-backbone point estimates.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: VoxZip's headline numbers are empirical benchmark measurements, not logical consequences of its construction equations.

full rationale

VoxZip is an empirical KV-cache compression method, and its central claims are measured accuracies and retention ratios on external benchmarks, not quantities derived from its own equations. The construction in Eqs. (3)–(5) slices acoustic latents by ASR timestamps, pools them, and adds text embeddings; Eq. (6) defines a decayed attention score for eviction. None of these equations analytically entails the reported benchmark accuracies, and no fitted parameter is renamed as a prediction. The disclosed sweep of the decay factor gamma on Vox-Infinity (Table 5) before reporting Vox-Infinity results is evaluation-set tuning and a validity risk, but it is not circularity: the reported accuracies are observed outcomes, not statistical consequences of gamma. Similarly, the PRR denominator Full KV lacks VoxZip's injected ASR text, so the >90% retention claim may confound compression fidelity with added transcript information; the paper itself attributes the gains to the semantic anchors in Section 4.2. That is a missing-control experimental-design concern, not a self-referential derivation. Self-citations, including the Vox-Infinity benchmark, are used as evaluation references rather than as load-bearing justification of the mechanism, and no uniqueness theorem or ansatz is imported from prior author work to force the design. The Limitations section explicitly acknowledges ASR-noise vulnerability and the simplicity of the fusion operation, further confirming that the method's behavior is treated as empirical rather than as a closed circular derivation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or latent entity; the fused embedding is a deterministic combination of existing representations. The free parameters are all engineering hyperparameters disclosed in the paper.

free parameters (4)
  • Temporal decay factor gamma = 0.95
    Set by sweeping gamma on Vox-Infinity at a 5% cache budget (Table 5); used for all reported results.
  • Attention sink size T = 4
    Empirically set in Section 3.2.2 for generation stability; no sensitivity analysis provided.
  • Recent window fraction = 20% of total KV cache budget
    Dynamically allocated to the newest M tokens in Section 3.2.2; chosen empirically.
  • ASR confidence threshold = 0.3
    Segments with lower confidence are treated as background in Section 3.2.1; hand-chosen without sensitivity analysis.
assumptions (4)
  • domain assumption Qwen3-Omni's audio encoder produces acoustic features at a rate uniform in real time, so Eq. (3) linearly maps ASR timestamps to token indices.
    Invoked in Section 3.2.1 to slice E_A. No calibration against the encoder's actual downsampling schedule is provided.
  • domain assumption Element-wise addition of pooled audio embeddings and text embeddings in the shared latent space preserves both semantic content and paralinguistic cues.
    Used to construct fused anchors in Section 3.2.1; the authors themselves flag it as simplistic in Limitations.
  • domain assumption Attention scores computed on the compressed, fused sequence are reliable proxies for token importance on the original audio.
    Underlies the Stage-2 eviction in Eq. (6); standard in KV eviction literature but untested here for fused tokens.
  • domain assumption Whisper-Turbo transcriptions, after confidence filtering at 0.3, provide sufficiently accurate timestamps under noise and accent variation.
    Relied on in Section 3.2.1; the paper's robustness discussion is qualitative and the Limitations section concedes noise vulnerability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference." pith.science (2026). https://pith.science/paper/I24L2MWN

@misc{pith2026260808569,
  author       = {Pith},
  title        = {Pith review of: VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I24L2MWN}},
  note         = {Machine review of arXiv:2608.08569}
}
read the original abstract

Recent advancements in Speech Large Language Models have demonstrated remarkable capabilities in understanding complex audio tasks. Despite this progress, their long-context inference remains severely bottlenecked by prohibitive KV cache memory demands. Existing text-centric compression methods struggle here, often disrupting speech continuity or discarding crucial semantic cues. To address this, we propose VoxZip, a train-free, two-stage semantic-anchored KV cache compression framework. The first stage uses automatic speech recognition (ASR) transcriptions as explicit semantic anchors to temporally align, compress, and fuse audio tokens, significantly reducing the initial KV cache while elevating token information density. To further improve the compression ratio, the second stage employs a dynamic filtering strategy based on temporally decayed accumulated attention to evict non-essential tokens while mitigating early-token bias. Comprehensive evaluations on Qwen3-Omni across six diverse audio benchmarks demonstrate the superiority of our approach. VoxZip excels in long-audio reasoning and consistently maintains high-fidelity perception on short-form tasks. Notably, it sustains over 90\% of the uncompressed baseline performance even under an aggressive 20x KV cache compression in long-context scenarios. Furthermore, at a 4x compression ratio, VoxZip yields a 1.9x increase in inference throughput alongside a 3.3x reduction in peak memory overhead. Code and models will be available at https://github.com/MM-Speech/VoxZip.

Figures

Figures reproduced from arXiv: 2608.08569 by the authors.

Figure 1
Figure 1. Attention score distributions in Qwen3-Omni [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The VoxZip framework. Stage 1 (left) performs semantic-anchored audio compression during the prefill stage. Stage 2 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. GPU peak memory across conversational turns. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 11 canonical work pages

  1. [1]

    Smith, Yulia Tsvetkov, and Sachin Kumar

    Orevaoghene Ahia, Martijn Bartelds, Kabir Ahuja, Hila Gonen, Valentin Hofmann, Siddhant Arora, Shuyue Stella Li, Vishal Puttagunta, Mofetoluwa Adeyemi, Char- ishma Buchireddy, Ben Walls, Noah Bennett, Shinji Watanabe, Noah A. Smith, Yulia Tsvetkov, and Sachin Kumar. 2025. BLAB: Brutally Long Audio Bench. arXiv:2505.03054 [cs.AI] https://arxiv.org/abs/2505.03054

    arXiv 2025

  2. [2]

    Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Yucheng Li, Tianyu Liu, Kem- ing Lu, Wayne Xiong, Yue Dong, Junjie Hu, and Wen Xiao. 2025. PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling. arXiv:2406.02069 [cs.CL] https://arxiv.org/abs/2406.02069

    arXiv 2025

  3. [3]

    Di Cao, Dongjie Fu, Hai Yu, Siqi Zheng, Xu Tan, and Tao Jin. 2026. X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs. arXiv:2603.24596 [eess.AS] https://arxiv.org/abs/2603.24596

    arXiv 2026

  4. [4]

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. 2024. An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models. arXiv:2403.06764 [cs.CV] https://arxiv.org/abs/2403.06764

    arXiv 2024

  5. [5]

    Xize Cheng, Dongjie Fu, Chenyuhao Wen, Tao Jin, Hai Yu, Di Cao, Qinying Liu, Yexin Yang, Zehan Wang, Shengpeng Ji, Siqi Zheng, Xu Tan, and Zhou Zhao

    work page

  6. [6]

    Guo, and Irwin King

    Wenqian Cui, Dianzhi Yu, Xiaoqi Jiao, Ziqiao Meng, Guangyan Zhang, Qichao Wang, Steven Y. Guo, and Irwin King. 2025. Recent Advances in Speech Language Models: A Survey. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pileh...

    doi:10.18653/v1/ 2025

  7. [7]

    Dongjie Fu, Xize Cheng, Linjun Li, Xiaoda Yang, Lujia Yang, and Tao Jin. 2025. PACHAT: Persona-Aware Speech Assistant for Multi-party Dialogue. InProceed- ings of the 2025 Conference on Empirical Methods in Natural Language Process- ing. Association for Computational Linguistics, Suzhou, China, 29325–29342. doi:10.18653/v1/2025.emnlp-main.1492

    doi:10.18653/v1/2025.emnlp-main.1492 2025

  8. [8]

    Dongjie Fu, Fangming Feng, Xize Cheng, Linjun Li, Zhou Zhao, and Tao Jin

    work page

Show all 36 references
  1. [9]

    Sreyan Ghosh, Arushi Goel, Jaehyeon Kim, Sonal Kumar, Zhifeng Kong, Sang gil Lee, Chao-Han Huck Yang, Ramani Duraiswami, Dinesh Manocha, Rafael Valle, and Bryan Catanzaro. 2025. Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models. InThe T...

    2025

  2. [10]

    arXiv:2604.13804 [cs.LG] https://arxiv.org/abs/2604.13804

    Character Beyond Speech: Leveraging Role-Playing Evaluation in Audio Large Language Models via Reinforcement Learning. arXiv:2604.13804 [cs.LG] https://arxiv.org/abs/2604.13804

    arXiv

  3. [11]

    Peize He, Zichen Wen, Yubo Wang, Yuxuan Wang, Xiaoqian Liu, Jiajie Huang, Ze- hui Lei, Zhuangcheng Gu, Xiangqi Jin, Jiabing Yang, Kai Li, Zhifei Liu, Weijia Li, Cunxiang Wang, Conghui He, and Linfeng Zhang. 2025. AudioMarathon: A Com- prehensive Benchmark for Long-Context Audi...

    2025

  4. [12]

    Sreyan Ghosh, Zhifeng Kong, Sonal Kumar, S Sakshi, Jaehyeon Kim, Wei Ping, Rafael Valle, Dinesh Manocha, and Bryan Catanzaro. 2025. Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities. arXiv:2503.03983 [cs.SD] https://arxiv.or...

    2025 arXiv

  5. [13]

    KimiTeam, Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, Zhengtao Wang, Chu Wei, Yifei Xin, Xinran Xu, Jianwei Yu, Yutao Zhang, Xinyu Zhou, Y. Charles, Jun Chen, Yanru Chen, Yulun Du, Weiran He, Zhenxing Hu, Guok...

    2025 arXiv

  6. [14]

    Shengpeng Ji, Yifu Chen, Minghui Fang, Jialong Zuo, Jingyu Lu, Hanting Wang, Ziyue Jiang, Long Zhou, Shujie Liu, Xize Cheng, Xiaoda Yang, Zehan Wang, Qian Yang, Jian Li, Yidi Jiang, Jingzhen He, Yunfei Chu, Jin Xu, and Zhou Zhao. 2024. WavChat: A Survey of Spoken Dialogue Mode...

    2024 arXiv

  7. [15]

    Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. 2024. SnapKV: LLM Knows What You are Looking for Before Generation. InAdvances in Neural Information Processing Systems, A. Globerson, L. Mackey, D....

    2024 doi

  8. [16]

    Haoyang Li, Yiming Li, Anxin Tian, Tianhao Tang, Zhanchao Xu, Xuejia Chen, Nicole Hu, Wei Dong, Qing Li, and Lei Chen. 2025. A Survey on Large Language Model Acceleration based on KV Cache Management. arXiv:2412.19442 [cs.AI] https://arxiv.org/abs/2412.19442

    2025 arXiv

  9. [17]

    Yueqian Lin, Yuzhe Fu, Jingyang Zhang, Yudong Liu, Jianyi Zhang, Jingwei Sun, Hai Helen Li, and Yiran Chen. 2025. SpeechPrune: Context-Aware Token Pruning for Speech Information Retrieval. In2025 IEEE International Conference on Multimedia and Expo (ICME). 1–6. doi:10.1109/ICM...

    2025

  10. [18]

    Zongqian Li, Yinhong Liu, Yixuan Su, and Nigel Collier. 2024. Prompt Com- pression for Large Language Models: A Survey. arXiv:2410.12388 [cs.CL] https://arxiv.org/abs/2410.12388

    2024 arXiv

  11. [19]

    Yanyu Liu, Jingying Fu, Sixiang Liu, Yitian Zou, You Fu, Jiehan Zhou, and Shouhua Zhang. 2025. KV Cache Compression for Inference Efficiency in LLMs: A Review. arXiv:2508.06297 [cs.DC] https://arxiv.org/abs/2508.06297

    2025 arXiv

  12. [20]

    Xiang Liu, Zhenheng Tang, Peijie Dong, Zeyu Li, Yue Liu, Bo Li, Xuming Hu, and Xiaowen Chu. 2025. ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference. arXiv:2502.00299 [cs.CL] https://arxiv. org/abs/2502.00299

    2025

  13. [21]

    Jing Peng, Yucheng Wang, Bohan Li, Yiwei Guo, Hankun Wang, YanGui Fang, Yu Xi, Haoyu Li, Xu Li, Ke Zhang, Shuai Wang, and Kai Yu. 2026. A Survey on Speech Large Language Models for Understanding.IEEE Journal of Selected Topics in Signal Processing20, 1 (Jan. 2026), 2–31. doi:1...

    2026

  14. [22]

    Ziyang Ma, Yinghao Ma, Yanqiao Zhu, Chen Yang, Yi-Wen Chao, Ruiyang Xu, Wenxi Chen, Yuanzhe Chen, Zhuo Chen, Jian Cong, Kai Li, Keliang Li, Siyou Li, Xinfeng Li, Xiquan Li, Zheng Lian, Yuzhe Liang, Minghao Liu, Zhikang Niu, Tianrui Wang, Yuping Wang, Yuxuan Wang, Yihao Wu, Gua...

    2025

  15. [23]

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust Speech Recognition via Large-Scale Weak Supervision. arXiv:2212.04356 [eess.AS] https://arxiv.org/abs/2212.04356

    2022 arXiv

  16. [24]

    Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Brad- bury, Anselm Levskaya, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. 2022. Efficiently Scaling Transformer Inference. arXiv:2211.05102 [cs.LG] https://arxiv.org/abs/2211.05102

    2022 arXiv

  17. [25]

    Keda Tao, Can Qin, Haoxuan You, Yang Sui, and Huan Wang. 2025. DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR). 18992–19001

    2025

  18. [26]

    Sakshi Sakshi, Utkarsh Tyagi, Sonal Kumar, Ashish Seth, Ramaneswaran Selvakumar, Oriol Nieto, Ramani Duraiswami, Sreyan Ghosh, and Di- nesh Manocha. 2025. MMAU: A Massive Multi-Task Audio Under- standing and Reasoning Benchmark. InInternational Conference on Learn- ing Represe...

    2025

  19. [27]

    Zhongwei Wan, Ziang Wu, Che Liu, Jinfa Huang, Zhihong Zhu, Peng Jin, Longyue Wang, and Li Yuan. 2024. LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference. InFindings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-O...

    2024 doi

  20. [28]

    Zhongwei Wan, Xinjian Wu, Yu Zhang, Yi Xin, Chaofan Tao, Zhihong Zhu, Xin Wang, Siqi Luo, Jing Xiong, Longyue Wang, and Mi Zhang. 2025. D2O: Dynamic Discriminative Operations for Efficient Long-Context Inference of Large Language Models. arXiv:2406.13035 [cs.CL] https://arxiv....

    2025 arXiv

  21. [29]

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024. Efficient Streaming Language Models with Attention Sinks. InThe Twelfth Inter- national Conference on Learning Representations. https://openreview.net/forum? id=NG7sS51zVF

    2024

  22. [30]

    Dingdong WANG, Jincenzi Wu, Junan Li, Dongchao Yang, Xueyuan Chen, Tian- hua Zhang, and Helen M. Meng. 2026. MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmark. InThe Fourteenth Interna- tional Conference on Learning Representations. https://openr...

    2026

  23. [31]

    Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, Yuanjun Lv, Yongqi Wang, Dake Guo, He Wang, Linhan Ma, Pei Zhang, Xinyu Zhang, Hongkun Hao, Zishan Guo, Baosong Yang, Bin Zhang, Ziyang Ma, Xipin Wei, Shuai Bai, Ke...

    2025 arXiv

  24. [32]

    Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. 2025. Qwen2.5-Omni Technical Report. arXiv:2503.20215 [cs.CL] https://arxiv.org/abs/2503.20215

    2025 arXiv

  25. [33]

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Re, Clark Barrett, Zhangyang Wang, and Beidi Chen. 2023. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models. InThirty-seventh...

    2023

  26. [34]

    Aohan Zeng, Zhengxiao Du, Mingdao Liu, Kedong Wang, Shengmin Jiang, Lei Zhao, Yuxiao Dong, and Jie Tang. 2024. GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbot. arXiv:2412.02612 [cs.CL] https://arxiv. org/abs/2412.02612

    2024 arXiv

  27. [36]

    Zixuan Zhou, Xuefei Ning, Ke Hong, Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan, Xiuhong Li, Shengen Yan, Guohao Dai, Xiao- Ping Zhang, Yuhan Dong, and Yu Wang. 2024. A Survey on Efficient Inference for Large Language Models. arXiv:2404.14294 [cs.CL]...

    2024 arXiv

  28. [2026]

    https://openreview.net/forum?id=6dKwqnT7bu

    Vox-Infinity: Benchmarking the Limits of Long-Context Spoken Language Models. https://openreview.net/forum?id=6dKwqnT7bu

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.