Pith. sign in

REVIEW 3 major objections 3 minor 52 references

OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training-free OmniPack keeps 98% of omni-modal accuracy at 16.7% of the compute.

desk verdict Genuine two-stage compression recipe with solid ablations; the 6.8% FLOPs headline is overstated because the metric excludes text and ignores GQA. read the letter →

arxiv 2608.03812 v1 pith:YAPVLZ4S submitted 2026-08-04 cs.CV

Wanshun Su , Yang Shi , Feihu Liu , Ziwen Yu , Yan Min , Zhuoran Zhang , Qixun Wang , Haotian Wang
show 5 more authors
Shixuan Liu Yuanxing Zhang Peng Wu Chengfu Huo Liang Ding
This is my paper
classification cs.CV
keywords tokencompressionomni-modallargelanguagemodelsaudio-visualunderstandinginferenceaccelerationtraining-freemergingquery-conditionedpruninglong-video
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

OmniPack claims that token compression for omni-modal LLMs fails when it happens in only one place: pruning before the LLM can discard globally scattered evidence, while pruning inside the LLM underuses cross-modal cues. The paper proposes a training-free, two-stage scheme that first cuts structural redundancy in each modality using importance, coverage, and similarity-aware merging, then, after enough layers of multimodal interaction, prunes again using the text question and the other modality as guidance. If the claim holds, a plug-in compression method can cut audio-visual token compute by roughly an order of magnitude, or more, while losing only a few points of accuracy across five audio-visual benchmarks. On Qwen2.5-Omni-7B, OmniPack reports 98.0% of full accuracy at 16.7% of FLOPs, and 92.9% accuracy at 6.8% of FLOPs.

What carries the argument

The load-bearing mechanism is a stage-specialized progressive compression schedule. Stage one (pre-LLM) selects tokens by three complementary signals: attention-based importance, global coverage via DPC-KNN (density-peaks clustering over a combined feature and position distance), and similarity-aware merging that folds unselected tokens into retained representatives. Stage two (inner-LLM) waits until after the first ℓ transformer blocks (layer 18 for the 28-layer backbones) and scores each audio/visual token using a relevance formula that combines textual guidance, audio-visual collaboration through a pooled prototype of the other modality, and within-modality representativeness, followed by

What would settle it

A concrete test: on a fixed benchmark with the pre-LLM stage unchanged, compare OmniPack's layer-18 token selection against random selection at the same inner-LLM budget. If random selection matches OmniPack's accuracy, the query-conditioned scores are not doing the work. A complementary check is to feed the tokens OmniPack prunes at layer 18 to the LLM instead of the retained set and see whether they support correct answers.

Watch

Extended reading notes

Core claim

The central claim is that the right locus of compression depends on how semantically mature the representations are. Before the LLM, audio and visual tokens have not interacted much, so compression should be structural: keep salient tokens via encoder attention plus temporal/spatial variation, preserve global coverage with density-peak representatives over a feature-plus-position distance, and merge discarded tokens into retained representatives rather than dropping them. Inside the LLM, after enough transformer layers, cross-modal and text semantics have formed, so compression should be query-conditioned: score each audio/visual token by its similarity to the pooled question, to the most re

Load-bearing premise

The load-bearing premise is that after the first 18 transformer layers, cosine similarity between a token, the pooled question, and a pooled prototype of the other modality correctly identifies which audio and visual tokens the final answer will depend on.

Editorial extensions

If this is right

  • At a 15%/7.5% retention budget, OmniPack reports 95.6% of original accuracy with a 10.0× FLOP reduction and a 4.5× prefill speedup on Qwen2.5-Omni-7B, so order-of-magnitude compute savings do not force accuracy collapse.
  • The method transfers across three Omni-LLM backbones without training, so the benefit is tied to the compression schedule rather than to one model's weights.
  • On VideoMME long videos, OmniPack retains 107.4% of the uncompressed score at the 15%/7.5% setting, suggesting that aggressive redundancy removal can help most when sequences are long.
  • Swapping either stage for an existing method's strategy lowers average accuracy, indicating that the structural pre-LLM stage and the query-conditioned inner-LLM stage are complementary rather than interchangeable.
  • At a 10%/5% budget the method keeps 92.9% accuracy with 6.8% of FLOPs, showing graceful degradation instead of a performance cliff under extreme compression.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The fixed layer-18 compression point embeds a timing assumption; a natural extension would be to measure cross-modal alignment per layer and make the inner-LLM compression layer adaptive to input length or task difficulty.
  • Because the pre-LLM merging step uses weighted averaging, combining it with quantization or memory-aware dropping of the retained tokens could yield further savings; the paper reports FLOPs but not peak memory.
  • The structural-then-semantic schedule could transfer to video-only or audio-only LLMs and to streaming settings, where coverage selection would enforce recency and importance would track salient events; the paper only evaluates offline batch inputs.
  • A sharper test of the inner-LLM stage would compare its selected tokens against an oracle that knows which tokens are actually needed for the correct answer; the paper does not report such an upper-bound comparison.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes OmniPack, a training-free two-stage token-compression framework for omni-modal LLMs. Before the LLM, modality-specific compression selects tokens by importance, coverage, and similarity-aware merging; inside the LLM, after a chosen layer, a query-conditioned stage uses text relevance, audio-visual collaboration, and within-modality diversity to prune further. Experiments on five audio-visual benchmarks across Qwen2.5-Omni-3B/7B and MiniCPM-o-2.6 report state-of-the-art performance-efficiency trade-offs, including 98.0% relative performance at 16.7% FLOPs and 92.9% at 6.8% FLOPs on Qwen2.5-Omni-7B.

Significance. If the efficiency numbers are taken at face value, the contribution is practically useful: a training-free, plug-in method that preserves most multimodal understanding while cutting prefill compute by an order of magnitude would be valuable for deploying omni-modal LLMs. The paper includes a broad benchmark evaluation, ablations of the main components, sensitivity analyses for several hyperparameters, and comparisons against a reasonable set of baselines. It also reports measured prefill speedups, which is a useful complement to the analytical FLOPs estimate. The main caveat is that the analytical FLOPs metric, which underlies the headline claims, does not count text tokens and uses a full-MHA cost model, making the absolute efficiency claims stronger than what the model actually saves.

major comments (3)
  1. [Appendix D, Eq. (18)] The FLOPs metric defines n as vision+audio tokens only and excludes text tokens, while charging full multi-head attention cost 2n^2d. The Qwen2.5-Omni and MiniCPM-o backbones use GQA-style KV heads, so the true n^2 coefficient is smaller. Because token compression reduces n and the n^2 term dominates the reported savings, the headline '6.8% of original FLOPs' and '10.0x reduction' overstate actual model FLOPs savings. A rough recalculation with ~500 text tokens and a GQA ratio of 7 moves the 15%/7.5% setting to roughly 10–16% of total LLM FLOPs and the reduction from 10x to about 6–9x. The measured prefill speedups in Table 3 (4.5x at 15%/7.5% but only 3.0x at 25%/12.5% where the FLOPs claim is 6.0x) corroborate this gap. The efficiency axis is load-bearing for the paper's central claim and needs recalibration or explicit qualification.
  2. [§4.4, Figure 3 and Table 15; §4.1] Default hyperparameters and the inner-LLM compression layer are selected using the same five benchmarks on which the final results are reported. For example, layer 18 is chosen because it performs best on WorldSense and DailyOmni in Figure 3, and (eta_v, eta_a), lambda, tau, and zeta are selected from sensitivity sweeps in Table 15. This makes the 'consistently best performance' claim partly in-sample. The authors should either fix defaults a priori, use a validation split or benchmark-wise cross-validation, or explicitly state that the reported numbers are tuned on these benchmarks. This does not invalidate the method, but it affects the strength of the generalization claim.
  3. [Tables 1, 2, 10] All results are single-run point estimates with no variance, confidence intervals, or significance testing. Several comparisons are close: for example, in Table 2, MiniCPM-o-2.6 OmniPack reaches 100.8% versus 100.4% for OmniPack (w/o M), and in Table 10, Random at 25% reaches 102.3% on MiniCPM-o-2.6. Without repeated runs or significance tests, the claim of 'outperforming all existing methods' is not statistically supported for small differences. This is a standard but important methodological concern given the paper's strong comparative claims.
minor comments (3)
  1. [Table 1] Several cells have missing spaces in the text extraction, e.g., '59.866.433.9', '58.760.7', '58.144.7'. Please check the typesetting.
  2. [Eq. (11)] The notation N(·) for min-max normalization is easily confused with N_m for token counts. Consider renaming the normalization operator, e.g., Norm(·).
  3. [§3.3, Eq. (9)] The merge-affinity function uses tau_m with modality-specific values but the text says 'tau_m controls the modality-specific positional constraint'; consider clarifying whether the two tau values are tied to the DPC distance weighting or are independent.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: OmniPack's performance claims are measured on external benchmarks and its efficiency numbers follow a defined FLOPs proxy; hyperparameter sweeps and background self-citations are not load-bearing circular steps.

full rationale

Walking the derivation chain: the compression pipeline (Eqs. 1-12) defines token-selection and merging rules from encoder attention, positional/feature distances, and inner-LLM relevance scores; no predicted quantity reduces by construction to a fitted input. The headline performance numbers (e.g., 98.0% relative performance at 25%/12.5% retention, 95.6% at 15%/7.5%) are measured accuracies on five external benchmarks (AVUT, WorldSense, DailyOmni, VideoMME, LVOmniBench) compared against independent baselines (FastV, VisionZip, OmniZip, OmniSIFT, SEATS, Random), so the central claim is not self-definitional. The efficiency axis is an explicitly defined FLOPs proxy (Eq. 18 counts only vision+audio tokens n, excludes text tokens, and assumes full MHA; Appendix D states this and follows prior work [34]). While the skeptic is right that this proxy may overstate savings relative to actual model FLOPs, that is a correctness/validity concern about the metric, not circularity: the same proxy is applied uniformly to all methods, and relative rankings are not forced by construction. Hyperparameters (eta, lambda, tau, zeta, layer 18) were indeed selected from sweeps on the same benchmarks (Figure 3, Figure 6, Table 15), but the paper reports sensitivity analyses showing small performance variation, and the final results are measurements, not quantities derived from those sweeps; this is selection-bias risk, not equation-level circularity. Self-citations (e.g., OmniSIFT [7], MAVORS [29], the authors' earlier works [21,37,40,41,51]) appear as background or baselines, and no load-bearing premise, uniqueness theorem, or ansatz is imported from them. No step in the paper exhibits Eq. X equal to Eq. Y by definition, a fitted parameter renamed as a prediction, or a self-citation chain that forces the result. The honest finding is no significant circularity.

Assumptions & free parameters 9 free parameters · 5 assumptions · 0 invented entities

OmniPack introduces no new physical or model entities; the retention and recycle pools are bookkeeping labels for existing tokens. The central claim rests on a set of tuned hyperparameters and on assumptions about the geometry of embeddings and the fairness of the FLOPs proxy, not on new theoretical objects.

free parameters (9)
  • eta_m (importance allocation ratio) = eta_v=0.25, eta_a=0.35
    Fraction of the pre-LLM token budget assigned to importance selection (Eq. 6); balanced across modalities and tuned via sensitivity analysis in Figure 6.
  • lambda (positional distance weight) = 0.20
    Weight on positional distance in the joint distance used by coverage selection (Eq. 7); chosen by ablation in Table 15.
  • tau_m (merge positional constraints) = tau_v=0.10, tau_a=0.05
    Modality-specific positional weights in the merge affinity function (Eq. 9); selected by ablation in Table 15.
  • zeta (representative importance weight) = 0.10
    Weight on the normalized importance of the retained token in merge affinity (Eq. 9); tuned in Table 15.
  • k (DPC-KNN neighbors) = 7
    Number of nearest neighbors used in the local density estimate (Appendix B.1); fixed for all experiments rather than fit per benchmark.
  • inner-LLM compression layer l = 18 for 28-layer backbones, 26 for 36-layer
    Layer at which query-conditioned compression is applied; selected from Figure 3 as the best trade-off on WorldSense and DailyOmni.
  • inner-LLM retention ratio r'_m = 0.50
    Fraction of pre-LLM tokens retained after inner-LLM compression (Eq. 2); fixed at 50% in all experiments.
  • modality budget allocations R_v/R_a = Table 9 entries per backbone and retention ratio
    Visual and audio retention ratios are chosen to match each model's native tokenization; these allocations affect all comparisons and are also applied to baselines.
  • R_dec (assumed decoding length) = 100
    Decoding length used in the FLOPs model (Appendix D); the reported FLOPs ratios depend on this assumption.
assumptions (5)
  • domain assumption Encoder attention statistics reflect token importance for the downstream multimodal task.
    Used in Eq. 3 to construct the attention-centrality vector a_m; no independent evidence is given that encoder attention scores align with the task-relevance of audio-visual tokens.
  • domain assumption Cosine similarity in the LLM embedding space is a valid measure of semantic redundancy and information transferability for merging.
    Eqs. 7-10 and 11 align tokens by cosine similarity; if the embedding geometry does not preserve semantic redundancy, merging will corrupt evidence.
  • domain assumption By layer ℓ=18/26, mean-pooled text and other-modality prototypes provide reliable query-conditioned relevance.
    Eq. 11 computes relevance from cosine similarity to a mean-pooled query and a pooled other-modality prototype; the layer index is empirical, not derived.
  • domain assumption Excluding text tokens, encoders, projectors, selection operations, and the LM head yields a fair efficiency comparison.
    Appendix D states these exclusions; the resulting FLOPs ratios are lower than full-model wall-clock savings.
  • domain assumption Scores on AVUT, WorldSense, DailyOmni, VideoMME, and LVOmniBench are representative of general omni-modal performance.
    Section 4.1 and Appendix A describe the five benchmarks; there is no held-out set that validates generalization beyond these suites.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models." pith.science (2026). https://pith.science/paper/YAPVLZ4S

@misc{pith2026260803812,
  author       = {Pith},
  title        = {Pith review of: OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YAPVLZ4S}},
  note         = {Machine review of arXiv:2608.03812}
}
read the original abstract

Omni-modal large language models (Omni-LLMs) have achieved remarkable performance on audio-visual understanding tasks, but processing long and highly redundant visual and audio token sequences incurs substantial computational overhead, demanding aggressive token compression for efficient deployment. Existing methods often degrade at low token budgets: pre-LLM compression may discard structurally important and globally distributed evidence, whereas inner-LLM compression often underexploits query-conditioned audio-visual collaboration. To address these limitations, we propose OmniPack, a training-free framework that coordinates structural compression before the LLM with task-relevant semantic refinement within the LLM. Before the LLM, OmniPack removes structural redundancy through modality-specific importance, global coverage, and similarity-aware merging. After sufficient multimodal interaction, it further consolidates diverse, task-relevant representations through textual guidance and audio-visual collaboration. Extensive experiments on five benchmarks with three Omni-LLM backbones demonstrate that OmniPack consistently achieves the best performance-efficiency trade-off across diverse retention ratios, outperforming all existing methods. Notably, on Qwen2.5-Omni-7B, OmniPack preserves 98.0% of the original performance while reducing FLOPs to 16.7%, and still retains 92.9% of the original performance with only 6.8% of the original FLOPs.

Figures

Figures reproduced from arXiv: 2608.03812 by the authors.

Figure 1
Figure 1. Performance comparison on Qwen2.5-Omni-7B across [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the OmniPack framework. Before the LLM, modality-specific compression performs importance selection, coverage [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Effect of the inner-LLM compression layer. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Visualization of the Pre-LLM and Inner-LLM token [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison at the 25% pre-LLM retention [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Sensitivity analysis of the importance ratios [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 26 canonical work pages

  1. [1]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025. 2

    arXiv 2025

  2. [2]

    To- ken merging: Your vit but faster.arXiv preprint arXiv:2210.09461, 2022

    Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. To- ken merging: Your vit but faster.arXiv preprint arXiv:2210.09461, 2022. 2

    arXiv 2022

  3. [3]

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. InEuropean Conference on Computer Vision, 2024. 5, 2

    work page 2024

  4. [4]

    Avocado: An audiovisual video cap- tioner driven by temporal orchestration.arXiv preprint arXiv:2510.10395, 2025

    Xinlong Chen, Yue Ding, Weihong Lin, Jingyun Hua, Linli Yao, Yang Shi, Bozhou Li, Yuanxing Zhang, Qiang Liu, Pengfei Wan, et al. Avocado: An audiovisual video cap- tioner driven by temporal orchestration.arXiv preprint arXiv:2510.10395, 2025. 2

    arXiv 2025

  5. [5]

    Minicpm-o 4.5: Towards real- time full-duplex omni-modal interaction.arXiv preprint arXiv:2604.27393, 2026

    Junbo Cui, Bokai Xu, Chongyi Wang, Tianyu Yu, Weiyue Sun, Yingjing Xu, Tianran Wang, Zhihui He, Wenshuo Ma, Tianchi Cai, et al. Minicpm-o 4.5: Towards real- time full-duplex omni-modal interaction.arXiv preprint arXiv:2604.27393, 2026. 2

    arXiv 2026

  6. [6]

    OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Models

    Yuchen Deng, Zidang Cai, Hai-Tao Zheng, Jie Wang, Fei- diao Yang, and Yuxing Han. Omnirefine: Alignment-aware 8 cooperative compression for efficient omnimodal large lan- guage models.arXiv preprint arXiv:2605.12056, 2026. 2

    work page Pith review arXiv 2026

  7. [7]

    Omnisift: Modality-asymmetric token compression for efficient omni-modal large language models.arXiv preprint arXiv:2602.04804, 2026

    Yue Ding, Yiyan Ji, Jungang Li, Xuyang Liu, Xinlong Chen, Junfei Wu, Bozhou Li, Bohan Zeng, Yang Shi, Yushuo Guan, et al. Omnisift: Modality-asymmetric token compression for efficient omni-modal large language models.arXiv preprint arXiv:2602.04804, 2026. 1, 2, 5, 3

    arXiv 2026

  8. [8]

    Unified spatiotemporal token compression for video-llms at ultra-low retention

    Junhao Du, Jialong Xue, Anqi Li, Jincheng Dai, and Guo Lu. Unified spatiotemporal token compression for video-llms at ultra-low retention. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2026. 2

    work page 2026

Show all 52 references
  1. [9]

    Study on density peaks clustering based on k-nearest neighbors and principal component analysis.Knowledge-Based Systems, 2016

    Mingjing Du, Shifei Ding, and Hongjie Jia. Study on density peaks clustering based on k-nearest neighbors and principal component analysis.Knowledge-Based Systems, 2016. 4, 2

    2016

  2. [10]

    Flashvid: Efficient video large lan- guage models via training-free tree-based spatiotemporal to- ken merging.arXiv preprint arXiv:2602.08024, 2026

    Ziyang Fan, Keyu Chen, Ruilong Xing, Yulin Li, Li Jiang, and Zhuotao Tian. Flashvid: Efficient video large lan- guage models via training-free tree-based spatiotemporal to- ken merging.arXiv preprint arXiv:2602.08024, 2026. 2

    2026

  3. [11]

    Vita: Towards open- source interactive omni multimodal llm.arXiv preprint arXiv:2408.05211, 2024

    Chaoyou Fu, Haojia Lin, Zuwei Long, Yunhang Shen, Yuhang Dai, Meng Zhao, Yi-Fan Zhang, Shaoqi Dong, Yangze Li, Xiong Wang, et al. Vita: Towards open- source interactive omni multimodal llm.arXiv preprint arXiv:2408.05211, 2024. 1

    2024 arXiv

  4. [12]

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. InProceedings of the IEEE/CVF Conference on ...

    2025

  5. [13]

    Arc-hunyuan-video-7b: Structured video comprehension of real-world shorts.arXiv preprint arXiv:2507.20939, 2025

    Yuying Ge, Yixiao Ge, Chen Li, Teng Wang, Junfu Pu, Yizhuo Li, Lu Qiu, Jin Ma, Lisheng Duan, Xinyu Zuo, et al. Arc-hunyuan-video-7b: Structured video comprehension of real-world shorts.arXiv preprint arXiv:2507.20939, 2025. 1

    2025 arXiv

  6. [14]

    Echoingpixels: Cross-modal adap- tive token reduction for efficient audio-visual llms.arXiv preprint arXiv:2512.10324, 2025

    Chao Gong, Depeng Wang, Zhipeng Wei, Ya Guo, Huijia Zhu, and Jingjing Chen. Echoingpixels: Cross-modal adap- tive token reduction for efficient audio-visual llms.arXiv preprint arXiv:2512.10324, 2025. 2

    2025 arXiv

  7. [15]

    Echoingpixels: Aliasing-resistant joint token reduction for audio-visual llms

    Chao Gong, Depeng Wang, Zhipeng Wei, Ya Guo, Huijia Zhu, and Jingjing Chen. Echoingpixels: Aliasing-resistant joint token reduction for audio-visual llms. InForty-third International Conference on Machine Learning, 2026. 2

    2026

  8. [16]

    Gemini 3.1 pro model card.https: / / deepmind

    Google DeepMind. Gemini 3.1 pro model card.https: / / deepmind . google / models / model - cards / gemini-3-1-pro/, 2026. Accessed: 2026-06. 2

    2026

  9. [17]

    Worldsense: Evaluating real-world omni- modal understanding for multimodal llms.arXiv preprint arXiv:2502.04326, 2025

    Jack Hong, Shilin Yan, Jiayin Cai, Xiaolong Jiang, Yao Hu, and Weidi Xie. Worldsense: Evaluating real-world omni- modal understanding for multimodal llms.arXiv preprint arXiv:2502.04326, 2025. 5, 1

    2025 arXiv

  10. [18]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perel- man, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Weli- hinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 2

    2024 arXiv

  11. [19]

    Con- textguard: Structured self-auditing for context learning in language models.arXiv preprint arXiv:2605.26827, 2026

    Hongbo Jin, Chi Wang, Haoran Tang, Zhongjing Du, Xu Jiang, Jingqi Tian, Qiaoman Zhang, and Jiayu Ding. Con- textguard: Structured self-auditing for context learning in language models.arXiv preprint arXiv:2605.26827, 2026. 2

    2026 arXiv

  12. [20]

    Token pruning in audio trans- formers: Optimizing performance and decoding patch im- portance.arXiv preprint arXiv:2504.01690, 2025

    Taehan Lee and Hyukjun Lee. Token pruning in audio trans- formers: Optimizing performance and decoding patch im- portance.arXiv preprint arXiv:2504.01690, 2025. 2

    2025

  13. [21]

    Omnivideobench: Towards audio-visual understanding evaluation for omni mllms.arXiv preprint arXiv:2510.10689, 2025

    Caorui Li, Yu Chen, Yiyan Ji, Jin Xu, Zhenyu Cui, Shihao Li, Yuanxing Zhang, Wentao Wang, Zhenghao Song, Din- gling Zhang, et al. Omnivideobench: Towards audio-visual understanding evaluation for omni mllms.arXiv preprint arXiv:2510.10689, 2025. 1

    2025

  14. [22]

    Omnigaia: Towards native omni-modal ai agents

    Xiaoxi Li, Wenxiang Jiao, Jiarui Jin, Haoxuan Li, Hao Wang, Shijian Wang, Guanting Dong, Jiajie Jin, Yinuo Wang, Yuan Lu, et al. Omnigaia: Towards native omni-modal ai agents. arXiv preprint arXiv:2602.22897, 2026. 2

    2026 arXiv

  15. [23]

    Speech- prune: Context-aware token pruning for speech information retrieval

    Yueqian Lin, Yuzhe Fu, Jingyang Zhang, Yudong Liu, Jianyi Zhang, Jingwei Sun, Hai Helen Li, and Yiran Chen. Speech- prune: Context-aware token pruning for speech information retrieval. In2025 IEEE International Conference on Multi- media and Expo (ICME), 2025. 2

    2025

  16. [24]

    Video compression commander: Plug-and-play inference ac- celeration for video large language models

    Xuyang Liu, Yiyu Wang, Junpeng Ma, and Linfeng Zhang. Video compression commander: Plug-and-play inference ac- celeration for video large language models. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025. 5, 2

    2025

  17. [25]

    Gpt-5.5 system card.https://openai.com/ index/gpt- 5- 5- system- card/, 2026

    OpenAI. Gpt-5.5 system card.https://openai.com/ index/gpt- 5- 5- system- card/, 2026. Accessed: 2026-06. 2

    2026

  18. [26]

    Omnidrop: Layer- wise token pruning for omni-modal llms via query-guidance

    Yeo Jeong Park, Hyemi Jang, Minseo Choi, Jongsun Lee, Jooyoung Choi, and Yongkweon Jeon. Omnidrop: Layer- wise token pruning for omni-modal llms via query-guidance. arXiv preprint arXiv:2605.14458, 2026. 2

    2026 arXiv

  19. [27]

    Clustering by fast search-and-find of density peaks.science, 2014

    A Rodriguez Alex, Alessandro Laio, et al. Clustering by fast search-and-find of density peaks.science, 2014. 4, 2

    2014

  20. [28]

    Fastvid: Dynamic den- sity pruning for fast video large language models.Advances in Neural Information Processing Systems, 2026

    Leqi Shen, Guoqiang Gong, Tao He, Yifeng Zhang, Pengzhang Liu, Sicheng Zhao, et al. Fastvid: Dynamic den- sity pruning for fast video large language models.Advances in Neural Information Processing Systems, 2026. 2, 5

    2026

  21. [29]

    Mavors: Multi-granularity video representation for multimodal large language model

    Yang Shi, Jiaheng Liu, Yushuo Guan, Zhenhua Wu, Yuanx- ing Zhang, Zihao Wang, Weihong Lin, Jingyun Hua, Zekun Wang, Xinlong Chen, et al. Mavors: Multi-granularity video representation for multimodal large language model. InPro- ceedings of the 33rd ACM International Conference...

    2025

  22. [30]

    Audio- visual llm for video understanding

    Fangxun Shu, Lei Zhang, Hao Jiang, and Cihang Xie. Audio- visual llm for video understanding. InProceedings of the IEEE/CVF International Conference on Computer Vision,

  23. [31]

    To- kencarve: Information-preserving visual token compres- sion in multimodal large language models.arXiv preprint arXiv:2503.10501, 2025

    Xudong Tan, Peng Ye, Chongjun Tu, Jianjian Cao, Yaoxin Yang, Lin Zhang, Dongzhan Zhou, and Tao Chen. To- kencarve: Information-preserving visual token compres- sion in multimodal large language models.arXiv preprint arXiv:2503.10501, 2025. 2

    2025 arXiv

  24. [32]

    video- salmonn 2: Caption-enhanced audio-visual large language models.arXiv preprint arXiv:2506.15220, 2025

    Changli Tang, Yixuan Li, Yudong Yang, Jimin Zhuang, Guangzhi Sun, Wei Li, Zejun Ma, and Chao Zhang. video- salmonn 2: Caption-enhanced audio-visual large language models.arXiv preprint arXiv:2506.15220, 2025. 1

    2025

  25. [33]

    Dycoke: Dynamic compression of tokens for fast 9 video large language models

    Keda Tao, Can Qin, Haoxuan You, Yang Sui, and Huan Wang. Dycoke: Dynamic compression of tokens for fast 9 video large language models. InProceedings of the Com- puter Vision and Pattern Recognition Conference, 2025. 2

    2025

  26. [34]

    Omnizip: Audio-guided dynamic token compression for fast omnimodal large language models

    Keda Tao, Kele Shao, Bohan Yu, Weiqiang Wang, Jian Liu, and Huan Wang. Omnizip: Audio-guided dynamic token compression for fast omnimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 17682–17692, 2026. 1, 2, 5, 3

    2026

  27. [35]

    Lvomnibench: Pioneering long audio-video un- derstanding evaluation for omnimodal llms.arXiv preprint arXiv:2603.19217, 2026

    Keda Tao, Yuhua Zheng, Jia Xu, Wenjie Du, Kele Shao, Hesong Wang, Xueyi Chen, Xin Jin, Junhan Zhu, Bohan Yu, et al. Lvomnibench: Pioneering long audio-video un- derstanding evaluation for omnimodal llms.arXiv preprint arXiv:2603.19217, 2026. 5, 1

    2026

  28. [36]

    Qwen Team. Qwen3. 5-omni technical report.arXiv preprint arXiv:2604.15804, 2026. 2

    2026 arXiv

  29. [37]

    Monet: Reasoning in latent visual space beyond images and language.arXiv preprint arXiv:2511.21395, 2025

    Qixun Wang, Yang Shi, Yifei Wang, Yuanxing Zhang, Pengfei Wan, Kun Gai, Xianghua Ying, and Yisen Wang. Monet: Reasoning in latent visual space beyond images and language.arXiv preprint arXiv:2511.21395, 2025. 2

    2025

  30. [38]

    Beacon: Knowing when and how to perform agentic visual reasoning, 2026

    Qixun Wang, Yang Shi, Letian Cheng, Zhuoran Zhang, Yan He, Yuqi Tang, Qi Zhang, Xinlei Yu, Ruizhe Chen, Tianrun Xu, Yuanxing Zhang, Pengfei Wan, Haotian Wang, and Xi- anghua Ying. Beacon: Knowing when and how to perform agentic visual reasoning, 2026

    2026

  31. [39]

    Internvl3

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internvl3. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265, 2025. 2

    2025 arXiv

  32. [40]

    Varcmp: Adapting cross-modal pre-training models for video anomaly retrieval

    Peng Wu, Wanshun Su, Xiangteng He, Peng Wang, and Yan- ning Zhang. Varcmp: Adapting cross-modal pre-training models for video anomaly retrieval. InProceedings of the AAAI Conference on Artificial Intelligence, 2025. 1

    2025

  33. [41]

    Avadclip: Audio- visual collaboration for robust video anomaly detection

    Peng Wu, Wanshun Su, Guansong Pang, Yujia Sun, Qingsen Yan, Peng Wang, and Yanning Zhang. Avadclip: Audio- visual collaboration for robust video anomaly detection. IEEE Transactions on Multimedia, 2026. 1

    2026

  34. [42]

    Stage-adaptive token selection for efficient omni-modal llms.arXiv preprint arXiv:2605.20035, 2026

    Zijie Xin, Jie Yang, Ruixiang Zhao, Tianyi Wang, Fengyun Rao, Jing Lyu, and Xirong Li. Stage-adaptive token selection for efficient omni-modal llms.arXiv preprint arXiv:2605.20035, 2026. 2, 5, 3

    2026 arXiv

  35. [43]

    Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy re- duction.arXiv preprint arXiv:2410.17247, 2024

    Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, et al. Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy re- duction.arXiv preprint arXiv:2410.17247, 2024. 2

    2024 arXiv

  36. [44]

    Qwen2.5-omni technical report.arXiv preprint arXiv:2503.20215, 2025

    Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Jun- yang Lin. Qwen2.5-omni technical report.arXiv preprint arXiv:2503.20215, 2025. 2, 5

    2025 arXiv

  37. [45]

    Qwen3-omni technical report.arXiv preprint arXiv:2509.17765, 2025

    Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, et al. Qwen3-omni technical report.arXiv preprint arXiv:2509.17765, 2025. 1

    2025 arXiv

  38. [46]

    Humanomniv2: From understand- ing to omni-modal reasoning with context.arXiv preprint arXiv:2506.21277, 2025

    Qize Yang, Shimin Yao, Weixuan Chen, Shenghao Fu, De- tao Bai, Jiaxing Zhao, Boyuan Sun, Bowen Yin, Xihan Wei, and Jingren Zhou. Humanomniv2: From understand- ing to omni-modal reasoning with context.arXiv preprint arXiv:2506.21277, 2025. 1

    2025 arXiv

  39. [47]

    Visionzip: Longer is better but not necessary in vision language models

    Senqiao Yang, Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia. Visionzip: Longer is better but not necessary in vision language models. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 2, 5

    2025

  40. [48]

    Audio-centric video understanding benchmark without text shortcut

    Yudong Yang, Jimin Zhuang, Guangzhi Sun, Changli Tang, Yixuan Li, Peihan Li, Yifan Jiang, Wei Li, Zejun Ma, and Chao Zhang. Audio-centric video understanding benchmark without text shortcut. InProceedings of the 2025 Confer- ence on Empirical Methods in Natural Language Processing,

    2025

  41. [49]

    Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800, 2024

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800, 2024. 1, 5

    2024 arXiv

  42. [50]

    Lmms-eval: Re- ality check on the evaluation of large multimodal models

    Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, et al. Lmms-eval: Re- ality check on the evaluation of large multimodal models. In Findings of the Association for Computational Linguistics: ...

    2025

  43. [51]

    Debiasing multimodal large language models via penal- ization of language priors

    YiFan Zhang, Yang Shi, Weichen Yu, Qingsong Wen, Xue Wang, Wenjing Yang, Zhang Zhang, Liang Wang, and Rong Jin. Debiasing multimodal large language models via penal- ization of language priors. InProceedings of the 33rd ACM International Conference on Multimedia, 2025. 2

    2025

  44. [52]

    What is this?

    Ziwei Zhou, Rui Wang, Zuxuan Wu, and Yu-Gang Jiang. Daily-omni: Towards audio-visual reasoning with temporal alignment across modalities.arXiv preprint arXiv:2505.17862, 2025. 5, 1 10 OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models Supplement...

    2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.