Pith. sign in

REVIEW 2 major objections 6 minor 36 references

Redundancy-aware visual token pruning can be done in a single forward pass, replacing 112 ms iterative selection with 2.5 ms of parallel tensor computation while retaining 99% accuracy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 03:46 UTC pith:N7I4BXWU

load-bearing objection A genuinely non-iterative token-pruning method that works well empirically, but the complexity claims overstate what's structural. the 2 major comments →

arxiv 2607.23046 v1 pith:N7I4BXWU submitted 2026-07-25 cs.CV

Structured Redundancy Modeling for Efficient Visual Token Pruning in High-Resolution MLLMs

Jouwon Song , Woohyeong Kim , Kyeongbo Kong This is my paper
classification cs.CV
keywords visual token pruningmultimodal large language modelssingle-forward passridge leverage scoredirectional maskingredundancy modelinginference accelerationhigh-resolution MLLMs
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that redundancy-aware visual token pruning need not be an iterative optimization problem. By embedding redundancy control directly into the scoring space—through a semantics-guided ridge leverage score that down-weights dominant covariance directions, and a ranking-based directional mask that suppresses redundant lower-scoring tokens in one parallel operation—selection becomes a single forward pass. The result is that token selection latency stays roughly constant (about 2.5–3.5 ms) regardless of retention budget, where previously effective methods took tens to hundreds of milliseconds. If true, this makes aggressive token reduction translate into real end-to-end speedups for high-resolution multimodal LLMs while keeping accuracy near the unpruned model.

Core claim

SFPruner's central claim is that redundancy control can be separated from combinatorial subset construction. It computes a ridge leverage score for each visual token on the feature covariance matrix (via a Woodbury-duality to keep the inversion cheap when N > D), modulates it by instruction relevance and visual saliency, then applies an asymmetric similarity mask where higher-scoring tokens suppress lower-scoring ones proportional to their cosine overlap. The final top-K selection is a single tensor operation. Across the tested high-resolution image and video MLLM families, it retains 92–100% of unpruned accuracy with 2.5–3.5 ms selection latency—compared with up to 576 ms for iterative subs

What carries the argument

Two parallel structures carry the argument: (1) the semantics-guided ridge leverage score (SG-RLS), which inverts the D×D feature covariance (or, via Woodbury duality, the N×N Gram matrix when D > N) to down-weight tokens aligned with dominant, redundant directions and multiplies by an instruction-aware saliency score; and (2) ranking-based directional masking, which constructs the N×N cosine similarity matrix and, for each token, applies a penalty based on its maximum similarity to any higher-scoring token. Together they reproduce the effect of greedy subset selection without the sequential loop.

Load-bearing premise

The efficiency claim rests on the assumption that forming and processing the N×N pairwise similarity matrix in directional masking is cheap enough that the total selection time is dominated by the D×D covariance path; if N is much larger than D, this quadratic term dominates and the claimed constant low latency at 16K tokens would not be a structural guarantee.

What would settle it

Run SFPruner on synthetic token matrices of increasing length with fixed feature dimension, e.g., D=3584 and N=1K, 2K, 4K, 8K, 16K, 32K, measuring isolated selection latency and peak GPU memory. If latency grows approximately as N² or memory grows as N², the structural claim of constant low selection cost is falsified; a roughly linear trend would confirm it. Verify that the 16K-token latency remains near the reported 50 ms on hardware comparable to an RTX 4090.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Selection cost stops scaling with the retention budget: keeping 40% or 20% of tokens takes the same 2.5 ms, so pruning can be done aggressively without extra selection overhead.
  • Theoretical FLOP reductions become real speedups: in a 9,216-token stress test, end-to-end inference time drops from 6,101 s to 3,601 s, whereas iterative methods gain almost nothing because selection overhead dominates.
  • The same single-pass structure works for multi-patch images, dense single-sequence encodings, and multi-frame video, suggesting a common pruning mechanism across MLLM architectures.
  • Training-free pruning with SG-RLS and directional masking matches or beats both lightweight heuristics and iterative optimization methods at 92–100% relative performance.
  • At 16K tokens the pruning module runs in roughly 50 ms, making long-context multimodal inference practical without sacrificing accuracy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the structural claim holds, the same 'redundancy in the scoring space' trick could be applied to other sequential selection settings (sparse attention, retrieval, memory management) where greedy loops are the bottleneck.
  • A testable extension is to vary how the directional mask multiplies the SG-RLS score—softer, learned, or rank-aware combinations might push accuracy even higher without breaking single-pass latency.
  • The O(ND²) complexity argument assumes the N×N similarity matrix is cheap to form; at very large N this quadratic memory/time term may dominate, so the constant-latency claim should be stress-tested well beyond the reported 16K tokens.
  • Because SFPruner is training-free, it could be stacked with a learned importance scorer to close the remaining accuracy gap versus full fine-tuning, while keeping the single-forward-pass latency advantage.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes SFPruner, a training-free visual token pruning method for high-resolution MLLMs. The method computes a semantic guidance score from text relevance and visual saliency, combines it with a ridge leverage score computed from the D×D feature covariance matrix, and then applies a ranking-based directional masking step using the N×N cosine similarity matrix. The final top-k selection is obtained in a single forward pass without iterative subset construction. Experiments on LLaVA-NeXT-7B, Qwen2.5-VL-7B, and LLaVA-Video-7B report competitive accuracy and substantially lower selection latency than optimization-based baselines, with ablations separating the contributions of the covariance-level and pairwise-level components.

Significance. If the efficiency claims hold, SFPruner is a useful contribution: it shows that a substantial portion of the benefit of subset-optimization pruning can be obtained with parallel tensor operations, avoiding iterative greedy loops. The benchmark coverage is broad, the ablations in Table 5 cleanly isolate the two mechanisms, and the paper includes a code link, which strengthens reproducibility. The method is a heuristic scoring function validated on held-out benchmarks, so there is no circularity concern. However, the advertised 'structural scalability' is not established by the complexity analysis as written, because the N×N similarity matrix used by directional masking is omitted from the O(ND^2) claim. The empirical latency numbers may still be valid, but the explanation of why the method scales is incomplete and needs correction before the central efficiency claim can be accepted.

major comments (2)
  1. [Sec. 4.6 vs Eqs. (8)-(9)] The stated O(ND^2) complexity is incomplete. The D×D covariance path with Cholesky is O(ND^2 + D^3), and the directional masking step explicitly constructs C_sim = V V^T in R^{N×N}, which is O(N^2D) compute and O(N^2) memory, followed by an O(N^2) row-wise max. At N=16,384, D=3,584 this is ~268M entries and roughly 10^12 MACs. The retention-invariant latency is consistent with fixed N, but it does not demonstrate linear scaling in N; the 50 ms at 16K is an empirical GPU measurement, not a structural complexity result. Please report the full complexity, add N-scaling benchmarks for the complete pruning module, or replace the global Gram matrix with a block-local or top-m approximation before claiming structural scalability.
  2. [Sec. 4.1 and Eqs. (1), (3), (6)] The hyperparameters alpha, lambda, and tau are not reported in the main text; the paper defers to 'the supplementary material', which is not included in the arXiv submission. These parameters define the semantic guidance fusion, the ridge regularization, and the sharpness of the relevance distribution, so the benchmark results (e.g., 99.0% relative performance in Table 2) are not reproducible without them. Please provide the chosen values and at least a brief sensitivity analysis in the main text, or make the supplementary material available.
minor comments (6)
  1. [References / Sec. 4.1] The MME benchmark is cited as [28], but reference [28] is 'A survey on multimodal large language models'. The MME benchmark should be cited to the original MME paper.
  2. [References / Sec. 4.4] Reference [34], cited for LLaVA-Video, appears to have an incorrect title ('Video instruction tuning with synthetic data'). Please verify and cite the correct LLaVA-Video technical report.
  3. [Eq. (8)] The indicator M_ij = I(S_SG-RLS,j > S_SG-RLS,i) uses a strict inequality; ties in the score are left unspecified. Please define a tie-breaking rule or add epsilon to the comparison.
  4. [Sec. 3.1] For architectures without a text encoder (e.g., Qwen2.5-VL), the paper states that only visual saliency is used, so Eq. (3) effectively reduces to S_attn. Please clarify whether alpha is set to 0 in this case and how the 'semantics-guided' terminology applies.
  5. [Fig. 3] The caption says bar charts indicate total latency while the text describes pruning overhead. Please distinguish selection-module latency from end-to-end latency more clearly in the figure and caption.
  6. [Table 4] The 'Infer Time (sec)' column is ambiguous. Clarify whether this is total dataset inference time or average per-sample time, and state the batch size used for profiling.

Circularity Check

0 steps flagged

No material circularity; the internal N^2 complexity gap in Sec 4.6 is a correctness/scalability concern, not a derivation-from-inputs issue.

full rationale

I walked the derivation chain: base importance comes from text/visual similarities (Eqs. 1-3), covariance-level redundancy from the ridge-leverage inverse covariance (Eqs. 5-6), pairwise suppression from the asymmetric similarity mask (Eqs. 8-10), and the final subset is a plain top-k operation. Each stage is a concrete deterministic function of the stated inputs; no equation is defined in terms of the output, and no reported accuracy is a fitted parameter renamed as a prediction. Hyperparameters alpha, lambda, and tau are not given in the main text, but the paper does not claim to predict benchmark scores from a fitted model, and there is no evidence the reported benchmark numbers were used to set these parameters; even parameter tuning would not by itself make the derivation circular unless the same data were used as both fit and prediction. The two self-citations ([3] and [20]) are motivational or comparative: [3] is cited for the empirical attention-vs-diversity dichotomy and [20] is cited as one example of an MWIS-based iterative method. Neither is a uniqueness theorem, neither is invoked to forbid alternatives, and the central redundancy-control formulation does not reduce to either citation. The most significant issue found is an internal complexity inconsistency: Sec. 4.6 claims O(ND^2) via the Woodbury identity for the RLS path, but Sec. 3.3 explicitly constructs the N x N Gram matrix C_sim = V V^T (Eqs. 8-9), which is O(N^2 D) time and O(N^2) memory and is not reflected in the Sec. 4.6 analysis. This undermines the asymptotic scalability claim, but it is not circularity: the measured latencies and benchmark accuracies are external empirical evidence, and the O(ND^2) claim is not used to define the method's output or to manufacture its reported predictions. Accordingly, no circular step meets the quoted-evidence threshold, and the appropriate finding is no significant circularity (score 0).

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 0 invented entities

The central method rests on three tuned hyperparameters (α, λ, τ) and five domain heuristics: RLS-as-redundancy, cosine relevance, attention saliency, multiplicative gating, and max-similarity masking. None of these are derived from first principles, but they are standard modeling choices in the pruning literature. The paper does not introduce new physical or conceptual entities beyond the scoring mechanisms themselves.

free parameters (3)
  • α = not reported in main text (supplementary)
    Linear fusion weight in Eq (3), balancing textual relevance and visual saliency. Tuned, not derived.
  • λ = not reported in main text (supplementary)
    Ridge regularization constant in Eq (6), controlling attenuation of dominant covariance directions. Sensitivity deferred to supplementary.
  • τ = not reported in main text (supplementary)
    Temperature in Eq (1), sharpening the textual-relevance softmax distribution. Tuned, not derived.
axioms (6)
  • domain assumption Ridge leverage score ℓ_i = v_i^T(C+λI)^{-1}v_i is a valid measure of global redundancy/uniqueness of visual tokens.
    Invoked in Sec 3.2 to attenuate dominant covariance directions; relies on [17,16] but the mapping from RLS to task-relevant visual redundancy is assumed.
  • domain assumption Cosine similarity to the text embedding measures instruction relevance.
    Eq (1) assumes L2-normalized embeddings and temperature scaling preserve semantic relevance ordering for token pruning.
  • domain assumption CLS attention, or aggregated attention across spatial tokens when no CLS exists, is a valid visual saliency signal.
    Sec 3.1 constructs a proxy saliency for Qwen2.5-VL by aggregating self-attention weights; its fidelity to true saliency is unvalidated outside the reported benchmarks.
  • ad hoc to paper Multiplicative gating ℓ_i · S_guide,i implements a soft-AND of structural uniqueness and semantic relevance.
    Eq (7) is a design choice, not derived; it down-weights tokens lacking either property, but the exact product form is not justified.
  • ad hoc to paper Directional masking with a single max-similarity penalty approximates the pairwise exclusion of sequential subset selection.
    Eqs (8)-(10) replace iterative state updates with one parallel penalty; the ablation shows 97.9% vs 98.0% relative performance, so the approximation is imperfect.
  • standard math Woodbury identity allows inverting the smaller of the N×N Gram matrix and the D×D covariance matrix.
    Stated in Sec 3.2 and used in Sec 4.6; standard linear algebra, but the derivation is deferred to the supplementary.

pith-pipeline@v1.3.0-alltime-deepseek · 13735 in / 13932 out tokens · 145098 ms · 2026-08-01T03:46:03.142717+00:00 · methodology

0 comments
read the original abstract

Recent high-resolution Multimodal Large Language Models (MLLMs) generate thousands of visual tokens per input, leading to a visual token explosion that introduces severe latency bottlenecks. While token pruning mitigates this issue, state-of-the-art subset-optimization methods typically rely on iterative subset construction to jointly capture visual diversity and instruction relevance. As visual token counts scale, this sequential dependency introduces significant selection overhead, severely limiting the translation of theoretical FLOPs reductions into actual wall-clock speedups. To address this limitation, we propose Single-Forward Pruner (SFPruner), a structural reformulation of visual token pruning that embeds redundancy control directly into the scoring space, bypassing the need for iterative combinatorial optimization. Our non-iterative framework achieves redundancy-aware importance selection in a single forward pass through two complementary mechanisms. First, to attenuate redundancy at the covariance level, we introduce a semantics-guided ridge leverage scheme. By integrating instruction relevance and visual saliency, this mechanism suppresses dominant covariance directions and mitigates representation bias. Second, ranking-based directional masking resolves residual overlap through asymmetric similarity competition, where higher-scoring tokens explicitly suppress redundant lower-scoring alternatives via parallel tensor operations. Extensive evaluations demonstrate that our approach maintains stable selection costs, reducing the token selection process by up to 110 ms, from 112.4 ms to just 2.5 ms at 512 tokens in Qwen2.5-VL. This structural efficiency successfully translates theoretical token reductions into tangible inference speedups while preserving highly competitive performance against state-of-the-art techniques under aggressive compression.

Figures

Figures reproduced from arXiv: 2607.23046 by Jouwon Song, Kyeongbo Kong, Woohyeong Kim.

Figure 1
Figure 1. Figure 1: Conceptual comparison of token selection frameworks. (a) Heuris￾tic (Top-K) relies on isolated scores, lacking structural redundancy control. (b) Traditional sequential search mitigates redundancy but introduces iterative de￾pendencies. (c) Our framework enables efficient single-pass selection via SG-RLS and directional masking, eliminating iteration overhead. ages and long video sequences. Architectures l… view at source ↗
Figure 2
Figure 2. Figure 2: Pipeline of the non-iterative token pruning framework. (Top) Semantics-Guided RLS (SG-RLS): The semantic guidance (Sguide) is mod￾ulated by the leverage score map (ℓ) to suppress redundant background features, yielding a covariance-aware score (SSG-RLS). (Bottom) Ranking-Based Di￾rectional Masking: A ranking mask (M) is applied to the pairwise similarity matrix. Tokens receive a directional suppression pen… view at source ↗
Figure 3
Figure 3. Figure 3: End-to-end latency and peak memory analysis across vary￾ing numbers of input tokens (N). While existing token pruning meth￾ods (DivPrune, CDPruner) incur severe time overheads as sequence length in￾creases, ours maintains exceptionally low overhead even at a high resolution of N = 16384. It closely approaches the pure inference time of the 16K baseline (1,531 ms), delivering practical computational acceler… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 9 linked inside Pith

  1. [1]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Alvar, S.R., Singh, G., Akbari, M., Zhang, Y.: Divprune: Diversity-based visual token pruning for large multimodal models. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 9392–9401 (2025)

    2025

  2. [2]

    In: Proceedings of the AAAI Conference on Artificial In- telligence

    Arif, K.H.I., Yoon, J., Nikolopoulos, D.S., Vandierendonck, H., John, D., Ji, B.: Hired: Attention-guided token dropping for efficient inference of high-resolution vision-language models. In: Proceedings of the AAAI Conference on Artificial In- telligence. vol. 39, pp. 1773–1781 (2025)

    2025

  3. [3]

    In: The Fourteenth International Conference on Learning Representations (2026), https://openreview.net/forum?id=2NLkhPex1M

    Baek, C., Song, J., Kim, S., Kong, K.: Agilepruner: An empirical study of attention and diversity for adaptive visual token pruning in large vision-language models. In: The Fourteenth International Conference on Learning Representations (2026), https://openreview.net/forum?id=2NLkhPex1M

    2026

  4. [4]

    5-vl technical report

    Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., et al.: Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923 (2025)

    Pith/arXiv arXiv 2025

  5. [5]

    In: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Bolya, D., Fu, J., Dai, X., Feiszli, M.: Token merging: Your ViT but faster. In: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 4553–4563 (2023)

    2023

  6. [6]

    In: European Conference on Computer Vision

    Chen, L., Zhao, H., Liu, T., Bai, S., Lin, J., Zhou, C., Chang, B.: An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. In: European Conference on Computer Vision. pp. 19–35. Springer (2024)

    2024

  7. [7]

    In: CVPR (2025)

    Fu, C., Dai, Y., Luo, Y., Li, L., Ren, S., Zhang, R., Wang, Z., Zhou, C., Shen, Y., Zhang, M., et al.: Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. In: CVPR (2025)

    2025

  8. [8]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., Parikh, D.: Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 6904–6913 (2017)

    2017

  9. [9]

    In: Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition

    Hudson, D.A., Manning, C.D.: Gqa: A new dataset for real-world visual reasoning and compositional question answering. In: Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition. pp. 6700–6709 (2019)

    2019

  10. [10]

    In: The Thirteenth International Conference on Learning Representations (2025),https: //openreview.net/forum?id=8EfxjTCg2k

    Lin, C.H., Gao, S., Smith, J.S., Patel, A., Tuli, S., Shen, Y., Jin, H., Hsu, Y.C.: ModeGPT: Modular decomposition for large language model compression. In: The Thirteenth International Conference on Learning Representations (2025),https: //openreview.net/forum?id=8EfxjTCg2k

    2025

  11. [11]

    arXiv preprint arXiv:2412.19437 (2024)

    Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al.: Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 (2024)

    Pith/arXiv arXiv 2024

  12. [12]

    Liu, H., Li, C., Li, Y., Li, B., Zhang, Y., Shen, S., Lee, Y.J.: Llavanext: Improved reasoning, ocr, and world knowledge (2024)

    2024

  13. [13]

    Advances in neural information processing systems36, 34892–34916 (2023)

    Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. Advances in neural information processing systems36, 34892–34916 (2023)

    2023

  14. [14]

    Advances in Neural Information Processing Systems 35, 2507–2521 (2022)

    Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K.W., Zhu, S.C., Tafjord, O., Clark, P., Kalyan, A.: Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems 35, 2507–2521 (2022)

    2022

  15. [15]

    Mahoney, M.W.: Randomized algorithms for matrices and data. Found. Trends Mach. Learn.3(2), 123–224 (Feb 2011).https://doi.org/10.1561/2200000035, https://doi.org/10.1561/2200000035 Structured Redundancy Modeling for Efficient Visual Token Pruning 17

    doi:10.1561/2200000035 2011

  16. [16]

    In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R

    McCurdy, S.: Ridge regression and provable deterministic ridge leverage score sam- pling. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 31. Curran Associates, Inc. (2018),https://proceedings.neurips.cc/paper_files/ paper/2018/file/e1d5be1c7f2f456670de3...

    2018

  17. [17]

    In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R

    Musco, C., Musco, C.: Recursive sampling for the nystrom method. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017),https://proceedings.neurips.cc/paper_files/paper/ 2017/file/a03fa30821986dff10fc66647c84c9c3-Paper.pdf

    2017

  18. [18]

    In: ICCV (2025)

    Shang, Y., Cai, M., Xu, B., Lee, Y.J., Yan, Y.: Llava-prumerge: Adaptive token reduction for efficient large multimodal models. In: ICCV (2025)

    2025

  19. [19]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Singh, A., Natarajan, V., Shah, M., Jiang, Y., Chen, X., Batra, D., Parikh, D., Rohrbach, M.: Towards vqa models that can read. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8317–8326 (2019)

    2019

  20. [20]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings

    Song, J., Kim, S., Kong, K.: Uncertainty-guided graph formulation via mwis for token pruning in lvlms. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings. pp. 9510–9519 (June 2026)

    2026

  21. [21]

    In: The Fourteenth International Conference on Learning Representations (2026),https://openreview.net/forum?id=0zIcPe4CtY

    Song, Z., Xue, J., Zhang, J., Zhang, L.: Sublinear time quantum algorithm for attention approximation. In: The Fourteenth International Conference on Learning Representations (2026),https://openreview.net/forum?id=0zIcPe4CtY

    2026

  22. [22]

    arXiv preprint arXiv:2409.12191 (2024)

    Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., et al.: Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191 (2024)

    Pith/arXiv arXiv 2024

  23. [23]

    arXiv preprint arXiv:2502.11494 (2025)

    Wen, Z., Gao, Y., Wang, S., Zhang, J., Zhang, Q., Li, W., He, C., Zhang, L.: Stop looking for important tokens in multimodal language models: Duplication matters more. arXiv preprint arXiv:2502.11494 (2025)

    Pith/arXiv arXiv 2025

  24. [24]

    Wu, H., Li, D., Chen, B., Li, J.: Longvideobench: A benchmark for long-context interleaved video-language understanding (2024),https://arxiv.org/abs/2407. 15754

    2024

  25. [25]

    arXiv preprint arXiv:2410.17247 (2024)

    Xing, L., Huang, Q., Dong, X., Lu, J., Zhang, P., Zang, Y., Cao, Y., He, C., Wang, J., Wu, F., et al.: Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy reduction. arXiv preprint arXiv:2410.17247 (2024)

    Pith/arXiv arXiv 2024

  26. [26]

    In: Proceedings of the Com- puter Vision and Pattern Recognition Conference

    Yang, S., Chen, Y., Tian, Z., Wang, C., Li, J., Yu, B., Jia, J.: Visionzip: Longer is better but not necessary in vision language models. In: Proceedings of the Com- puter Vision and Pattern Recognition Conference. pp. 19792–19802 (2025)

    2025

  27. [27]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Ye, X., Gan, Y., Ge, Y., Zhang, X.P., Tang, Y.: Atp-llava: Adaptive token pruning for large vision language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 24972–24982 (2025)

    2025

  28. [28]

    National Science Review11(12), nwae403 (2024)

    Yin, S., Fu, C., Zhao, S., Li, K., Sun, X., Xu, T., Chen, E.: A survey on multimodal large language models. National Science Review11(12), nwae403 (2024)

    2024

  29. [29]

    In: Proceed- ings of the AAAI Conference on Artificial Intelligence (2026)

    Zhang, E., Yu, F., Wu, A., Wen, Z., Yan, K., Ding, S., Qi, B., Zhang, L.: D²pruner: Debiased importance and structural diversity for mllm token pruning. In: Proceed- ings of the AAAI Conference on Artificial Intelligence (2026)

    2026

  30. [30]

    arXiv preprint arXiv:2412.01818 (2024)

    Zhang, Q., Cheng, A., Lu, M., Zhang, R., Zhuo, Z., Cao, J., Guo, S., She, Q., Zhang, S.: Beyond text-visual attention: Exploiting visual cues for effective token pruning in vlms. arXiv preprint arXiv:2412.01818 (2024)

    Pith/arXiv arXiv 2024

  31. [31]

    arXiv e-prints pp

    Zhang, Q., Cheng, A., Lu, M., Zhuo, Z., Wang, M., Cao, J., Guo, S., She, Q., Zhang, S.: [cls] attention is all you need for training-free visual token pruning: Make vlm inference faster. arXiv e-prints pp. arXiv–2412 (2024) 18 J. Song et al

    2024

  32. [32]

    arXiv preprint arXiv:2506.10967 (2025)

    Zhang, Q., Liu, M., Li, L., Lu, M., Zhang, Y., Pan, J., She, Q., Zhang, S.: Be- yond attention or similarity: Maximizing conditional diversity for token pruning in mllms. arXiv preprint arXiv:2506.10967 (2025)

    Pith/arXiv arXiv 2025

  33. [33]

    In: International Conference on Machine Learning (2025)

    Zhang, Y., Fan, C.K., Ma, J., Zheng, W., Huang, T., Cheng, K., Gudovskiy, D., Okuno, T., Nakata, Y., Keutzer, K., et al.: Sparsevlm: Visual token sparsifica- tion for efficient vision-language model inference. In: International Conference on Machine Learning (2025)

    2025

  34. [34]

    Zhang, Y., Wu, J., Li, W., Li, B., Ma, Z., Liu, Z., Li, C.: Video instruction tuning with synthetic data (2024),https://arxiv.org/abs/2410.02713

    Pith/arXiv arXiv 2024

  35. [35]

    arXiv preprint arXiv:2406.04264 (2024)

    Zhou, J., Shu, Y., Zhao, B., Wu, B., Xiao, S., Yang, X., Xiong, Y., Zhang, B., Huang, T., Liu, Z.: Mlvu: A comprehensive benchmark for multi-task long video understanding. arXiv preprint arXiv:2406.04264 (2024)

    Pith/arXiv arXiv 2024

  36. [36]

    Zhu, J., Zhu, Y., Lu, X., Yan, W., Li, D., Liu, K., Fu, X., Zha, Z.J.: Visionselector: End-to-end learnable visual token compression for efficient multimodal llms (2025), https://arxiv.org/abs/2510.16598

    arXiv 2025