Pith. sign in

REVIEW 4 major objections 5 minor 32 references

Not All Redundant Tokens Are Alike: Analyzing Visual Token Pruning through Token Roles

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Dead image tokens are not always safe to prune.

desk verdict A useful probe and a real confound: the paper's role-protected pruning experiment can't separate preserving dead tokens from changing which other tokens get pruned, so the headline claim is plausible but not yet established. read the letter →

arxiv 2608.04483 v1 pith:GEBRLV5M submitted 2026-08-05 cs.CV

Hyeonyu Kim , Sehwan Lim , Youngwon Choi , Taeyoun Kwon , Jaejin Kim This is my paper
classification cs.CV
keywords visualtokenpruningvision-languagemodelsrolesEmbedLensalivetokensdeadsinkrole-protected
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vision-language models treat an image as a long sequence of visual tokens, and inference cost grows with that sequence, so pruning methods remove tokens that look redundant. This paper asks whether the functional role a token plays, alive, sink, or dead, as classified by the EmbedLens embedding-space analysis, tells you which tokens are safe to remove. Across three pruning methods and ten benchmarks, the authors show that each method has a distinct role bias, but that bias does not track downstream accuracy. When they protect one role from pruning under a fixed budget, preserving alive tokens improves performance on average, while preserving non-alive tokens, including dead tokens, often maintains or improves performance. The practical upshot is that a token's role label alone is not a reliable guide to dispensability; the composition of the retained token set matters.

What carries the argument

The machinery is the EmbedLens token-role taxonomy, which sorts projected visual tokens into four groups in the LLM input-embedding space: alive tokens (closest to text semantics and image-specific), ViT sink tokens (large norm in the vision encoder, stable centroid), LLM sink tokens (high activity on a small set of sink dimensions), and dead tokens (the largest, highly homogeneous cross-image cluster). The authors' refinement compares each visual token directly to stored cluster centroids, the three non-alive centroids plus a new alive-token centroid $C_A$, instead of comparing to vocabulary reference token IDs, which misclassified many sink and alive tokens. The functional role of each token is then probed by role-protected pruning: at a fixed budget, tokens of one role are excluded from the pruning candidate set, isolating whether that role contributes to accuracy. Two attention metrics close the argument: group-level attention mass $\mathrm{Mass}^{(\ell)}(r,s)$ and per-pair attention score $\mathrm{Pair}^{(\ell)}(r,s)$, which separate the effect of group size from the effect of per-token interaction strength.

What would settle it

Run the same role-protected experiment while holding the other pruned tokens fixed, e.g., randomly resampling which non-protected tokens are removed at the same budget; if preserving dead tokens only helps when the rest of the retained set is reshuffled, and never helps when the rest of the set is fixed, the claim that dead tokens contribute under pruning would not be supported. Also, a single benchmark where protecting dead tokens consistently degrades accuracy across all three methods at multiple budgets would count against the paper's generalization.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the functional roles of visual tokens in a vision-language model do not translate directly into optimal pruning decisions. FastV, DART, and DivPrune each prune tokens with a different mix of alive, sink, and dead roles across token budgets from 12.5% to 87.5% on ten benchmarks, yet these role biases do not predict which method performs best. To make role labels trustworthy, the authors refine EmbedLens's text-reference-ID assignment into a centroid-based assignment that also stores an alive-token centroid, which matches cluster-derived roles much more closely. With that assignment, role-protected pruning experiments at a 50% budget show that preserving alive tokens consistently improves accuracy by about 1% or more on average, while preserving dead tokens, the supposedly most redundant category, keeps or improves performance on most benchmarks, and protecting dead tokens under DivPrune gives an average gain of 0.93%. Attention analysis explains part of this: dead tokens have low per-token attention but, because they form the largest group, they account for a non-negligible share of group-level attention mass, so removing them changes how the remaining token roles interact.

Load-bearing premise

The conclusions rest on the assumption that the refined token-role labels are a valid, transferable grouping of visual tokens, and that the accuracy changes observed when protecting a role are caused by that role rather than by the incidental reshuffling of which other tokens get pruned.

Editorial extensions

If this is right

  • Pruning methods should report which token roles they remove, because the role mix can change substantially across methods and budgets even when accuracy does not.
  • Protecting alive tokens from pruning is a reliable, benchmark-agnostic safeguard, giving consistent accuracy gains across all three methods.
  • Treating dead or sink tokens as uniformly removable is unsafe: preserving them can hold or improve accuracy under a fixed budget, notably for DivPrune.
  • Pruning design should consider the composition of the retained token set and the group-level attention budget of each role, not only per-token saliency.
  • Attention-based importance and role labels can diverge: sink tokens receive high attention in early LLM layers despite weak semantic alignment, so attention maps alone do not reveal role utility.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper does not run: impose explicit role-balance constraints in a pruner, e.g., retain a minimum share of dead tokens, and compare against unconstrained pruning; if the paper's mechanism is right, balanced pruning should win at the same budget.
  • The result suggests a cheap diagnostic: monitoring role-wise retention rates during inference could flag when a pruning pass is about to hurt accuracy, even before evaluating on the task.
  • The group-level attention story points to a structural role for dead tokens that is compatible with attention-sink phenomena: their mass may soak up attention budget rather than carry semantics; testing this on other backbones and budgets would show whether the effect generalizes.
  • If the role taxonomy is not portable across VLM backbones, the empirical pattern could be specific to the LLaVA-v1.5-7B architecture; checking whether the alive-token centroid stays stable on other architectures would settle that.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper studies how visual token pruning in LLaVA-v1.5-7B interacts with the token-role taxonomy (alive, ViT sink, LLM sink, dead) proposed by EmbedLens. It analyzes the role composition of pruned sets for FastV, DART, and DivPrune over ten benchmarks, reports that each method has a distinct role bias that does not by itself predict performance, and proposes a refined centroid-based role assignment that adds an alive centroid. At a fixed 50% budget, role-protected pruning experiments indicate that protecting alive tokens helps, while protecting non-alive tokens, especially dead tokens, can sometimes maintain or improve performance. The paper attributes this to group-level attention mass of dead tokens and concludes that token-role labels alone are not a reliable pruning criterion.

Significance. If the central claim is established, this is a useful negative result: a token's EmbedLens role is not a sufficient statistic for pruning decisions, and the composition of the retained set matters. The paper's strengths are its systematic evaluation over ten public benchmarks, its public code release, and the fact that role labels are fixed by EmbedLens before the pruning experiments, so the central comparison is not circular. The descriptive attention analysis in Section 3.5 also carefully separates group-level attention mass from per-token interaction strength. However, the headline conclusion rests on small benchmark deltas from a single backbone, a confounded intervention, and an unquantified role-assignment refinement, so the result should be treated as a hypothesis generator rather than a settled finding.

major comments (4)
  1. [Section 3.4, Table 1] The role-protected pruning experiment does not identify the effect of preserving a token role. At a fixed 50% budget, excluding the protected role from the pruning candidate set forces the method to prune a different set of other tokens, so any performance change conflates preserving the protected role with a reshuffle of which other tokens are retained. For DivPrune, dead tokens are the largest role group, so protecting dead tokens forces more alive and sink tokens to be pruned; the +0.93% average gain in Table 1 could reflect a better composition of the remaining non-dead tokens under the greedy diversity objective rather than the preservation of dead tokens specifically. The paper does not include a control that protects a randomly selected set of tokens matched in size to the protected role, nor does it show that restoring the same number of randomly chosen tokens that the original method pruned produces no gain. Without such a control, the causal reading 'preserving non-alive tokens can sometimes improve performance' (Section 3.4) is not identified. The authors should also specify exactly how the 50% budget is enforced when a large protected role is excluded, and confirm that the protected role is never pruned in the reported runs.
  2. [Section 3.3] The refined centroid-based assignment with an alive centroid is validated only by t-SNE visualizations and centroid cosine similarities. No agreement rate, confusion matrix, or per-image accuracy against the cluster-derived token-role labels on COCO-10K is reported, so the degree of label noise in the role assignment used throughout Section 3.4 is unknown. Since all downstream role-protected conclusions depend on these labels, the missing quantitative validation is load-bearing. The authors should report the agreement between the centroid-based variants (with and without C_A) and the cluster-derived assignment, and should test whether the COCO-derived centroids transfer to the ten evaluation benchmarks instead of assuming transferability. The added alive centroid C_A is an assumption that also needs justification: a single global centroid may not adequately represent image-specific alive tokens across datasets.
  3. [Table 1] The headline deltas are small, relative changes from single evaluation runs with no confidence intervals or significance tests. For example, DivPrune with dead-token protection has a +0.93% macro-average gain, but individual benchmark changes range from -1.82% (TextVQA) to +6.12% (MMBench), and several values are below 0.5%; many FastV and DART entries are exactly 0.00% because those roles were already never pruned by the original method, so those entries carry no information. The claim that preserving non-alive tokens 'can sometimes improve' performance is therefore not clearly distinguished from evaluation noise or a single favorable benchmark. The authors should report variance estimates (e.g., multiple seeds, bootstrap over samples, or paired significance tests) and should separate uninformative zero-delta entries from actual effects.
  4. [Section 3.2 and Conclusion] All experiments are conducted on a single backbone, LLaVA-v1.5-7B, yet the abstract and conclusion make general claims about 'visual token pruning' and 'token roles'. The Discussion acknowledges that additional backbones are needed, but the central claim as stated is not supported beyond this one architecture. At minimum, the abstract and title should be scoped to the tested backbone, or the authors should add a replication on a second model family (e.g., a Qwen2-VL- or LLaVA-NeXT-based VLM) to show that the role-protection effects are not an artifact of this particular embedding space.
minor comments (5)
  1. [Section 3.1, Eq. (1)] Specify the value of k and the tie-breaking rule for the TopK operation; the role-assignment procedure appears to use k=1, but the general definition in Eq. (1) is left incomplete.
  2. [Figure 1] State explicitly whether the 'token budget' on the x-axis is the retained fraction or the pruned fraction; the caption currently leaves this ambiguous, and the number of pruned tokens increases as the budget decreases.
  3. [Table 1] Report absolute scores alongside the relative deltas; the relative-change formatting can make small absolute differences look larger, for example the +6.12% relative change on MMBench corresponds to an absolute gain of about 4.55 points, while other entries are under one point.
  4. [Section 3.5] The observation that visual-visual token pairs exhibit larger group-level attention mass than visual-textual pairs is asserted from figures without numeric support; include quantitative values in the text or a table.
  5. [Section 3.5 Discussion] Fix the duplicated word in 'the the attention patterns' and add a citation for the EmbedLens claim repeated in Section 3.2 that removing non-alive tokens does not degrade performance.

Circularity Check

1 steps flagged · score 2.0 of 10

No significant circularity; the only self-referential element is the validation of the refined assignment against the same cluster-derived labels, and it is not load-bearing for the empirical pruning results.

  1. self definitional [Section 3.3, paragraph introducing the centroid-based assignment with an alive centroid]
    "In addition to the non-alive centroids, we store an alive-token centroid C_A and classify a visual token as alive not only when it is closest to the text manifold, but also when it is closest to C_A. As shown in Fig. 2 (right, bottom right), this variant produces token role assignments that are much closer to the cluster-derived role assignments."

    The alive-token centroid C_A is computed from the cluster-derived role assignments that are used as the reference for the comparison. A nearest-centroid classifier that adds C_A will, by construction, reclassify many tokens previously labeled alive under the same cluster-derived scheme, so the reported agreement is a tautological consistency check rather than an independent confirmation. This step supports the choice of assignment variant but does not enter the downstream role-protected pruning experiments, whose performance numbers are measured independently; hence the circularity is minor and not load-bearing.

full rationale

The paper's central claims—pruning methods have distinct token-role biases, these biases do not directly track performance, and role-protected pruning changes performance—are empirical results over ten benchmarks with fixed pruning budgets. The role taxonomy is imported from EmbedLens [6] and operationalized with thresholds from prior work; no performance parameter is fitted to the role definitions. The role-protected comparisons at a fixed 50% budget do change which other tokens are pruned when one role is protected, so the causal attribution of the performance delta to the protected role is not fully identified, but this is a confound in the experimental design, not a circular derivation: the numbers are measured, not implied by the definitions. The only self-referential element is the validation of the refined centroid-based assignment against the same cluster-derived labels that define the alive centroid, which does not affect the downstream pruning results. No load-bearing self-citation chain is present; citations to [6] and [12] supply external definitions and are not used to forbid alternatives. Overall circularity is low.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities beyond a methodological refinement: an alive-token centroid added to the role assignment procedure. This centroid is a data-derived statistical reference, not a fitted parameter tuned to downstream performance, so it is treated as an ad hoc axiom rather than a free parameter or invented entity.

assumptions (3)
  • domain assumption EmbedLens token roles (dead, sink, alive) are functionally meaningful categories for visual tokens in VLMs.
    The whole analysis rests on the premise from Section 3.1 that these clusters correspond to real functional differences; if they do not, the role-bias and role-protection results lose meaning.
  • ad hoc to paper The added alive-token centroid C_A gives a valid transferable reference for labeling alive tokens.
    Introduced in Section 3.3 to correct misclassification; its suitability is shown only with t-SNE and cross-image cosine similarity, not with downstream validation. It is applied uniformly across all benchmarks.
  • domain assumption Cluster-derived role labels are an appropriate ground truth for evaluating the refined assignment.
    In Section 3.3 the cluster-derived assignment is used as the reference to claim the refined variant is closer, but no quantitative agreement metric is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Not All Redundant Tokens Are Alike: Analyzing Visual Token Pruning through Token Roles." pith.science (2026). https://pith.science/paper/GEBRLV5M

@misc{pith2026260804483,
  author       = {Pith},
  title        = {Pith review of: Not All Redundant Tokens Are Alike: Analyzing Visual Token Pruning through Token Roles},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GEBRLV5M}},
  note         = {Machine review of arXiv:2608.04483}
}
read the original abstract

Vision-language models (VLMs) process an image as a sequence of visual tokens, which creates a substantial computational bottleneck during inference. Recent visual token pruning methods address this issue by removing seemingly redundant tokens, yet it remains unclear how these pruning decisions relate to the functional roles of visual tokens. In this work, we analyze visual token pruning through the lens of token roles identified by EmbedLens. We first show that representative pruning methods exhibit distinct token-role biases, but these biases do not directly correlate with downstream performance. To better understand this behavior, we refine the token-role assignment procedure and evaluate role-protected pruning variants. Our results show that preserving non-alive tokens can sometimes maintain or improve performance, suggesting that tokens with weak direct semantic alignment may still affect model behavior under pruning. Our code is publicly available at https://github.com/jaykim9870/Not_All_Redundant_Tokens_Are_Alike.

Figures

Figures reproduced from arXiv: 2608.04483 by the authors.

Figure 1
Figure 1. Token-category distribution and downstream performance under different token budgets. For FastV, DART, and DivPrune, we report the composition of pruned tokens over four token categories and the corresponding performance across ten benchmarks. The x-axis denotes the token budget, while the y-axis indicates either the number of pruned tokens in each category or the downstream task performance. token categories. We re… view at source ↗
Figure 2
Figure 2. Visualization of token role assignment strategies in EmbedLens analysis. The left panel illustrates the gap between centroid-level text-ID matching and token-level role assignment, while the right panels show t-SNE visualizations of token roles on COCO-10K under different assignment strategies. Interestingly, this tendency remains consistent across different benchmarks. Although FastV incorporates text instructions … view at source ↗
Figure 3
Figure 3. Layer-wise group-level attention mass under three input settings: the original visual token sequence (top row), alive-token pruning (middle row), and dead-token pruning (bottom row). groups. For the ℓ-th LLM layer, let A(ℓ) be the head-averaged self-attention matrix. We define the attention mass from role r to role s as \mathrm {Mass}^{(\ell )}(r,s) = \frac {1}{|\mathcal {I}_r|} \sum _{i \in \mathcal {I}_r} \sum _{j… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Layer-wise group-level attention mass under three input settings: the original visual token sequence (top row), DivPrune (middle row), and DivPrune with dead￾token protection (bottom row). attention patterns among the remaining token groups remain largely similar to th…
Figure 5
Figure 5. Figure 5: Layer-wise pairwise attention score under three input settings: the original visual token sequence (top row), DivPrune (middle row), and DivPrune with dead￾token protection (bottom row). strength between individual token pairs. We report both metrics to analyze whether…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 13 canonical work pages

  1. [1]

    Alvar, S.R., Singh, G., Akbari, M., Zhang, Y.: Divprune: Diversity-based visual tokenpruningforlargemultimodalmodels.In:ProceedingsoftheComputerVision and Pattern Recognition Conference. pp. 9392–9401 (2025)

    2025

  2. [2]

    arXiv preprint arXiv:2303.08112 (2023)

    Belrose, N., Ostrovsky, I., McKinney, L., Furman, Z., Smith, L., Halawi, D., Bi- derman, S., Steinhardt, J.: Eliciting latent predictions from transformers with the tuned lens. arXiv preprint arXiv:2303.08112 (2023)

    arXiv 2023

  3. [3]

    Bolya, D., Fu, C.Y., Dai, X., Zhang, P., Feichtenhofer, C., Hoffman, J.: Token Merging: Your ViT But Faster (2023)

    work page 2023

  4. [4]

    In: Computer Vision – ECCV 2024, pp

    Chen, L., Zhao, H., Liu, T., Bai, S., Lin, J., Zhou, C., Chang, B.: An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models. In: Computer Vision – ECCV 2024, pp. 19–35 (2025)

    work page 2025

  5. [5]

    Chu, X., Qiao, L., Lin, X., Xu, S., Yang, Y., Hu, Y., Wei, F., Zhang, X., Zhang, B., Wei, X., Shen, C.: MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices (2023)

    work page 2023

  6. [6]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Fan,Y.,Tong,J.,Zhao,A.,Shen,X.:Whatdovisualtokensreallyencode?uncover- ing sparsity and redundancy in multimodal large language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11987–11997 (2026)

    work page 2026

  7. [7]

    Advances in Neural Information Processing Systems38(2026)

    Fu, C., Chen, P., Shen, Y., Qin, Y., Zhang, M., Lin, X., Yang, J., Zheng, X., Li, K., Sun, X., et al.: Mme: A comprehensive evaluation benchmark for multimodal large language models. Advances in Neural Information Processing Systems38(2026)

    work page 2026

  8. [8]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., Parikh, D.: Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 6904–6913 (2017)

    2017

Show all 32 references
  1. [9]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Gurari, D., Li, Q., Stangl, A.J., Guo, A., Lin, C., Grauman, K., Luo, J., Bigham, J.P.: Vizwiz grand challenge: Answering visual questions from blind people. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3608–3617 (2018)

    2018

  2. [10]

    In: Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition

    Hudson, D.A., Manning, C.D.: Gqa: A new dataset for real-world visual reasoning and compositional question answering. In: Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition. pp. 6700–6709 (2019)

    2019

  3. [11]

    arXiv preprint arXiv:2405.07987 (2024)

    Huh, M., Cheung, B., Wang, T., Isola, P.: The platonic representation hypothesis. arXiv preprint arXiv:2405.07987 (2024)

    2024 arXiv

  4. [12]

    arXiv preprint arXiv:2503.03321 (2025)

    Kang, S., Kim, J., Kim, J., Hwang, S.J.: See what you are told: Visual attention sink in large multimodal models. arXiv preprint arXiv:2503.03321 (2025)

    2025 arXiv

  5. [13]

    arXiv preprint arXiv:2602.00462 (2026)

    Krojer, B., Nayak, S., Mañas, O., Adlakha, V., Elliott, D., Reddy, S., Mosbach, M.: Latentlens: Revealing highly interpretable visual tokens in llms. arXiv preprint arXiv:2602.00462 (2026)

    2026 arXiv

  6. [14]

    arXiv preprint arXiv:2307.16125 (2023)

    Li, B., Wang, R., Wang, G., Ge, Y., Ge, Y., Shan, Y.: Seed-bench: Benchmarking multimodal llms with generative comprehension. arXiv preprint arXiv:2307.16125 (2023)

    2023 arXiv

  7. [15]

    Advances in Neural Information Processing Systems36, 41451–41530 (2023)

    Li, K., Patel, O., Viégas, F., Pfister, H., Wattenberg, M.: Inference-time inter- vention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems36, 41451–41530 (2023)

    2023

  8. [16]

    International Journal of Computer Vision pp

    Li, W., Yuan, Y., Liu, J., Tang, D., Wang, S., Qin, J., Zhu, J., Zhang, L.: To- kenpacker: Efficient visual projector for multimodal llm. International Journal of Computer Vision pp. 6794–6812 (2025) Not All Redundant Tokens Are Alike 17

    2025

  9. [17]

    In: Proceedings of the 2023 conference on empirical methods in natural language processing

    Li, Y., Du, Y., Zhou, K., Wang, J., Zhao, X., Wen, J.R.: Evaluating object hal- lucination in large vision-language models. In: Proceedings of the 2023 conference on empirical methods in natural language processing. pp. 292–305 (2023)

    2023

  10. [18]

    In: European conference on computer vision

    Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: European conference on computer vision. pp. 740–755. Springer (2014)

    2014

  11. [19]

    In: Proceedings of the AAAI Confer- ence on Artificial Intelligence

    Lin, Z., Lin, M., Lin, L., Ji, R.: Boosting multimodal large language models with visual tokens withdrawal for rapid inference. In: Proceedings of the AAAI Confer- ence on Artificial Intelligence. pp. 5334–5342 (2025)

    2025

  12. [20]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Liu, H., Li, C., Li, Y., Lee, Y.J.: Improved baselines with visual instruction tun- ing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 26296–26306 (2024)

    2024

  13. [21]

    Advances in neural information processing systems pp

    Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. Advances in neural information processing systems pp. 34892–34916 (2023)

    2023

  14. [22]

    Liu, Y., Duan, H., Zhang, Y., Li, B., Zhang, S., Zhao, W., Yuan, Y., Wang, J., He, C., Liu, Z., et al.: Mmbench: Is your multi-modal model an all-around player? In: European conference on computer vision. pp. 216–233. Springer (2024)

    2024

  15. [23]

    Science China Information Sciences67(12), 220102 (2024)

    Liu, Y., Li, Z., Huang, M., Yang, B., Yu, W., Li, C., Yin, X.C., Liu, C.L., Jin, L., Bai, X.: Ocrbench: on the hidden mystery of ocr in large multimodal models. Science China Information Sciences67(12), 220102 (2024)

    2024

  16. [24]

    Advances in neural information processing systems35, 2507– 2521 (2022)

    Lu, P., Mishra, S., Xia, T., Qiu, L., Chang, K.W., Zhu, S.C., Tafjord, O., Clark, P., Kalyan, A.: Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in neural information processing systems35, 2507– 2521 (2022)

    2022

  17. [25]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Shang, Y., Cai, M., Xu, B., Lee, Y.J., Yan, Y.: Llava-prumerge: Adaptive token reduction for efficient large multimodal models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 22857–22867 (2025)

    2025

  18. [26]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Singh, A., Natarajan, V., Shah, M., Jiang, Y., Chen, X., Batra, D., Parikh, D., Rohrbach, M.: Towards vqa models that can read. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8317–8326 (2019)

    2019

  19. [27]

    ImportantTokens

    Wen, Z., Gao, Y., Wang, S., Zhang, J., Zhang, Q., Li, W., He, C., Zhang, L.: StopLookingfor“ImportantTokens” inMultimodalLanguageModels:Duplication Matters More. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. pp. 9972–9991 (2025)

    2025

  20. [28]

    Xing, L., Huang, Q., Dong, X., Lu, J., Zhang, P., Zang, Y., Cao, Y., He, C., Wang,J.,Wu,F.,Lin,D.:PyramidDrop:AcceleratingYourLargeVision-Language Models via Pyramid Visual Redundancy Reduction (2025)

    2025

  21. [29]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yang, S., Chen, Y., Tian, Z., Wang, C., Li, J., Yu, B., Jia, J.: Visionzip: Longer is better but not necessary in vision language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 19792– 19802 (2025)

    2025

  22. [30]

    In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing

    Yin, Y., Zhao, Y., Zhang, Y., Zhang, Y., Lin, K., Wang, J., Tao, X., Wan, P., Zhang, W., Zhao, F.: Sea: Supervised embedding alignment for token-level visual- textual integration in mllms. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processi...

    2025

  23. [31]

    Zhang, Y., Fan, C.K., Ma, J., Zheng, W., Huang, T., Cheng, K., Gudovskiy, D., Okuno, T., Nakata, Y., Keutzer, K., Zhang, S.: SparseVLM: Visual Token Sparsi- fication for Efficient Vision-Language Model Inference (2025) 18 Kim et al

    2025

  24. [32]

    arXiv preprint arXiv:2310.01405 (2023)

    Zou, A., Phan, L., Chen, S., Campbell, J., Guo, P., Ren, R., Pan, A., Yin, X., Mazeika, M., Dombrowski, A.K., et al.: Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405 (2023)

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.