Pith. sign in

REVIEW 3 major objections 4 minor 8 cited by

A Glimpse to Compress: Dynamic Visual Token Pruning for Large Vision-Language Models

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single forward-pass 'glimpse' can discard 92.6% of visual tokens before a vision-language model answers, keeping average free-form VQA accuracy intact and letting fine-tuning reach 110% of baseline.

desk verdict The supplied full text is the wrong paper, so the pruning claims are unverifiable; the abstract is plausible but the aggregate wording needs per-benchmark scrutiny before review. read the letter →

arxiv 2508.01548 v1 pith:H2OIZBHN submitted 2025-08-03 cs.CV

Quan-Sheng Zeng , Yunheng Li , Qilong Wang , Peng-Tao Jiang , Zuxuan Wu , Ming-Ming Cheng , Qibin Hou This is my paper
classification cs.CV
keywords visualtokenpruninglargevision-languagemodelsdynamiccompressionfree-formVQAefficientinferencedata-drivenglimpserelevancePrune
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that most of the visual tokens a large vision-language model consumes are irrelevant to the question at hand, and that a single data-driven 'glimpse' taken in the same forward pass can identify which tokens can be pruned before answer generation begins. The claim is that this dynamic pruning removes 92.6% of visual tokens while on average fully retaining baseline performance on free-form VQA tasks, and that the computational budget saved can be reinvested in fine-tuning: an enhanced version, GlimpsePrune+, reaches 110% of baseline accuracy at a similarly high pruning rate. If true, the bottleneck that forces vision-language models to trade image resolution against compute cost would loosen considerably, since a cheap early signal would decide how many tokens each image is worth. The motivating contrast is with fixed-ratio compression, which cannot adapt to scenes of different complexity and therefore tends to discard informative tokens on hard inputs.

What carries the argument

The load-bearing mechanism is the data-driven 'glimpse', an early signal computed in the same forward pass that assigns a relevance judgement to visual tokens before any answer text is generated. It does the work of replacing fixed compression ratios with per-image, dynamic pruning decisions, so that the number of tokens kept is determined by the scene rather than by a constant. Exactly how the glimpse is computed, which layer or scoring function produces it, is not specified in the abstract; what the abstract asserts is that this single-pass signal is sufficient to discard a large majority of tokens without average accuracy loss.

What would settle it

Build a VQA evaluation where correct answers depend on visually unremarkable details, such as counting small objects, reading faint or tiny text, or locating a peripheral object named by the question, and compare which tokens the glimpse prunes against the image regions the ground-truth answer actually needs. If per-example accuracy on that subset drops far below baseline whenever pruning is aggressive, the premise that relevance is decidable before generation fails; if a benchmark's hard examples are not protected by the glimpse, the mechanism is selection on easy cases.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that visual-token relevance is decidable before decoding: a data-driven glimpse computed in a single forward pass scores which visual tokens the language model will need, and that signal is strong enough to support pruning 92.6% of tokens while on average fully retaining free-form VQA accuracy. Because the pruned sequence is smaller, the saved compute can be spent on fine-tuning, and the fine-tuned GlimpsePrune+ reports 110% of baseline performance at a similarly high pruning rate. The argument is framed as an efficiency claim with an accuracy claim attached: dynamic, input-dependent pruning beats fixed compression ratios precisely because it lets simple images shed most of their tokens while complex scenes keep more.

Load-bearing premise

The load-bearing premise is that the visual tokens a decoder will need can be told apart from irrelevant ones by a single early glance, before any answer text has been generated; a second, quieter premise is that holding baseline performance 'on average' across VQA tasks counts as full retention.

Editorial extensions

If this is right

  • If the claim holds, roughly an order of magnitude of visual-token processing can be cut from LVLM inference without an average accuracy cost, which directly relaxes the resolution-versus-compute trade-off for high-resolution image inputs.
  • Dynamic pruning adapts to scene complexity: images that are simple or irrelevant to the question shed most of their tokens, while cluttered images keep more, addressing the failure mode the paper attributes to fixed-ratio compression.
  • The compute freed by pruning can be reinvested: the paper's GlimpsePrune+ shows that fine-tuning on the pruned token stream raises accuracy to 110% of the baseline while keeping a similarly high pruning rate.
  • Because pruning happens before generation, the language-model decode stage sees a much shorter token sequence, cutting both latency and memory in the autoregressive phase.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same glimpse signal, if it truly scores answer-relevance, could be reused beyond pruning: as a saliency or explanation map, as a controller that decides when to request a higher-resolution crop, or as a scheduler that spends compute per image, none of which the paper states.
  • The word 'average' hides per-benchmark variance; the natural stress test is fine-grained counting and OCR-style VQA, where early-glance saliency is most likely to miss answer-critical tokens, and testing whether pruning rate is automatically lower on hard examples would separate genuine relevance detection from selection on easy cases.
  • The 110% fine-tuning result suggests that dropping a large majority of tokens acts as a regularizer, focusing the model's capacity on image content the question actually needs; if that is the mechanism, token-reduction curricula of this kind may transfer to other modalities or to video, a hypothesis the paper leaves implicit.
  • Editorial note on evidence: the text supplied alongside this abstract is a different manuscript, about adversarial robustness of prompts in language models, so the abstract's numbers are the only claims that can be weighed here; reproducing the 92.6% and 110% figures requires the experiments and benchmarks the abstract refers to but the supplied text does not include.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The submission consists of an abstract for a paper titled "A Glimpse to Compress: Dynamic Visual Token Pruning for Large Vision-Language Models" (arXiv:2508.01548, cs.CV), followed by a full text that is in fact a different paper, arXiv:2508.01554, on prompt dissection and component-wise adversarial robustness of LLMs. The abstract claims that GlimpsePrune prunes 92.6% of visual tokens in a single forward pass before answer generation while on average fully retaining baseline performance on free-form VQA tasks, and that GlimpsePrune+ reaches 110% of baseline performance. The full text contains none of the GlimpsePrune method, no description of the glimpse scoring mechanism, no LVLM experiments, no VQA benchmarks, no baseline model, and no definition of the averaging rule. Consequently, the central claims of the abstract are entirely unsupported by the submitted manuscript body.

Significance. If the abstract's claims were supported by a proper method description and evaluation, the contribution would be significant: a single-pass, content-adaptive visual token pruning scheme with a 92.6% pruning rate and no loss of average VQA accuracy would be a practical advance for LVLM inference, and the reported 110%-of-baseline fine-tuned variant would strengthen the case further. However, as submitted, the significance cannot be assessed. No machine-checked proofs, reproducible code, parameter-free derivations, or falsifiable per-benchmark predictions are present in the supplied text. The reader is left with only an abstract whose key quantitative claims are undefined aggregates and whose full experimental support is missing.

major comments (3)
  1. [Full Text (§1–§5)] The body of the manuscript is a different paper: it presents PROMPT ANATOMY and COMPERTURB for adversarial robustness of prompt components in LLMs (arXiv:2508.01554) and never mentions GlimpsePrune, visual token pruning, LVLMs, VQA, or the 92.6% pruning result. The central claims in the abstract therefore have no supporting method or evaluation in the submitted text; this is a load-bearing gap that cannot be repaired by local revision.
  2. [Abstract] The phrases "on average fully retaining the baseline performance" and "110% of the baseline performance" are undefined aggregates. The text does not specify the baseline model, the set of free-form VQA benchmarks, the evaluation metric, the number of runs or standard errors, or the rule by which per-benchmark results are averaged. Without these definitions, the 92.6% pruning figure with "fully retained" performance permits per-benchmark degradation that is masked by averaging.
  3. [Abstract, method sentence] The claim that visual-token relevance is decidable "in a single forward pass before answer generation" is a testable mechanistic premise, but the full text contains no description of the glimpse scoring function, no equation, no ablation on question-dependent referents, and no failure analysis. The premise remains unverified; a concrete test would be to measure accuracy on samples where the correct answer depends on small or visually inconspicuous objects that may receive low glimpse scores.
minor comments (4)
  1. [§1] "Base on these findings" should read "Based on these findings."
  2. [§1] "everal works propose" is missing the initial "S" and should read "Several works propose."
  3. [Table 2] The header "Model COM PPL" is ambiguous: the "COM" column appears to list prompt components and "PPL" the perturbation types, but this is not explained before the table; the note "A VG = Average Result" also appears truncated.
  4. [§3.1] Algorithm 1 references "Smiss" and "LLMClassify(s_i, C_i)" without specifying the exact classification prompt or output schema beyond the XML-style tags, which hampers reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity can be assessed: the supplied full text is a different paper, and the GlimpsePrune abstract provides no derivation chain to audit.

full rationale

The submitted manuscript text for arXiv:2508.01548 is not present; the supplied full text is arXiv:2508.01554, an unrelated paper on prompt-component adversarial robustness for LLMs. The abstract of the GlimpsePrune paper states the claimed pruning rates and performance retention, but contains no equations, no fitted parameters, no thresholds, no benchmark definitions, and no derivation of the pruning rule from the performance objective. Under the hard rules, circularity may only be claimed when the paper itself exhibits a specific reduction, such as an equation being equal to its input by construction or a fitted parameter being renamed as a prediction. No such reduction can be quoted from the available material. The absence of the actual method and evaluation protocol is a completeness or correctness problem, not evidence of circularity. Therefore the honest finding is no significant circularity, with a score of 0.

Assumptions & free parameters 1 free parameters · 2 assumptions · 0 invented entities

The ledger is abstract-limited. The two domain assumptions are the load-bearing premises: token relevance is decidable pre-generation from one glimpse, and average retention across VQA tasks counts as full retention. One potential free parameter is flagged, the threshold or ratio behind the 92.6% figure, but it is an unknown rather than a confirmed fit. The method itself introduces no separately postulated physical or conceptual entities beyond its own machinery, so the invented-entities ledger is empty.

free parameters (1)
  • pruning threshold or target token-pruning ratio = not visible; headline rate is 92.6% of visual tokens pruned
    The abstract reports a 92.6% pruning rate as the outcome, but the mechanism that sets it, a score threshold, a fixed ratio, or a per-image dynamic policy, is not described. If the threshold is calibrated per benchmark to preserve accuracy, the retained-performance claim would depend on that fitted choice. This entry marks an unknown, not a confirmed free parameter.
assumptions (2)
  • domain assumption Visual-token relevance for answering a VQA question can be decided before generation starts, from a data-driven glimpse computed in one forward pass.
    The abstract's mechanism sentence requires that irrelevance is knowable pre-generation. If the tokens needed for a correct answer only become identifiable during decoding, pruning would remove answer-critical content. Location: Abstract, method sentence.
  • domain assumption Averaged retention across free-form VQA tasks is an acceptable operationalization of 'fully retaining baseline performance'.
    The abstract reports 'on average fully retaining the baseline performance', which permits per-benchmark drops where other tasks rise. Whether the average hides systematic losses on specific benchmarks cannot be checked from the abstract. Location: Abstract, results sentence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Glimpse to Compress: Dynamic Visual Token Pruning for Large Vision-Language Models." pith.science (2026). https://pith.science/paper/H2OIZBHN

@misc{pith2026250801548,
  author       = {Pith},
  title        = {Pith review of: A Glimpse to Compress: Dynamic Visual Token Pruning for Large Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H2OIZBHN}},
  note         = {Machine review of arXiv:2508.01548}
}
read the original abstract

Visual token compression is critical for Large Vision-Language Models (LVLMs) to efficiently process high-resolution inputs. Existing methods that typically adopt fixed compression ratios cannot adapt to scenes of varying complexity, often causing imprecise pruning that discards informative visual tokens and results in degraded model performance. To address this issue, we introduce a dynamic pruning framework, GlimpsePrune, inspired by human cognition. It takes a data-driven ''glimpse'' and prunes irrelevant visual tokens in a single forward pass before answer generation. This approach prunes 92.6% of visual tokens while on average fully retaining the baseline performance on free-form VQA tasks. The reduced computational cost also enables more effective fine-tuning: an enhanced GlimpsePrune+ achieves 110% of the baseline performance while maintaining a similarly high pruning rate. Our work paves a new way for building more powerful and efficient LVLMs.

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Semantic-Geometric Dual Compression: Training-Free Visual Token Reduction for Ultra-High-Resolution Remote Sensing Understanding

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    DualComp uses a lightweight router to split visual token compression into a semantic stream with size-adaptive clustering and a geometric stream with path-tracing recovery, enabling low-cost high-fidelity UHR remote s...

  2. WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Wavelet-based temporal and spatial token condensation preserves 99.6% of full-token video question-answering accuracy at 10x compression without retraining.

  3. WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    WaveZip, a training-free wavelet method for video token condensation, reports about 10x token reduction while retaining roughly 98-99% accuracy on multi-benchmark LVLM evaluation.

  4. Spectral Evolution-Guided Token Pruning in Multimodal Large Language Models

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    CLSE prunes tokens in MLLMs by quantifying cross-layer spectral redistribution in the frequency domain to preserve semantically active tokens and reduce compute.

  5. Accelerating Multimodal Large Language Models with Prior-Corrected Token Reduction

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    PriorTR estimates model-induced prior attention via a null token in one forward pass and contrasts it with task-conditioned attention to improve visual token pruning accuracy-efficiency trade-offs in MLLMs.

  6. Beyond Surrogate Gradients: Fully Differentiable Token Pruning for Vision-Language Models

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    DiffPrune reformulates visual token pruning as continuous control of token information using an Information Throttler with importance-conditioned variance-preserving noise, enabling fully differentiable learning of sc...

  7. See What I Mean: Aligning Vision and Language Representations for Video Fine-grained Object Understanding

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    SWIM aligns cross-attention maps from object nouns to ground-truth masks during training on the new NL-Refer dataset to enable text-only fine-grained video object understanding in MLLMs.

  8. DiffPrune: differentiable information throttling for token pruning in vision-language models

    cs.CV 2026-08 conditional novelty 4.0 of 10

    DiffPrune replaces Gumbel-Softmax surrogate gradients with a differentiable noise-throttling path for visual token scoring, achieving high accuracy retention under aggressive pruning on three VLM families.

Reference graph

Works this paper leans on

47 extracted references · 26 canonical work pages · cited by 7 Pith papers

  1. [1]

    Efficient large multi-modal models via visual context compression

    Jieneng Chen, Luoxin Ye, Ju He, Zhao-Yang Wang, Daniel Khashabi, and Alan Yuille. Efficient large multi-modal models via visual context compression. Advances in Neural Information Processing Systems , 37:73986--74007, 2024

    work page 2024

  2. [2]

    Llama-vid: An image is worth 2 tokens in large language models

    Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models. In European Conference on Computer Vision , pages 323--340. Springer, 2024

    work page 2024

  3. [3]

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. In European Conference on Computer Vision , pages 19--35. Springer, 2024

    work page 2024

  4. [4]

    Visionzip: Longer is better but not necessary in vision language models

    Senqiao Yang, Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia. Visionzip: Longer is better but not necessary in vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19792--19802, 2025

    work page 2025

  5. [5]

    Vscan: Rethinking visual token reduction for efficient large vision-language models

    Ce Zhang, Kaixin Ma, Tianqing Fang, Wenhao Yu, Hongming Zhang, Zhisong Zhang, Yaqi Xie, Katia Sycara, Haitao Mi, and Dong Yu. Vscan: Rethinking visual token reduction for efficient large vision-language models. arXiv preprint arXiv:2505.22654 , 2025

    arXiv 2025

  6. [6]

    MLLM s know where to look: Training-free perception of small visual details with multimodal LLM s

    Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, and Filip Ilievski. MLLM s know where to look: Training-free perception of small visual details with multimodal LLM s. In International Conference on Learning Representations , 2025

    work page 2025

  7. [7]

    Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy reduction

    Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, et al. Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy reduction. arXiv preprint arXiv:2410.17247 , 2024

    arXiv

  8. [8]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International Conference on Machine Learning , pages 19730--19742. PMLR, 2023

    work page 2023

Show all 47 references
  1. [9]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in Neural Information Processing Systems , 36:34892--34916, 2023

    2023

  2. [10]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vis...

    2024

  3. [11]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. Science China Information Sciences , 67(12):220101, 2024

    2024

  4. [12]

    Enhancing the reasoning ability of multimodal large language models via mixed preference optimization

    Weiyun Wang, Zhe Chen, Wenhai Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Jinguo Zhu, Xizhou Zhu, Lewei Lu, Yu Qiao, and Jifeng Dai. Enhancing the reasoning ability of multimodal large language models via mixed preference optimization. arXiv preprint arXiv:2411.10442 , 2024

    2024 arXiv

  5. [13]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271 , 2024

    2024 arXiv

  6. [14]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479 , 2025

    2025 arXiv

  7. [15]

    Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966 , 2023

    2023 arXiv

  8. [16]

    Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model's pe...

    2024 arXiv

  9. [17]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Han...

    2025 arXiv

  10. [18]

    Free video-llm: Prompt-guided visual perception for efficient training-free video llms

    Kai Han, Jianyuan Guo, Yehui Tang, Wei He, Enhua Wu, and Yunhe Wang. Free video-llm: Prompt-guided visual perception for efficient training-free video llms. arXiv preprint arXiv:2410.10441 , 2024

    2024 arXiv

  11. [19]

    Retake: Reducing temporal and knowledge redundancy for long video understanding

    Xiao Wang, Qingyi Si, Jianlong Wu, Shiyu Zhu, Li Cao, and Liqiang Nie. Retake: Reducing temporal and knowledge redundancy for long video understanding. arXiv preprint arXiv:2412.20504 , 2024

    2024 arXiv

  12. [20]

    Beyond attention or similarity: Maximizing conditional diversity for token pruning in mllms

    Qizhe Zhang, Mengzhen Liu, Lichen Li, Ming Lu, Yuan Zhang, Junwen Pan, Qi She, and Shanghang Zhang. Beyond attention or similarity: Maximizing conditional diversity for token pruning in mllms. arXiv preprint arXiv:2506.10967 , 2025

    2025 arXiv

  13. [21]

    Beyond text-visual attention: Exploiting visual cues for effective token pruning in vlms

    Qizhe Zhang, Aosong Cheng, Ming Lu, Renrui Zhang, Zhiyong Zhuo, Jiajun Cao, Shaobo Guo, Qi She, and Shanghang Zhang. Beyond text-visual attention: Exploiting visual cues for effective token pruning in vlms. arXiv preprint arXiv:2412.01818 , 2025

    2025 arXiv

  14. [22]

    Pact: Pruning and clustering-based token reduction for faster visual language models

    Mohamed Dhouib, Davide Buscaldi, Sonia Vanier, and Aymen Shabou. Pact: Pruning and clustering-based token reduction for faster visual language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14582--14592, 2025

    2025

  15. [23]

    Divprune: Diversity-based visual token pruning for large multimodal models

    Saeed Ranjbar Alvar, Gursimran Singh, Mohammad Akbari, and Yong Zhang. Divprune: Diversity-based visual token pruning for large multimodal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9392--9401, 2025

    2025

  16. [24]

    Topv: Compatible token pruning with inference time optimization for fast and low-memory multimodal vision language model

    Cheng Yang, Yang Sui, Jinqi Xiao, Lingyi Huang, Yu Gong, Chendi Li, Jinghua Yan, Yu Bai, Ponnuswamy Sadayappan, Xia Hu, et al. Topv: Compatible token pruning with inference time optimization for fast and low-memory multimodal vision language model. In Proceedings of the IEEE/C...

    2025

  17. [25]

    Llava-scissor: Token compression with semantic connected components for video llms

    Boyuan Sun, Jiaxing Zhao, Xihan Wei, and Qibin Hou. Llava-scissor: Token compression with semantic connected components for video llms. arXiv preprint arXiv:2506.21862 , 2025

    2025 arXiv

  18. [26]

    Fu, Stefano Ermon, Atri Rudra, and Christopher R \'e

    Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher R \'e . Flash A ttention: Fast and memory-efficient exact attention with IO -awareness. In Advances in Neural Information Processing Systems , 2022

    2022

  19. [27]

    Flash A ttention-2: Faster attention with better parallelism and work partitioning

    Tri Dao. Flash A ttention-2: Faster attention with better parallelism and work partitioning. In International Conference on Learning Representations , 2024

    2024

  20. [28]

    Chat-univi: Unified visual representation empowers large language models with image and video understanding

    Peng Jin, Ryuichi Takanobu, Wancai Zhang, Xiaochun Cao, and Li Yuan. Chat-univi: Unified visual representation empowers large language models with image and video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13700-...

    2024

  21. [29]

    Dynamic- LL a VA : Efficient multimodal large language models via dynamic vision-language context sparsification

    Wenxuan Huang, Zijie Zhai, Yunhang Shen, Shaosheng Cao, Fei Zhao, Xiangfeng Xu, Zheyu Ye, and Shaohui Lin. Dynamic- LL a VA : Efficient multimodal large language models via dynamic vision-language context sparsification. In International Conference on Learning Representations , 2025

    2025

  22. [30]

    Dycoke: Dynamic compression of tokens for fast video large language models

    Keda Tao, Can Qin, Haoxuan You, Yang Sui, and Huan Wang. Dycoke: Dynamic compression of tokens for fast video large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18992--19001, 2025

    2025

  23. [31]

    Dynamic token reduction during generation for vision language models

    Xiaoyu Liang, Chaofeng Guan, Jiaying Lu, Huiyao Chen, Huan Wang, and Haoji Hu. Dynamic token reduction during generation for vision language models. arXiv preprint arXiv:2501.14204 , 2025

    2025 arXiv

  24. [32]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing , 568:127063, 2024

    2024

  25. [33]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 , 2024

    2024 arXiv

  26. [34]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 26296--26306, 2024

    2024

  27. [35]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6700--6709, 2019

    2019

  28. [36]

    Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning

    Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. Advances in Neural Information Processing Systems ,...

    2024

  29. [37]

    Lo RA : Low-rank adaptation of large language models

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lo RA : Low-rank adaptation of large language models. In International Conference on Learning Representations , 2022

    2022

  30. [38]

    Qwen2.5 technical report

    A Yang Qwen, Baosong Yang, B Zhang, B Hui, B Zheng, B Yu, Chengpeng Li, D Liu, F Huang, H Wei, et al. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115 , 2024

    2024 arXiv

  31. [39]

    Lmms-eval: Reality check on the evaluation of large multimodal models

    Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Lmms-eval: Reality check on the evaluation of large multimodal models. arXiv preprint arXiv:2407.12772 , 2024

    2024 arXiv

  32. [40]

    Making the v in vqa matter: Elevating the role of image understanding in visual question answering

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6904--6913, 2017

    2017

  33. [41]

    Vizwiz grand challenge: Answering visual questions from blind people

    Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 36...

    2018

  34. [42]

    Learn to explain: Multimodal reasoning via thought chains for science question answering

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems , 35:2507--...

    2022

  35. [43]

    Evaluating object hallucination in large vision-language models

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355 , 2023

    2023 arXiv

  36. [44]

    Mme: A comprehensive evaluation benchmark for multimodal large language models

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:/2306.13394 , 2023

    2023 arXiv

  37. [45]

    Mmbench: Is your multi-modal model an all-around player? In European Conference on Computer Vision , pages 216--233

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European Conference on Computer Vision , pages 216--233. Springer, 2024

    2024

  38. [46]

    Seed-bench: Benchmarking multimodal llms with generative comprehension

    Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed-bench: Benchmarking multimodal llms with generative comprehension. arXiv preprint arXiv:2307.16125 , 2023

    2023 arXiv

  39. [47]

    V*: Guided visual search as a core mechanism in multimodal llms

    Penghao Wu and Saining Xie. V*: Guided visual search as a core mechanism in multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13084--13094, 2024

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.