Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Blink shows that a frozen multimodal LLM's attention naturally shifts across layers, and that expanding salient tokens in one forward pass improves visual perception and understanding.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Blink dynamically expands high-saliency visual tokens and drops them when attention shifts, improving LLaVA-1.5 and LLaVA-NeXT across seven multimodal benchmarks.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection Neat single-pass token expansion/drop framework; the hard evidence that saliency-guided selection beats random selection is missing. the 4 major comments →

arxiv 2512.10548 v3 pith:Z2LW6I2B submitted 2025-12-11 cs.CV

Blink: Dynamic Visual Token Resolution for Enhanced Multimodal Understanding

Yuchen Feng , Zhenyu Zhang , Naibin Gu , Yilong Chen , Peng Fu , Zheng Lin , Shuohuan Wang , Yu Sun
show 3 more authors
Hua Wu Weiping Wang Haifeng Wang
This is my paper
classification cs.CV
keywords Blinkmultimodal large language modelsvisual perceptionvisual attentiontoken super-resolutiondynamic token resolutionsaliency-guided scanningsingle forward pass
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether a multimodal large language model can perceive images more like a human: scanning broadly, fixating on salient regions, then shifting focus. It answers yes in a restricted sense: the model's own attention maps already move across layers, and inserting extra tokens for the currently salient patch at a middle layer improves downstream answers. The proposed Blink does this dynamically in a single forward pass, expanding high-saliency patches through a small trainable token super-resolution module and dropping them when attention shifts. Across seven benchmarks the method consistently beats the unchanged backbone, and even a training-free interpolation variant helps. The central surprise is that a frozen backbone can be improved this way, without a second forward pass or retraining.

Core claim

Blink's central claim is that multimodal LLMs already possess a layer-by-layer visual scan path: the region the final text token attends to changes across depth and is sharpest in middle layers. Using that signal, Blink defines a per-layer saliency ratio from the query-key similarity between the last text token and visual tokens, aggregated into image patches. When the ratio is high, the salient patch's hidden states are upsampled and refined by a learned token super-resolution module and reinserted into the sequence, effectively zooming in within the same pass; when the ratio drops, the expanded tokens are removed. The paper asserts that this expand-and-drop cycle improves visual perception

What carries the argument

The load-bearing object is the saliency ratio rho(L), defined as the total attention of the most salient p x p image patch divided by the total visual attention at layer L, computed from the query of the last text token and the keys of visual tokens. It decides, via thresholds tau_exp and tau_drop, whether to invoke the TokenSR module (three convolutional layers that refine upsampled hidden states of the salient patch to mimic features of the corresponding crop) or to drop previously expanded tokens. This turns a static sequence of visual tokens into a variable-resolution sequence inside one forward pass.

Load-bearing premise

The load-bearing premise is that the saliency ratio computed from the last text token's attention, aggregated into image patches, marks the region whose higher-resolution processing will help the answer, and that the hand-set thresholds trigger expansion only when that signal is trustworthy.

What would settle it

Run Blink on images where the top-attended patch at the expansion layers is deliberately wrong (for example, a prominent distractor) while the correct small object lies elsewhere; if Blink does not clearly underperform an oracle that expands the correct patch, or if expanding a random patch produces the same gains, then the saliency signal is not doing the causal work the paper assigns to it.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A frozen multimodal backbone can be upgraded at inference time with small plug-in modules, avoiding full retraining or a second forward pass.
  • Attention maps from middle layers can serve as a usable saliency signal for perceptual enhancement, not just for visualization.
  • Expanding tokens where attention is sharp and dropping them when attention becomes diffuse makes improved perception and controlled compute compatible.
  • The training-free interpolation variant shows that the dynamic policy itself contributes gains independently of the learned amplifier.
  • The sensitivity to layer range indicates that the method exploits the depth window where attention is correct, pointing to where in the network visual grounding actually happens.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the per-benchmark hand-tuning of thresholds and layer ranges suggests the saliency ratio is not scale-free; a learned or calibrated threshold policy could make Blink applicable without manual configuration.
  • Editorial extension: because the current expansion targets a single top patch per layer, a natural extension is to maintain multiple active patches with a schedule that migrates tokens between them, better matching images with several salient objects.
  • Editorial extension: if attention shifts in middle layers genuinely track answer-relevant objects, the same saliency maps could provide weak localization supervision or serve as a training-free hint for downstream grounding tasks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Blink, a method that enhances multimodal large language models (MLLMs) within a single forward pass by dynamically expanding high-saliency visual tokens and dropping them when attention shifts. The method uses attention-based saliency scores (Eq. 2–3) to select a patch, a trainable token super-resolution (TokenSR) module to expand that patch, and a threshold-based policy to decide when to expand or drop. Experiments on LLaVA-1.5-7B and LLaVA-NeXT-7B report improvements on seven vision-language benchmarks, with ablations showing both modules contribute. The paper also presents two 'key insights': that attention shifts across layers and that allocating more computation to salient tokens improves perception.

Significance. If substantiated, Blink would offer a lightweight, training-efficient plug-in for frozen MLLMs that improves multimodal understanding without extra forward passes, which is a practically valuable contribution. The paper provides a reasonable amount of evidence: consistent (though often small) improvements on two backbones, ablations isolating the dynamic token resolution component, and a training-free variant (Blink-interp) that also helps. However, the empirical support is weakened by (i) lack of error bars or significance testing, (ii) per-benchmark hyperparameter tuning reported in Appendix B, and (iii) the ablation showing that replacing saliency-guided selection with random selection has almost no effect. These issues directly affect the paper's central claims about the role of saliency-guided scanning, making the current evidence insufficient for acceptance.

major comments (4)
  1. [§4.2, Table 1] Many reported improvements are very small: GQA +0.05, MMBench +0.09, POPE +0.42, SQA +0.20, and MM-Vet +1.20 for Blink over Vanilla. No error bars, confidence intervals, or significance tests are provided, and the paper does not state the number of runs or seeds. For such margins, single-run results are within plausible run-to-run variability and do not support the claim of 'consistent improvement.' Please report multiple seeds or bootstrap intervals, at least for the main benchmarks.
  2. [Appendix B] The method uses a different configuration for nearly every benchmark: layers 12–18 with τ_exp=0.5 for MME/MM-Vet; layers 12–18 with τ_exp=0.6 for GQA; layer 18 with τ_exp=0.5 for POPE; layer 18 with τ_exp=0.25 for MMBench/CN/SQA. This per-benchmark tuning means the reported results are partly selected on the test sets, undermining the claim of a general plug-in. Moreover, Table 3 shows that shifting the layer range from 12–18 to 13–19 changes MME total by −7.42 points, indicating high sensitivity to these choices. Please either use a single fixed configuration across all benchmarks or describe a principled validation-based selection procedure.
  3. [Table 2, w/o SGS] Removing saliency-guided scanning (w/o SGS) and replacing it with random patch selection changes MME total by only −2.10 for Blink-interp and −2.15 for Blink, out of ~1880 points (~0.1%). This is substantially smaller than the effect of removing DTR (−41.07 for Blink) and is likely within run-to-run noise. Since the paper's core claim is that the saliency signal (Eq. 2–3) identifies the region whose expansion improves perception, this ablation does not support the load-bearing role of saliency. The authors should either provide statistical evidence that this difference is meaningful, or temper the claim and refocus the method on the dynamic expansion/drop mechanism.
  4. [§4.4, Fig. 6] The authors themselves show (Case 2) that after expansion at layer 12, attention at layer 13 points to an incorrect region, with correction only by layer 20. While the paper frames this as evidence of progressive attention, it also demonstrates that the saliency-based expansion can amplify a wrong region. This raises the risk that the method often invests computation in irrelevant areas. Please quantify how frequently the selected patch at expansion layers matches the ground-truth region across the benchmark data, and analyze whether performance varies with selection accuracy.
minor comments (5)
  1. [Throughout] The backbone name 'LLaV A-1.5' appears with an extra space; it should be 'LLaVA-1.5'.
  2. [Eq. (2)] The saliency score is a raw dot product between query and key vectors. Consider scaling (e.g., by sqrt(d)) for numerical stability, or justify why scaling is unnecessary.
  3. [Figs. 2, 3] The captions do not specify how attention weights are aggregated across heads or how the 'most attended region' is defined beyond total attention. Please clarify.
  4. [Appendix B] For MMBench/CN/SQA, τ_exp=0.25 is used while others use 0.5 or 0.6. The choice is not explained; a brief rationale would help.
  5. [Table 3] The layer-range sensitivity analysis is useful but would be strengthened by reporting per-category (Perception/Cognition) as done, and by adding statistical significance for the differences.

Circularity Check

0 steps flagged

No significant circularity: Blink is an empirical engineering contribution; no equation, prediction, or uniqueness claim reduces to its inputs.

full rationale

The paper does not claim a formal derivation or a uniqueness theorem; it presents an empirical framework motivated by a pilot observation. The pilot analysis (copying high-attention tokens at fixed layers improves MME scores) is a motivating experiment, not a definitional input. Blink's actual components—saliency ratio thresholds, dynamic expand/drop policy, and the TokenSR module—are evaluated on external benchmarks, and the improvement does not follow by construction from the pilot observation. The w/o SGS ablation showing only a ~2-point drop for random selection is an empirical-support weakness, not a circular-reasoning defect. Per-benchmark threshold/layer settings in Appendix B are hyperparameter choices, not fitted quantities relabeled as predictions. No load-bearing self-citations appear; the cited attention-map literature is standard and the authors' own prior works are only contextual. The Blink-interp variant, which removes the trained module, also provides a check that the inference pipeline itself is not just the training objective restated. Therefore no circular step is present.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

No new physical entities are introduced. The primary free parameters are hand-tuned design choices (thresholds, layer ranges, patch count); the per-benchmark settings in Appendix B are the largest source of selection bias. The remaining assumptions are standard ML/domain assumptions about saliency, supervision, and trainability of inserted tokens.

free parameters (3)
  • Expansion/drop thresholds τ_exp, τ_drop = τ_exp=0.5, τ_drop=0.4 for MME/MM-Vet; GQA τ_exp=0.6; POPE/MMBench/CN/SQA τ_exp=0.25 (Appendix B)
    Chosen per benchmark; high-threshold variants in Table 2 change MME total by -1.39 to -15.99, showing results are sensitive to these hand-set values.
  • Selected layer range L_sel = 12–18 for MME/MM-Vet/GQA; layer 18 for POPE/MMBench/CN/SQA; 14–20 or 12–20 for LLaVA-NeXT (Appendix B)
    Layer-range shifts in Table 3 swing MME total from +0.96 to -7.42, indicating the choice is influential and tuned per dataset.
  • Patch partition p×p = 2×2 in main configuration (Appendix E)
    Ablation shows 3×3 and 4×4 degrade MME total by 8.73–46.79, so the partition size is a consequential hand choice.
axioms (4)
  • domain assumption Attention maps from the final text token are a valid saliency proxy for visual token importance.
    Eq. (2) uses q_tn·k_v; the method expands the patch with highest aggregated saliency. If attention points to a wrong region, expansion reinforces an error (Fig. 6 shows a layer-13 mis-focus).
  • domain assumption Allocating more computation to high-attention visual tokens improves perception.
    Key Insight 2 in Sec. 2.2 (copying salient tokens at layers 14/16/18 improves MME). This observation motivates the method; Blink assumes it generalizes beyond the copy experiment.
  • domain assumption Hidden states of a cropped image are a valid teacher for TokenSR output on full-image tokens.
    Sec. 3.2.1/Appendix A.2: TokenSR minimizes KL divergence between hs_SR and hs_crop. This assumes the frozen backbone's features for a crop are compatible with and desirable for the full-image token stream.
  • domain assumption Inserting extra tokens and updating attention masks/positional embeddings does not break the frozen LLM's internal alignment.
    Sec. 3.2.1: positional embeddings and attention masks are updated to accommodate the expanded sequence; no backbone fine-tuning. If injected embeddings are out-of-distribution, gains could vanish.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Blink: Dynamic Visual Token Resolution for Enhanced Multimodal Understanding." pith.science (2026). https://pith.science/paper/Z2LW6I2B

@misc{pith2026251210548,
  author       = {Pith},
  title        = {Pith review of: Blink: Dynamic Visual Token Resolution for Enhanced Multimodal Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z2LW6I2B}},
  note         = {Machine review of arXiv:2512.10548}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Multimodal large language models (MLLMs) have achieved remarkable progress on various vision-language tasks, yet their visual perception remains limited. Humans, in comparison, perceive complex scenes efficiently by dynamically scanning and focusing on salient regions in a sequential "blink-like" process. Motivated by this strategy, we first investigate whether MLLMs exhibit similar behavior. Our pilot analysis reveals that MLLMs naturally attend to different visual regions across layers and that selectively allocating more computation to salient tokens can enhance visual perception. Building on this insight, we propose Blink, a dynamic visual token resolution framework that emulates the human-inspired process within a single forward pass. Specifically, Blink includes two modules: saliency-guided scanning and dynamic token resolution. It first estimates the saliency of visual tokens in each layer based on the attention map, and extends important tokens through a plug-and-play token super-resolution (TokenSR) module. In the next layer, it drops the extended tokens when they lose focus. This dynamic mechanism balances broad exploration and fine-grained focus, thereby enhancing visual perception adaptively and efficiently. Extensive experiments validate Blink, demonstrating its effectiveness in enhancing visual perception and multimodal understanding.

Figures

Figures reproduced from arXiv: 2512.10548 by HaiFeng Wang, Hua Wu, Naibin Gu, Peng Fu, Shuohuan Wang, Weiping Wang, Yilong Chen, Yuchen Feng, Yu Sun, Zheng Lin, Zhenyu Zhang.

Figure 1
Figure 1. Figure 1: Comparison of the conventional post-hoc paradigm for [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 4
Figure 4. Figure 4: Performance on MME tasks when copying salient visual [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figure 3
Figure 3. Figure 3: Attention ratio across layers. Purple lines indicate the [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Illustration of our proposed Blink. The left side shows the saliency-guided scanning that determines whether to expand or drop [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Visualization of attention redistribution after token ex [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Performance changes on the total MME score with dif [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Visualization of attention redistribution after token expansion. Red boxes in the original image indicate the ground-truth important [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Can LoRA Fusion Support Cross-Domain Tasks in Cloud-Edge Collaboration?

    cs.DC 2026-04 conditional novelty 5.0

    On a new composite cross-domain benchmark, standard LoRA fusion under cloud-edge prune-train-recover often loses to the base model; a shared-subspace conflict gate recovers modest accuracy.

Reference graph

Works this paper leans on

52 extracted references · 12 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Pith/arXiv arXiv

  2. [2]

    Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736,

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736,

  3. [3]

    Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond, 2023

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond, 2023. 8

    2023

  4. [4]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 1

    Pith/arXiv arXiv 2025

  5. [5]

    Hallucination of multimodal large language models: A survey, 2025

    Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. Hallucination of multimodal large language models: A survey, 2025. 1, 8

    2025

  6. [6]

    Geopqa: Bridging the visual perception gap in mllms for geometric reasoning,

    Guizhen Chen, Weiwen Xu, Hao Zhang, Hou Pong Chan, Deli Zhao, Anh Tuan Luu, and Yu Rong. Geopqa: Bridging the visual perception gap in mllms for geometric reasoning,

  7. [7]

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. InEuropean Conference on Computer Vision, pages 19–35. Springer, 2024. 3

    2024

  8. [8]

    Nacl: A general and effective kv cache eviction framework for llms at inference time, 2024

    Yilong Chen, Guoxia Wang, Junyuan Shang, Shiyao Cui, Zhenyu Zhang, Tingwen Liu, Shuohuan Wang, Yu Sun, Di- anhai Yu, and Hua Wu. Nacl: A general and effective kv cache eviction framework for llms at inference time, 2024. 3

    2024

  9. [9]

    Inner thinking transformer: Lever- aging dynamic depth scaling to foster adaptive internal think- ing, 2025

    Yilong Chen, Junyuan Shang, Zhenyu Zhang, Yanxi Xie, Ji- awei Sheng, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu, and Haifeng Wang. Inner thinking transformer: Lever- aging dynamic depth scaling to foster adaptive internal think- ing, 2025. 2

    2025

  10. [10]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 24185–24198, 2024. 1, 8

    2024

  11. [11]

    Spatial- rgpt: Grounded spatial reasoning in vision language models,

    An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Rui- han Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu. Spatial- rgpt: Grounded spatial reasoning in vision language models,

  12. [12]

    Atlas: Mapping attention’s location and size to probe five modes of serial and parallel search.Attention, Perception, & Psychophysics, 86(6):1938–1962, 2024

    Gregory Davis. Atlas: Mapping attention’s location and size to probe five modes of serial and parallel search.Attention, Perception, & Psychophysics, 86(6):1938–1962, 2024. 2

    1938

  13. [13]

    Image super-resolution using deep convolutional net- works.IEEE transactions on pattern analysis and machine intelligence, 38(2):295–307, 2015

    Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Image super-resolution using deep convolutional net- works.IEEE transactions on pattern analysis and machine intelligence, 38(2):295–307, 2015. 5

    2015

  14. [14]

    Acceler- ating the super-resolution convolutional neural network

    Chao Dong, Chen Change Loy, and Xiaoou Tang. Acceler- ating the super-resolution convolutional neural network. In European conference on computer vision, pages 391–407. Springer, 2016. 5

    2016

  15. [15]

    Multi-modal hal- lucination control by visual information grounding, 2024

    Alessandro Favero, Luca Zancato, Matthew Trager, Sid- dharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, and Stefano Soatto. Multi-modal hal- lucination control by visual information grounding, 2024. 1, 8

    2024

  16. [16]

    Dive into moe: Diversity-enhanced reconstruction of large language mod- els from dense into mixture-of-experts.arXiv preprint arXiv:2506.09351, 2025

    Yuchen Feng, Bowen Shen, Naibin Gu, Jiaxuan Zhao, Peng Fu, Zheng Lin, and Weiping Wang. Dive into moe: Diversity-enhanced reconstruction of large language mod- els from dense into mixture-of-experts.arXiv preprint arXiv:2506.09351, 2025. 9

    Pith/arXiv arXiv 2025

  17. [17]

    Mme: A comprehensive evaluation benchmark for multimodal large language models, 2025

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, Rongrong Ji, Caifeng Shan, and Ran He. Mme: A comprehensive evaluation benchmark for multimodal large language models, 2025. 3, 6

    2025

  18. [18]

    Tracking the will to attend: Cortical activity indexes self-generated, voluntary shifts of attention.Attention, Perception, & Psy- chophysics, 78(7):2176–2184, 2016

    Leon Gmeindl, Yu-Chin Chiu, Michael S Esterman, Adam S Greenberg, Susan M Courtney, and Steven Yantis. Tracking the will to attend: Cortical activity indexes self-generated, voluntary shifts of attention.Attention, Perception, & Psy- chophysics, 78(7):2176–2184, 2016. 2

    2016

  19. [19]

    Beamlora: Beam-constraint low-rank adap- tation

    Naibin Gu, Zhenyu Zhang, Xiyu Liu, Peng Fu, Zheng Lin, Shuohuan Wang, Yu Sun, Hua Wu, Weiping Wang, and Haifeng Wang. Beamlora: Beam-constraint low-rank adap- tation. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11871–11883, 2025. 9

    2025

  20. [20]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 6700–6709, 2019. 6

    2019

  21. [21]

    Scaling up visual and vision-language representa- tion learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representa- tion learning with noisy text supervision. InInternational conference on machine learning, pages 4904–4916. PMLR,

  22. [22]

    Hallucination augmented contrastive learning for multimodal large language model, 2024

    Chaoya Jiang, Haiyang Xu, Mengfan Dong, Jiaxing Chen, Wei Ye, Ming Yan, Qinghao Ye, Ji Zhang, Fei Huang, and Shikun Zhang. Hallucination augmented contrastive learning for multimodal large language model, 2024. 1, 8

    2024

  23. [23]

    Cortical mechanisms for shifting and hold- ing visuospatial attention.Cerebral cortex, 18(1):114–125,

    Todd A Kelley, John T Serences, Barry Giesbrecht, and Steven Yantis. Cortical mechanisms for shifting and hold- ing visuospatial attention.Cerebral cortex, 18(1):114–125,

  24. [24]

    Accurate image super-resolution using very deep convolutional net- works

    Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. Accurate image super-resolution using very deep convolutional net- works. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1646–1654, 2016. 5

    2016

  25. [25]

    Visual genome: Connecting language and vision using crowdsourced dense image annotations.International journal of computer vision, 123(1):32–73, 2017

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalan- tidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations.International journal of computer vision, 123(1):32–73, 2017. 6

    2017

  26. [26]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Zi- wei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024. 1

    Pith/arXiv arXiv 2024

  27. [27]

    Dyfo: A training-free dynamic focus visual search for enhancing lmms in fine-grained visual understanding

    Geng Li, Jinglin Xu, Yunzhen Zhao, and Yuxin Peng. Dyfo: A training-free dynamic focus visual search for enhancing lmms in fine-grained visual understanding. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 9098–9108, 2025. 2, 8

    2025

  28. [28]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InIn- ternational conference on machine learning, pages 19730– 19742. PMLR, 2023. 1, 8

    2023

  29. [29]

    Evaluating object hallucina- tion in large vision-language models.arXiv preprint arXiv:2305.10355, 2023

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucina- tion in large vision-language models.arXiv preprint arXiv:2305.10355, 2023. 6

    Pith/arXiv arXiv 2023

  30. [30]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer, 2014. 6

    2014

  31. [31]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023. 8

    2023

  32. [32]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26296–26306, 2024. 1, 2, 6

    2024

  33. [33]

    Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024. 1

    2024

  34. [34]

    Mmbench: Is your multi-modal model an all-around player? InEuropean conference on computer vi- sion, pages 216–233

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? InEuropean conference on computer vi- sion, pages 216–233. Springer, 2024. 6

    2024

  35. [35]

    Learn to explain: Multimodal reasoning via thought chains for science question answering.Advances in Neural Information Processing Systems, 35:2507–2521,

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering.Advances in Neural Information Processing Systems, 35:2507–2521,

  36. [36]

    Neuronal mechanisms of visual atten- tion.Annual review of vision science, 1(1):373–391, 2015

    John HR Maunsell. Neuronal mechanisms of visual atten- tion.Annual review of vision science, 1(1):373–391, 2015. 2

    2015

  37. [37]

    Ocr-vqa: Visual question answering by reading text in images

    Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. Ocr-vqa: Visual question answering by reading text in images. In2019 international conference on document analysis and recognition (ICDAR), pages 947–

  38. [38]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021. 8

    2021

  39. [39]

    Deepspeed: System optimizations enable train- ing deep learning models with over 100 billion parameters

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable train- ing deep learning models with over 100 billion parameters. InKDD ’20: The 26th ACM SIGKDD Conference on Knowl- edge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pages 3505–3506. ACM, 2020. 1

    2020

  40. [40]

    Towards vqa models that can read

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8317–8326, 2019. 6

    2019

  41. [41]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of con- text.arXiv preprint arXiv:2403.05530, 2024

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of con- text.arXiv preprint arXiv:2403.05530, 2024. 1

    Pith/arXiv arXiv 2024

  42. [42]

    Eyes wide shut? exploring the visual shortcomings of multimodal llms, 2024

    Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms, 2024. 1, 8

    2024

  43. [43]

    Vl-cache: Sparsity and modality-aware kv cache com- pression for vision-language model inference acceleration

    Dezhan Tu, Danylo Vashchilenko, Yuzhe Lu, and Panpan Xu. Vl-cache: Sparsity and modality-aware kv cache com- pression for vision-language model inference acceleration. arXiv preprint arXiv:2410.23317, 2024. 3

    Pith/arXiv arXiv 2024

  44. [44]

    Transformers: State-of-the-art natural language processing

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chau- mond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of-the-art ...

    2020

  45. [45]

    V*: Guided visual search as a core mechanism in multimodal llms, 2023.URL https://arxiv

    Penghao Wu and Saining Xie. V*: Guided visual search as a core mechanism in multimodal llms, 2023.URL https://arxiv. org/abs/2312.14135, 5. 1, 2, 8

    Pith/arXiv arXiv 2023

  46. [46]

    Grounded chain-of-thought for multimodal large language models,

    Qiong Wu, Xiangcong Yang, Yiyi Zhou, Chenxin Fang, Baiyang Song, Xiaoshuai Sun, and Rongrong Ji. Grounded chain-of-thought for multimodal large language models,

  47. [47]

    Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy re- duction.arXiv preprint arXiv:2410.17247, 2024

    Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, et al. Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy re- duction.arXiv preprint arXiv:2410.17247, 2024. 3

    Pith/arXiv arXiv 2024

  48. [48]

    Fit and prune: Fast and training-free visual token pruning for multi- modal large language models

    Weihao Ye, Qiong Wu, Wenhao Lin, and Yiyi Zhou. Fit and prune: Fast and training-free visual token pruning for multi- modal large language models. InProceedings of the AAAI Conference on Artificial Intelligence, pages 22128–22136,

  49. [49]

    Introducing vi- sual perception token into multimodal large language model

    Runpeng Yu, Xinyin Ma, and Xinchao Wang. Introducing vi- sual perception token into multimodal large language model. arXiv preprint arXiv:2502.17425, 2025. 2, 9

    Pith/arXiv arXiv 2025

  50. [50]

    Mm-vet: Evaluating large multimodal models for integrated capabilities.arXiv preprint arXiv:2308.02490, 2023

    Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities.arXiv preprint arXiv:2308.02490, 2023. 6

    Pith/arXiv arXiv 2023

  51. [51]

    Mllms know where to look: Training-free per- ception of small visual details with multimodal llms.arXiv preprint arXiv:2502.17422, 2025

    Jiarui Zhang, Mahyar Khayatkhoei, Prateek Chhikara, and Filip Ilievski. Mllms know where to look: Training-free per- ception of small visual details with multimodal llms.arXiv preprint arXiv:2502.17422, 2025. 2, 9

    Pith/arXiv arXiv 2025

  52. [52]

    Open eyes, then reason: Fine-grained visual mathematical understanding in mllms, 2025

    Shan Zhang, Aotian Chen, Yanpeng Sun, Jindong Gu, Yi- Yu Zheng, Piotr Koniusz, Kai Zou, Anton van den Hengel, and Yuan Xue. Open eyes, then reason: Fine-grained visual mathematical understanding in mllms, 2025. 2 Blink: Dynamic Visual Token Resolution for Enhanced Multimodal Understanding Supplementary Material A. Training Details A.1. Configurations For ...

    arXiv 2025

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.