Pith. sign in

REVIEW 3 major objections 5 minor 87 references

High-resolution VQA fails less because evidence is missing than because already-encoded evidence gets diluted before the answer is generated.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 00:35 UTC pith:GAG6SZH2

load-bearing objection Training-free HR-VQA routing method with consistent gains; the method deserves serious review, but the 'evidence routing' mechanism is not fully isolated from compression. the 3 major comments →

arxiv 2607.27830 v1 pith:GAG6SZH2 submitted 2026-07-30 cs.CV

Thinking Once Is Enough: Intermediate-Layer Evidence Routing for High-Resolution VQA

Zhongkuan Mao , Xianjie Liu , Tianyu Meng , Yidong Wang , Wenzhuo Zhao , Ronghao Xian , Yao Jiang , Fei Shen
show 4 more authors
Junfeng Fang Yong Dai Yi Zhang Keren Fu
This is my paper
classification cs.CV
keywords high-resolution VQAevidence routingintermediate layersquestion-conditioned attentiontoken reductiontraining-free inferencemultimodal LLMsingle visual pass
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that fine-grained visual evidence for high-resolution VQA often survives initial encoding but gets diluted in later layers, so failures are not only about missing information. It proposes Thinking-Once, a training-free method that, at an intermediate layer, reconstructs attention from the question to visual tokens, keeps the tokens that matter, summarizes the rest into background cells, and routes this compact set through the remaining layers. Across five multimodal LLMs, this raises V*Bench, HRBench-4K, and HRBench-8K by about 3 points each while cutting peak memory by roughly 4 GB. The gains are largest on spatial and cross-instance questions, supporting the claim that preserved relational context, not token count, is what helps.

Core claim

The central discovery is that the model's own intermediate layers already contain identifiable, question-relevant visual evidence, and that this evidence is subsequently lost or diluted before answer generation. Intervening at that layer — preserving the evidence core at full resolution, keeping coarse spatial context through grid pooling, and letting the remaining layers run on this compact sequence — both improves accuracy and reduces memory. The paper shows this on five base models and three benchmarks, and the effect is strongest where the answer depends on relations or comparisons between objects.

What carries the argument

The routing operator at layer L: (1) it reconstructs the exact causal attention row from each routing query (entity tokens from the question plus a global question token) to all visual tokens, averaged over heads; (2) for each query it takes the minimum-cardinality token set covering a fraction ρ of the attention mass, then unions these per-query core sets; (3) it mean-pools unselected tokens inside each 8×8 grid cell into one background token; and (4) it merges core and background tokens in raster order, keeping MRoPE coordinates, so later layers see a shorter but spatially structured sequence. The formal guarantees are that routing is causally safe and never lengthens the sequence.

Load-bearing premise

The attention probabilities that the routing queries read at layer L truly mark the visual tokens that matter for the answer; if they are miscalibrated for a question, the core set drops needed evidence and routing cannot recover it.

What would settle it

Run Thinking-Once but replace the question-conditioned selection with the same number of tokens chosen uniformly at random from the same intermediate layer. If random selection matches the accuracy gains on V*Bench and HRBench, then the specific routing mechanism is not the cause of improvement; if it falls short, the claim that query-conditioned evidence selection drives the gains is supported.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • HR-VQA benchmarks can be improved without changing the visual input pipeline: one full-image encoding and one forward pass suffice.
  • Peak memory drops by roughly 4 GB on average because layers past the routing point process a shorter KV cache.
  • Question-conditioned routing helps most on relation, comparison, and cross-instance questions, not just isolated attributes.
  • The method generalizes across different MLLM families, suggesting the dilution phenomenon is architectural, not model-specific.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The routing layer L could be chosen adaptively per sample or per question type; the paper fixes it per model, but the evidence-routing window argument implies a tunable knob.
  • If the dilution account is right, routing gains should grow with image clutter; that is a testable prediction the paper states but does not fully quantify across a clutter spectrum.
  • The same evidence-routing idea could extend to video or multi-image inputs, where per-query visible sets are already handled in the appendix's generalization.
  • A practical extension would cache entity extraction or route multiple questions from one image in a single pass, since the current pipeline invokes the model twice sequentially.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper argues that a substantial class of HR-VQA failures is caused not by missing visual evidence but by dilution or displacement of evidence that has already been encoded and localized in intermediate layers. It proposes Thinking-Once, a training-free, single-visual-pass method that, at a chosen intermediate layer L, reconstructs question-conditioned attention probabilities for entity queries and a global question query, selects a per-query minimum-coverage core set with threshold ρ, summarizes unselected tokens into 8×8 background grid tokens via mean pooling, and routes the compact sequence through deeper layers. The method is evaluated on V*Bench, HRBench-4K, and HRBench-8K across five MLLMs, reporting average gains of +3.1/+3.0/+2.7 points and roughly 4 GB memory savings, plus comparisons with 11 baselines. The appendix provides formal properties of the routing operator (normalization, coverage, causal safety, length bound) and controlled evidence-coverage analyses.

Significance. If the results hold, the paper makes a valuable conceptual contribution: it reframes high-resolution VQA from an evidence-acquisition problem to an evidence-utilization problem and offers a simple, inference-time intervention that improves accuracy and memory across multiple models without additional visual encoding. The strengths are substantial: the layer-wise oracle interventions with same-budget random and GT-core tokens (Fig. 2), the matched-budget coverage analyses of routing and context allocation (Appendix F, Table 12), the controlled compression-first comparison at reduced resolution (Table 9), and the explicit formal lemmas with a clear statement of what they do not establish. The paper is also honest about its utilization-limited boundary and gives falsifiable predictions. However, the central mechanistic interpretation is not yet conclusively separated from a compression effect, and the experimental reporting lacks variance and selection details; these are addressable with additional experiments.

major comments (3)
  1. [§5.3 / Appendix F (Table 9, Table 12)] The central claim is that question-conditioned attention routing, not sequence shortening, drives the gains. The evidence stops short of this claim: the matched-budget analysis (Table 12) measures GT-evidence recall and spatial coverage, not end-task accuracy, and Table 9 matches input resolution and retention ratio for compression baselines but does not include a random-core variant of Thinking-Once with the same L, ρ, background grid, and total token count. The oracle intervention (Fig. 2) shows token identity matters when GT boxes are used, but not that the attention-derived p_j identifies the right tokens at inference. I ask for an ablation that keeps all components identical except replacing top-p core selection with random selection at the same per-sample core budget, and reports the three benchmark accuracies. If random selection matches Full, the 'evidence routing' interpretation
  2. [§5.1–5.2, Appendix F (Fig. 7)] L and ρ are free parameters and their selection protocol is not described. Appendix F reports L15 for Qwen2.5-VL-7B and L19 for Qwen3-VL-8B, and ρ=0.7 for the enrichment diagnostic, but the main experiments do not state how L, ρ, or the 8×8 grid were chosen, whether on a validation split, and over what range. Since the method is training-free, the risk is that per-model tuning on test benchmarks contributes to the +2.9 average. Please report the selection procedure and a sensitivity table over L and ρ for at least one model (e.g., Qwen2.5-VL-7B). Relatedly, Table 1 reports single runs with no error bars or significance tests; several entries are ties or small deltas (e.g., ZwZ-4B V* 90.4 vs 90.4), so a repeated-seed or paired analysis would strengthen the claim.
  3. [§4.1, Appendix C ('Solution for empty entity extraction')] The entity extractor is a second failure point. If it returns empty, the fallback uses the cleaned question stem, and Table 3 shows 'Global only' (a deliberately query-ablated variant) is below Base on the combined mean (71.2 vs 72.5). This means the method's benefit is contingent on extractor success, since the global query alone is insufficient. Please report the entity-extraction success/failure rate on the three benchmarks and an oracle-entity ablation where routing queries are built from ground-truth or human-provided entities. This bounds the impact of extraction errors and would also clarify whether attention-derived selection can be trusted when the extractor succeeds.
minor comments (5)
  1. [§4.1, Eq. (2)] The main text should specify that the softmax is taken over the full visible key set and then restricted to visual positions; Appendix A is precise, but the main text can be misread as renormalizing the visual slice before head averaging.
  2. [§5.1, Table 1] Memory is reported as '~20 GB'/'~16 GB' with no measurement details. State whether these are dataset-averaged peak GPU usages, single-GPU, and how the rounding is performed.
  3. [Abstract / Appendix C] The 'training-free' label is slightly misleading because a separate text-only invocation is used for entity extraction; the appendix clarifies this, but consider using 'inference-time, no gradient updates' in the abstract to avoid confusion.
  4. [Appendix F, Fig. 7 caption] The caption says dashed lines and yellow markers denote outputs of the routing layers, but the text says routing is applied after the marked layer. Reconcile the wording so the first affected layer (L+1) is unambiguous.
  5. [Appendix F, Eq. (29) context] The mass-preserving back-projection of background-summary attention redistributes mass uniformly over the source set U_g; state this assumption explicitly, as it is a diagnostic choice rather than an exact inversion of the attention operation.

Circularity Check

0 steps flagged

No circular derivation: the central claim rests on external benchmarks and controlled interventions, not on the paper's own definitions.

full rationale

The paper's derivation chain is not circular. Thinking-Once is defined by a routing operator (Eqs. 1-8) whose formal properties in Appendix A (normalization, minimum coverage, causal safety, length bound) are true by construction; however, the paper does not use these properties to prove accuracy. Appendix A explicitly states: 'These properties do not establish that p_j(i) exactly measures causal importance... Those claims require empirical validation.' The central claim—that routing already-encoded evidence improves HR-VQA—is supported by external benchmark evaluations (V*Bench, HRBench), ablations, and controlled oracle interventions using ground-truth boxes as an evaluation oracle, not as routing input. The routing queries are derived from the question and the model's own attention, not from the benchmark labels or from the reported gains. The matched-budget coverage analyses (Table 12) compare selection strategies under fixed token counts, so the reported advantages are not forced by construction. The paper's self-citations (if any) are not load-bearing: the supporting citations to LongCat-Next, DeepEyes, TreeVGR, and Ma et al. are external published results used only to motivate the utilization-limited regime, and the formal appendix disclaims any stronger role. No 'prediction' is equivalent by definition to a fitted input; no uniqueness theorem is imported from the authors' prior work; no known result is merely renamed. The most serious limitation—that p_j may not measure causal importance—is acknowledged by the authors and would be a soundness concern, not a circularity concern. Therefore, no circular step can be exhibited, and the score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 0 invented entities

The central claim rests on the empirical effectiveness of attention-derived evidence selection, not on any new physical or mathematical entity. The main uncharged assumptions are the reliability of attention as an evidence proxy, the existence and choice of a usable routing layer, and the sufficiency of grid-pooled background. These are plausible and partially validated by controlled analyses, but ρ and L are not derived from a validation protocol.

free parameters (4)
  • Routing layer L per model = L15 for Qwen2.5-VL-7B, L19 for Qwen3-VL-8B; other models not fully specified
    The choice of intermediate layer is load-bearing and appears to be selected per model; no validation protocol is given.
  • Coverage threshold ρ = 0.7 appears in an appendix diagnostic; main-method value not stated
    Controls the size of the core token set; if tuned on test benchmarks, gains are optimistic.
  • Background grid size = 8×8
    Chosen by hand for mean-pooling background context; not swept in the paper.
  • Entity extraction prompt and anchor rule = last non-punctuation token per entity; template in appendix
    Design choice that affects routing accuracy and requires an extra text-only model call.
axioms (6)
  • domain assumption Head-averaged attention probabilities p_j(i) at layer L approximate question-relevant visual evidence.
    The whole routing mechanism relies on this; Appendix A explicitly says p_j is not proven to measure causal importance.
  • domain assumption The vision encoder retains fine-grained, recoverable information from the original image.
    Adopted from LongCat-Next reconstruction evidence and stated as a necessary condition for single-visual-pass routing.
  • domain assumption There exists an intermediate layer where evidence is identifiable and still influential on later layers.
    Supported by oracle interventions, but the practical layer must be chosen without ground-truth boxes.
  • domain assumption Mean-pooling unselected tokens into 8×8 background cells preserves enough spatial and relational context.
    The ablations show background helps, but the grid resolution is a fixed choice not derived from a principle.
  • domain assumption The text-only entity extractor reliably identifies all question-relevant entities.
    Entity queries are the main localization signal; the fallback to question-derived phrases is deterministic but not guaranteed to recover missed entities.
  • domain assumption The last non-punctuation question token summarizes the whole question for the global query.
    Assumed under causal self-attention; used to condition on general attribute/relation demands.

pith-pipeline@v1.3.0-daily-deepseek · 34994 in / 9352 out tokens · 97762 ms · 2026-08-01T00:35:33.047822+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Thinking Once Is Enough: Intermediate-Layer Evidence Routing for High-Resolution VQA." pith.science (2026). https://pith.science/paper/GAG6SZH2

@misc{pith2026260727830,
  author       = {Pith},
  title        = {Pith review of: Thinking Once Is Enough: Intermediate-Layer Evidence Routing for High-Resolution VQA},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GAG6SZH2}},
  note         = {Machine review of arXiv:2607.27830}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

High-resolution visual question answering (HR-VQA) is often treated as a problem of insufficient evidence acquisition, where failing multimodal large language models must inspect images again through cropping, re-encoding, or multi-round search. We show that this view is incomplete: in many cases, fine-grained evidence has already survived visual encoding and become identifiable and influential within an intermediate-layer routing window, but is later diluted before answer generation. We propose Thinking-Once, a \textbf{training-free, single-visual-pass} evidence-routing method that reconstructs question-conditioned attention at this window, preserves core entity tokens and compact background context, and routes this evidence to later layers without extra visual encoding. Across five base models, Thinking-Once consistently improves or matches the corresponding base setting, increasing the average scores on V$^*$Bench, HRBench-4K, and HRBench-8K by \textit{+3.1}, \textit{+3.0}, and \textit{+2.7} points while reducing the average peak memory by about 4,GB. On Qwen2.5-VL-7B, it improves the three benchmarks by \textit{+9.9}, \textit{+4.6}, and \textit{+5.5} points, raising the cross-benchmark mean from 72.5 to 79.1. With the ZwZ-8B base model, Thinking-Once reaches a mean score of 82.7. Against 11 open-source HR-VQA baselines, it obtains the best or tied-best score on all three benchmark averages and the best overall mean; for example, compared with DeepScan, it reduces V$^*$Bench inference time by \textbf{97.2\%} while improving the cross-benchmark mean from 77.8 to 79.1. These results show that HR-VQA can be improved by routing already encoded evidence rather than repeatedly acquiring new visual inputs. Code is available in the appendix.

Figures

Figures reproduced from arXiv: 2607.27830 by Fei Shen, Junfeng Fang, Keren Fu, Ronghao Xian, Tianyu Meng, Wenzhuo Zhao, Xianjie Liu, Yao Jiang, Yidong Wang, Yi Zhang, Yong Dai, Zhongkuan Mao.

Figure 1
Figure 1. Figure 1: The base model gradually loses focus on the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Layer-wise oracle token intervention on V [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 5
Figure 5. Figure 5: Left: GT-evidence recall across query-routing [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Overview of Thinking-Once. At a routing layer, question-conditioned queries independently select core evidence, [PITH_FULL_IMAGE:figures/full_fig_p005_6.png] view at source ↗
Figure 1
Figure 1. Figure 1: Supporting evidence from prior work on visual reconstruction from frozen vision encoders. A lightweight pixel [PITH_FULL_IMAGE:figures/full_fig_p015_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Supporting evidence from the MED analysis of vision tool-use reinforcement learning. The analysis decomposes [PITH_FULL_IMAGE:figures/full_fig_p018_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Single-object attribute case. The base model predicts the wrong color for a small motorcycle in a high-resolution [PITH_FULL_IMAGE:figures/full_fig_p024_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Multi-object relational case. The base model gives an incorrect spatial answer, whereas Thinking-Once extracts both [PITH_FULL_IMAGE:figures/full_fig_p025_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Layer-wise qualitative example of question-guided evidence localization. The visualization shows how the model [PITH_FULL_IMAGE:figures/full_fig_p026_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Additional layer-wise qualitative example of question-guided evidence localization. The example further illustrates [PITH_FULL_IMAGE:figures/full_fig_p027_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Layer-wise object-attention enrichment on V [PITH_FULL_IMAGE:figures/full_fig_p029_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

87 extracted references · 8 linked inside Pith

  1. [1]

    2026 , eprint=

    LongCat-Next: Lexicalizing Modalities as Discrete Tokens , author=. 2026 , eprint=

    2026

  2. [2]

    The Fourteenth International Conference on Learning Representations , year=

    DeepEyes: Incentivizing ''Thinking with Images'' via Reinforcement Learning , author=. The Fourteenth International Conference on Learning Representations , year=

  3. [3]

    Forty-third International Conference on Machine Learning Position Paper Track , year=

    Position: Your VLM May Not Be Thinking with Interleaved Images , author=. Forty-third International Conference on Machine Learning Position Paper Track , year=

  4. [4]

    HiDe: Rethinking The Zoom-

    Liu, Xianjie and Hu, Yiman and Zou, Yixiong and Wu, Liang and Xu, Jian and Zheng, Bo , booktitle=. HiDe: Rethinking The Zoom-. 2026 , url=

    2026

  5. [5]

    International Conference on Machine Learning , year=

    Reliable Thinking with Images , author=. International Conference on Machine Learning , year=

  6. [6]

    International Conference on Learning Representations , volume=

    Mllms know where to look: Training-free perception of small visual details with multimodal llms , author=. International Conference on Learning Representations , volume=

  7. [7]

    Advances in Neural Information Processing Systems , volume=

    Vlm-r ^3 : Region recognition, reasoning, and refinement for enhanced multimodal chain-of-thought , author=. Advances in Neural Information Processing Systems , volume=

  8. [8]

    Proceedings of the AAAI Conference on Artificial Intelligence , number=

    HiPrune: Training-Free Visual Token Pruning via Hierarchical Attention in Vision-Language Models (Student Abstract) , author=. Proceedings of the AAAI Conference on Artificial Intelligence , number=

  9. [11]

    The Fourteenth International Conference on Learning Representations , year=

    Traceable evidence enhanced visual grounded reasoning: Evaluation and method , author=. The Fourteenth International Conference on Learning Representations , year=

  10. [12]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Variation-aware vision token dropping for faster large vision-language models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  11. [13]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Deepscan: A training-free framework for visually grounded reasoning in large vision-language models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  12. [14]

    Token Merging: Your

    Bolya, Daniel and Fu, Cheng-Yang and Dai, Xiaoliang and Zhang, Peizhao and Feichtenhofer, Christoph and Hoffman, Judy , booktitle=. Token Merging: Your

  13. [15]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    V*: Guided visual search as a core mechanism in multimodal llms , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  14. [16]

    Proceedings of the AAAI Conference on Artificial Intelligence , number=

    Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models , author=. Proceedings of the AAAI Conference on Artificial Intelligence , number=

  15. [18]

    2025 , howpublished =

    OpenAI , title =. 2025 , howpublished =

    2025

  16. [19]

    2025 , eprint=

    Qwen2.5-VL Technical Report , author=. 2025 , eprint=

    2025

  17. [21]

    Proceedings of the 2023 conference on empirical methods in natural language processing , pages=

    Evaluating object hallucination in large vision-language models , author=. Proceedings of the 2023 conference on empirical methods in natural language processing , pages=

    2023

  18. [22]

    International Conference on Learning Representations , volume=

    Mme-realworld: Could your multimodal llm challenge high-resolution real-world scenarios that are difficult for humans? , author=. International Conference on Learning Representations , volume=

  19. [24]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Visionzip: Longer is better but not necessary in vision language models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  20. [25]

    Proceedings of the 33rd ACM International Conference on Multimedia , pages=

    Nexus-o: An omni-perceptive and-interactive model for language, audio, and vision , author=. Proceedings of the 33rd ACM International Conference on Multimedia , pages=

  21. [26]

    2023 , eprint=

    Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond , author=. 2023 , eprint=

    2023

  22. [27]

    European Conference on Computer Vision , pages=

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  23. [28]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Llava-prumerge: Adaptive token reduction for efficient large multimodal models , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  24. [29]

    International Journal of Computer Vision , volume=

    Tokenpacker: Efficient visual projector for multimodal llm , author=. International Journal of Computer Vision , volume=. 2025 , publisher=

    2025

  25. [30]

    Proceedings of the AAAI Conference on Artificial Intelligence , number=

    Filter, correlate, compress: Training-free token reduction for mllm acceleration , author=. Proceedings of the AAAI Conference on Artificial Intelligence , number=

  26. [31]

    Transactions on Machine Learning Research , year=

    VScan: Rethinking Visual Token Reduction for Efficient Large Vision-Language Models , author=. Transactions on Machine Learning Research , year=

  27. [32]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Zoomeye: Enhancing multimodal llms with human-like zooming capabilities through tree-based image exploration , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  28. [33]

    International conference on machine learning , pages=

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models , author=. International conference on machine learning , pages=. 2023 , organization=

    2023

  29. [34]

    Advances in neural information processing systems , volume=

    Instructblip: Towards general-purpose vision-language models with instruction tuning , author=. Advances in neural information processing systems , volume=

  30. [35]

    International Conference on Learning Representations , volume=

    Minigpt-4: Enhancing vision-language understanding with advanced large language models , author=. International Conference on Learning Representations , volume=

  31. [36]

    Advances in neural information processing systems , volume=

    Flamingo: a visual language model for few-shot learning , author=. Advances in neural information processing systems , volume=

  32. [37]

    Advances in neural information processing systems , volume=

    Visual instruction tuning , author=. Advances in neural information processing systems , volume=

  33. [38]

    International Conference on Learning Representations , volume=

    Efficient streaming language models with attention sinks , author=. International Conference on Learning Representations , volume=

  34. [39]

    International conference on learning representations , volume=

    Vision transformers need registers , author=. International conference on learning representations , volume=

  35. [40]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Dyfo: A training-free dynamic focus visual search for enhancing lmms in fine-grained visual understanding , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  36. [41]

    The Thirteenth International Conference on Learning Representations , year=

    See What You Are Told: Visual Attention Sink in Large Multimodal Models , author=. The Thirteenth International Conference on Learning Representations , year=

  37. [42]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Selfelicit: Your language model secretly knows where is the relevant evidence , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  38. [43]

    Advances in Neural Information Processing Systems , volume=

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? , author=. Advances in Neural Information Processing Systems , volume=

  39. [44]

    International Journal of Computer Vision , volume=

    Exploring vision-language models for imbalanced learning , author=. International Journal of Computer Vision , volume=. 2024 , publisher=

    2024

  40. [45]

    Advances in Neural Information Processing Systems , volume=

    Balanced token pruning: Accelerating vision language models beyond local optimization , author=. Advances in Neural Information Processing Systems , volume=

  41. [47]

    arXiv preprint arXiv:2508.11630 , year=

    Thyme: Think beyond images , author=. arXiv preprint arXiv:2508.11630 , year=

    Pith/arXiv arXiv

  42. [48]

    International Conference on Learning Representations , volume=

    Flashattention-2: Faster attention with better parallelism and work partitioning , author=. International Conference on Learning Representations , volume=

  43. [49]

    Alayrac, J.-B.; Donahue, J.; Luc, P.; Miech, A.; Barr, I.; Hasson, Y.; Lenc, K.; Mensch, A.; Millican, K.; Reynolds, M.; et al. 2022. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35: 23716--23736

    2022

  44. [50]

    Bai, J.; Bai, S.; Yang, S.; Wang, S.; Tan, S.; Wang, P.; Lin, J.; Zhou, C.; and Zhou, J. 2023. Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond. arXiv:2308.12966

    Pith/arXiv arXiv 2023

  45. [51]

    Bai, S.; Cai, Y.; Chen, R.; Chen, K.; Chen, X.; Cheng, Z.; Deng, L.; Ding, W.; Gao, C.; Ge, C.; et al. 2025 a . Qwen3-vl technical report. arXiv preprint arXiv:2511.21631

    Pith/arXiv arXiv 2025

  46. [52]

    Bai, S.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Song, S.; Dang, K.; Wang, P.; Wang, S.; Tang, J.; Zhong, H.; Zhu, Y.; Yang, M.; Li, Z.; Wan, J.; Wang, P.; Ding, W.; Fu, Z.; Xu, Y.; Ye, J.; Zhang, X.; Xie, T.; Cheng, Z.; Zhang, H.; Yang, Z.; Xu, H.; and Lin, J. 2025 b . Qwen2.5-VL Technical Report. arXiv:2502.13923

    Pith/arXiv arXiv 2025

  47. [53]

    Bolya, D.; Fu, C.-Y.; Dai, X.; Zhang, P.; Feichtenhofer, C.; and Hoffman, J. 2023. Token Merging: Your ViT but Faster. In International Conference on Learning Representations

    2023

  48. [54]

    Chen, J.; Liu, X.; Wen, Z.; Wang, Y.; Huang, S.; and Chen, H. 2026 a . Variation-aware vision token dropping for faster large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3489--3499

    2026

  49. [55]

    Chen, L.; Zhao, H.; Liu, T.; Bai, S.; Lin, J.; Zhou, C.; and Chang, B. 2024. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. In European Conference on Computer Vision, 19--35. Springer

    2024

  50. [56]

    Chen, X.; Gao, C.; Li, Y.; Chen, X.; et al. 2026 b . Balanced token pruning: Accelerating vision language models beyond local optimization. Advances in Neural Information Processing Systems, 38: 62740--62764

    2026

  51. [57]

    N.; and Hoi, S

    Dai, W.; Li, J.; Li, D.; Tiong, A.; Zhao, J.; Wang, W.; Li, B.; Fung, P. N.; and Hoi, S. 2023. Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in neural information processing systems, 36: 49250--49267

    2023

  52. [58]

    Dao, T. 2024. Flashattention-2: Faster attention with better parallelism and work partitioning. In International Conference on Learning Representations, volume 2024, 35549--35562

    2024

  53. [59]

    Darcet, T.; Oquab, M.; Mairal, J.; and Bojanowski, P. 2024. Vision transformers need registers. In International conference on learning representations, volume 2024, 2632--2652

    2024

  54. [60]

    Han, Y.; Liu, X.; Zhang, Z.; Ding, P.; Chen, J.; Chen, H.; Wang, D.; Yan, Q.; and Huang, S. 2026. Filter, correlate, compress: Training-free token reduction for mllm acceleration. In Proceedings of the AAAI Conference on Artificial Intelligence, 6, 4601--4609

    2026

  55. [61]

    P.; Perelman, A.; Ramesh, A.; Clark, A.; Ostrow, A.; Welihinda, A.; Hayes, A.; Radford, A.; et al

    Hurst, A.; Lerer, A.; Goucher, A. P.; Perelman, A.; Ramesh, A.; Clark, A.; Ostrow, A.; Welihinda, A.; Hayes, A.; Radford, A.; et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    Pith/arXiv arXiv 2024

  56. [62]

    Jiang, C.; Heng, Y.; Ye, W.; Xu, H.; Yan, M.; Zhang, J.; Huang, F.; and Zhang, S. 2026. Vlm-r ^3 : Region recognition, reasoning, and refinement for enhanced multimodal chain-of-thought. Advances in Neural Information Processing Systems, 38: 63841--63869

    2026

  57. [63]

    Kang, S.; Kim, J.; Kim, J.; and Hwang, S. J. 2025. See What You Are Told: Visual Attention Sink in Large Multimodal Models. In The Thirteenth International Conference on Learning Representations

    2025

  58. [64]

    Khayatkhoei, M.; Chhikara, P.; Ilievski, F.; et al. 2025. Mllms know where to look: Training-free perception of small visual details with multimodal llms. In International Conference on Learning Representations, volume 2025, 68194--68213

    2025

  59. [65]

    Li, G.; Xu, J.; Zhao, Y.; and Peng, Y. 2025 a . Dyfo: A training-free dynamic focus visual search for enhancing lmms in fine-grained visual understanding. In Proceedings of the Computer Vision and Pattern Recognition Conference, 9098--9108

    2025

  60. [66]

    Li, H.; Yang, Y.; Lin, Y.; Dai, X.; Yang, M.; and Peng, X. 2026 a . Reliable Thinking with Images. In International Conference on Machine Learning

    2026

  61. [67]

    Li, J.; Li, D.; Savarese, S.; and Hoi, S. 2023 a . Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, 19730--19742. PMLR

    2023

  62. [68]

    Li, W.; Yuan, Y.; Liu, J.; Tang, D.; Wang, S.; Qin, J.; Zhu, J.; and Zhang, L. 2025 b . Tokenpacker: Efficient visual projector for multimodal llm. International Journal of Computer Vision, 133(10): 6794--6812

    2025

  63. [69]

    Li, Y.; Du, Y.; Zhou, K.; Wang, J.; Zhao, X.; and Wen, J.-R. 2023 b . Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 conference on empirical methods in natural language processing, 292--305

    2023

  64. [70]

    Li, Y.; Zhan, H.; Chen, J.; Gong, Y.; Liu, Q.; and Lu, Y. 2026 b . Deepscan: A training-free framework for visually grounded reasoning in large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19155--19164

    2026

  65. [71]

    Liu, C.; Zhang, Y.; Zhang, D.; Zhang, W.; Gong, C.; Lu, Y.; Zhou, S.; Gan, Z.; Wang, Z.; Wu, H.; et al. 2025 a . Nexus-o: An omni-perceptive and-interactive model for language, audio, and vision. In Proceedings of the 33rd ACM International Conference on Multimedia, 10787--10796

    2025

  66. [72]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2023. Visual instruction tuning. Advances in neural information processing systems, 36: 34892--34916

    2023

  67. [73]

    Liu, J.; Zhu, G.; and Du, F. 2026. HiPrune: Training-Free Visual Token Pruning via Hierarchical Attention in Vision-Language Models (Student Abstract). In Proceedings of the AAAI Conference on Artificial Intelligence, 48, 41275--41277

    2026

  68. [74]

    Liu, X.; Hu, Y.; Zou, Y.; Wu, L.; Xu, J.; and Zheng, B. 2026. HiDe: Rethinking The Zoom- IN method in High Resolution MLLM s via Hierarchical Decoupling. In Forty-third International Conference on Machine Learning

    2026

  69. [75]

    A.; Adkathimar, R.; Wei, T.; and Tong, H

    Liu, Z.; Amjad, R. A.; Adkathimar, R.; Wei, T.; and Tong, H. 2025 b . Selfelicit: Your language model secretly knows where is the relevant evidence. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 9153--9173

    2025

  70. [76]

    Ma, Y.; Zhang, W.; Li, T.; Du, L.; Shen, X.; and Liu, P. 2026. What Does Vision Tool-Use Reinforcement Learning Really Learn? Disentangling Tool-Induced and Intrinsic Effects for Crop-and-Zoom. arXiv preprint arXiv:2602.01334

    Pith/arXiv arXiv 2026

  71. [77]

    J.; and Yan, Y

    Shang, Y.; Cai, M.; Xu, B.; Lee, Y. J.; and Yan, Y. 2025. Llava-prumerge: Adaptive token reduction for efficient large multimodal models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 22857--22867

    2025

  72. [78]

    Shen, H.; Zhao, K.; Zhao, T.; Xu, R.; Zhang, Z.; Zhu, M.; and Yin, J. 2025. Zoomeye: Enhancing multimodal llms with human-like zooming capabilities through tree-based image exploration. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 6613--6629

    2025

  73. [79]

    Team, M. L. 2026. LongCat-Next: Lexicalizing Modalities as Discrete Tokens. arXiv:2603.27538

    arXiv 2026

  74. [80]

    Wang, H.; Li, X.; Huang, Z.; Wang, A.; Wang, J.; Zhang, T.; Bai, S.; Kang, Z.; Feng, J.; Zhuochen, W.; et al. 2026. Traceable evidence enhanced visual grounded reasoning: Evaluation and method. In The Fourteenth International Conference on Learning Representations

    2026

  75. [81]

    Wang, W.; Ding, L.; Zeng, M.; Zhou, X.; Shen, L.; Luo, Y.; Yu, W.; and Tao, D. 2025. Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, 8, 7907--7915

    2025

  76. [82]

    Wang, Y.; Yu, Z.; Wang, J.; Heng, Q.; Chen, H.; Ye, W.; Xie, R.; Xie, X.; and Zhang, S. 2024. Exploring vision-language models for imbalanced learning. International Journal of Computer Vision, 132(1): 224--237

    2024

  77. [83]

    Wei, L.; He, L.; Lan, J.; Dong, L.; Cai, Y.; Li, S.; Zhu, H.; Wang, W.; Kong, L.; Wang, Y.; et al. 2026. Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception. arXiv preprint arXiv:2602.11858

    arXiv 2026

  78. [84]

    Wu, P.; and Xie, S. 2024. V*: Guided visual search as a core mechanism in multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13084--13094

    2024

  79. [85]

    Xiao, G.; Tian, Y.; Chen, B.; Han, S.; and Lewis, M. 2024. Efficient streaming language models with attention sinks. In International Conference on Learning Representations, volume 2024, 21875--21895

    2024

  80. [86]

    Yang, S.; Chen, Y.; Tian, Z.; Wang, C.; Li, J.; Yu, B.; and Jia, J. 2025. Visionzip: Longer is better but not necessary in vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 19792--19802

    2025

Showing first 80 references.