Pith. sign in

REVIEW 4 major objections 5 minor 42 references

Capturing Token Tendencies for Training-Free Token Pruning in Multimodal Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-07-31 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Token importance in multimodal LLMs evolves across layers, so pruning must track attention trajectories and reopen discarded tokens.

desk verdict Solid training-free MLLM pruning paper: trajectory reactivation is a real systems move, but the headline ~23-token win is under-controlled and the “late-blooming” story sits awkwardly with keeping downward tokens. read the letter →

arxiv 2607.28341 v1 pith:OQJ7MIPY submitted 2026-07-30 cs.CV

Jie Ma , Zhike Qiu , Jie Gao , Jiayi Ji , Qian Chen , Xiaoshuai Sun , Rongrong Ji This is my paper · ORCID
classification cs.CV
keywords TokenPruningMultimodalLargeLanguageModelsSparsificationAttentionflowTraining-freeinferenceVisualtokensCross-layerdynamics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multimodal large language models pay a heavy cost for long visual token sequences. Most training-free pruners decide once, from a single layer’s attention scores, and never reverse that decision. This paper argues that token importance is not fixed: shallow layers spread attention widely, while deeper layers concentrate on task-relevant regions, so early snapshots discard cues that only become critical later. Trend-aware Pruning reframes the problem as trajectory modeling. It keeps a short history of attention scores, classifies each token’s flow as upward, downward, or fluctuating, and unions those trend sets with ordinary Top-k retention so “late-blooming” tokens can be reactivated. On standard LLaVA and Qwen vision-language benchmarks the method keeps competitive accuracy while cutting visual tokens by more than three-quarters, leaving roughly two dozen visual tokens in the final layer.

What carries the argument

Trend-aware Pruning: a Layer-wise Token Collector builds a sliding window of attention scores; Adaptive Flow Identification scores upward, downward, and fluctuating tendencies via discrete differences and adaptive z-score thresholds; Flow Activation unions those trend sets with static Top-k so previously pruned tokens can re-enter the retained set.

What would settle it

Under the paper’s default window and threshold, on a detail-heavy benchmark such as OCRBench or GQA, measure whether tokens reactivated by the trend rules actually raise answer accuracy relative to pure Top-k at the same final-layer budget; if the union set does not improve or harms accuracy, the trajectory proxy fails.

Watch

Extended reading notes

Core claim

Visual token pruning in MLLMs should be treated as a reversible, cross-layer trajectory problem rather than a static snapshot: capturing the momentum of attention flow and selectively reactivating tokens with upward, fluctuating, or downward trends prevents premature loss of cues needed for deep reasoning, yielding a better efficiency–performance trade-off than irreversible Top-k methods.

Load-bearing premise

A short window of layer-to-layer attention differences plus one adaptive threshold is a good enough signal of future task importance to justify reactivating tokens, including ones whose attention is falling.

Editorial extensions

If this is right

  • Aggressive visual sparsification (down to ~23 final-layer tokens) remains viable without fine-tuning if pruning is made reversible across layers.
  • Static single-layer attention rankings are systematically incomplete for hierarchical MLLM decoders.
  • The same collector-plus-flow pattern can be dropped into other LLaVA- and Qwen-style stacks as a plug-in inference step.
  • Detail-sensitive tasks (OCR, fine attributes) benefit most from recovering upward- and even downward-trending tokens that early Top-k would drop.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If attention momentum is a reliable proxy, similar trajectory recovery could reduce irreversible pruning errors in pure-language long-context transformers.
  • Region- or concept-level trends, rather than per-token scores, are a natural next granularity once token-level reactivation is established.
  • Adaptive window length conditioned on sequence length or task type could tighten the method’s main free parameters without training.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Trend-aware Pruning, a training-free visual token pruning method for MLLMs that treats pruning as cross-layer trajectory modeling rather than irreversible single-layer Top-k filtering. A Layer-wise Token Collector maintains a sliding window of attention scores (Eqs. 1–3); Adaptive Flow Identification scores upward, downward, and fluctuating tendencies via sign consistency and volatility (Eqs. 4–7); and Flow Activation forms the retained set as the union of static Top-k with these trend sets (Eq. 8), allowing selective reactivation of tokens undervalued at the current layer. Experiments on LLaVA-v1.5-7B/13B, LLaVA-Next, LLaVA-OV-0.5B, and Qwen2.5-VL across multiple sparsity levels report competitive average retention (e.g., 98.89% at 50% prune, 96.03% at 77.8% prune with ~23 final-layer visual tokens) and often the lowest FLOPs among compared training-free baselines, with ablations on trend components, window size W, and threshold λ.

Significance. Visual token redundancy is a genuine bottleneck for high-resolution and multi-image MLLM inference, and the observation that token importance evolves across decoder layers is well motivated by the paper’s attention visualizations and by prior hierarchical analyses. Reformulating pruning as reversible, trend-aware selection is a clear conceptual contribution relative to static or layer-isolated heuristics (FastV, VisionZip, etc.). If the trajectory signal is genuinely predictive of task-relevant semantics—not merely a way to keep a few extra tokens—the method would offer a practical, plug-and-play efficiency tool with broad applicability. Strengths include multi-architecture evaluation, explicit last-token and FLOPs reporting, and component ablations (Table 4, Fig. 5). The main significance hinges on whether the reported gains are attributable to momentum modeling rather than cardinality or schedule differences.

major comments (4)
  1. [§4.4, Table 4; §3.4, Eq. (8)] Table 4 reports cumulative gains from adding Upward, then Fluctuating, then Downward tokens on top of Top-k, but does not control for the number of retained tokens. Each added set increases |S_final| (Eq. 8), so improved MMB/GQA/OCRBench/InfoVQA scores are consistent with “keep more tokens by any rule.” A load-bearing control is missing: (i) matched-budget pure Top-k that retains the same |S_final| as the full union at each layer, and (ii) ablations that drop S_down (and/or the full trend set) while holding final cardinality fixed. Without these, the claim that short-window attention momentum (Eqs. 2–6) proxies future semantic importance—and that trajectory modeling, not extra tokens, drives the aggressive-sparsity trade-off in Table 1—is under-supported.
  2. [Abstract; §1; §3.3–3.4; Table 4] The abstract and introduction center the method on “late-blooming” tokens with rising semantic importance, yet Flow Activation treats Downward-trending tokens as a first-class reactivation set (Eqs. 5, 8), and Table 4 attributes the largest OCRBench jump (25.70→31.00) to adding Downward tokens. The paper should reconcile this narrative tension: either justify why decreasing attention trajectories still warrant reactivation (e.g., complementary cues, anti-distraction), or show that S_down is necessary under matched budgets. As written, the headline story and the retention rule are misaligned.
  3. [Table 1, Retain 128 Tokens row; §4.2] At 77.8% prune (Table 1), SparseVLM reports slightly higher average retention (96.29% vs. 96.03%) with 36 last-layer tokens and 37.41% FLOPs, while the paper’s strongest differentiator is ~23 last-layer tokens and 32.20% FLOPs. Superiority is therefore driven primarily by extreme final-layer sparsity rather than accuracy. The manuscript should clarify the per-layer Top-k / retention schedule that produces ~23 tokens, state whether baselines were run under identical progressive schedules, and report accuracy–FLOPs (or accuracy–last-token) Pareto curves so the efficiency claim is not conflated with a more aggressive hand-chosen schedule.
  4. [§4.3, Table 3] The Qwen2.5-VL experiment (Table 3) compares only against a consistent Top-k baseline, not against SparseVLM, PDrop, FastV, or VisionZip. Given that the LLaVA results already show SparseVLM competitive on average accuracy, a single Top-k comparison is insufficient to support the claim of generalization to “advanced architectures.” At least one strong training-free baseline under the same retention budget is needed.
minor comments (5)
  1. [Figure 1] Figure 1(a) uses “+” annotations for recovered tokens but does not define how many candidates were considered or the baseline Top-k count at each layer; a short caption clarification would help.
  2. [§3.2, Eq. (1)] Notation: a_l is described as “aggregated attention score” for N_v_l tokens, but the aggregation over heads/query tokens (e.g., text-to-vision vs. full self-attention) is not specified. State the exact attention source used for the collector.
  3. [§4.4, Figure 5] Default W=5 and λ=0.5 are chosen from Fig. 5 on five benchmarks that overlap the main evaluation suite; briefly note whether any hold-out was used or treat them explicitly as validated hyperparameters.
  4. [§3.2–3.3; Figure 1] Typos/clarity: “vision representation of each layer to be the same” (§3.2); “Noted that, this adaptive strategy provide” (§3.3); “T oken” spacing in Fig. 1 legend. Minor copy-edit pass recommended.
  5. [§3.5; Table 1] FLOPs formula (Eqs. 9–10) is standard; reporting wall-clock latency or peak memory on A100 alongside FLOPs would strengthen the efficiency claims for practitioners.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical systems method evaluated on external benchmarks; no derivation reduces to its inputs by construction.

full rationale

Trend-aware Pruning is a training-free engineering method, not a first-principles derivation. The pipeline (layer-wise attention cache W_l, discrete flow δ_k, trend scores T_up/T_down/T_fluct, z-threshold τ, and union retention S_final = S_rank ∪ S_up ∪ S_down ∪ S_fluct) is a designed heuristic whose value is measured by external MLLM benchmarks (MME, GQA, POPE, SQA, MMB, etc.) and FLOPs against independent baselines (FastV, PDrop, SparseVLM, VisionZip, FasterVLM). Hyperparameters W and λ are chosen via a standard ablation (Fig. 5) and then held fixed; the paper does not fit a constant to a target quantity and then re-label that fit as a prediction. Citations to prior pruning work and to layer-wise MLLM analyses are ordinary related-work support, not load-bearing uniqueness theorems by the same authors that force the result. There is no self-definitional loop, no fitted-input-called-prediction, and no renaming of a known closed-form result. Weaknesses (missing matched-cardinality Top-k controls, tension between the “late-blooming” narrative and keeping downward tokens) are experimental-design / correctness concerns, not circularity. Score 0 is the honest finding.

Assumptions & free parameters 3 free parameters · 4 assumptions · 4 invented entities

The central empirical claim rests on a small set of modeling choices and free knobs rather than on deep new theory. Load-bearing background is that layer-wise attention is a usable importance signal and that short-horizon score differences encode recoverable ‘tendency.’ Free parameters W and λ directly gate how many non-Top-k tokens re-enter. Invented machinery (collector, flow labels, flow activation union) is definitional scaffolding for the algorithm, not independently measured natural entities. No formal proof obligations beyond standard transformer FLOPs accounting.

free parameters (3)
  • window size W = 5 (default)
    Length of the sliding historical attention cache used to compute token flow; chosen by ablation (peak reported at W=5) and fixes how much history defines a ‘trend.’
  • trend threshold sensitivity λ = 0.5 (default)
    Multiplier on the std of trend scores for adaptive selection τ=μ+λσ; controls pruning aggressiveness and was swept on eval metrics (default 0.5).
  • per-setting Top-k / retention schedule = e.g. 288, 192, 128 of 576 on LLaVA-v1.5-7B
    Coarse static keep-counts (e.g., retain 288/192/128 of 576, and analogous ratios on other models) are experiment knobs that define the sparsity operating points the claims reference.
assumptions (4)
  • domain assumption Aggregated attention scores over visual tokens are a sufficient instantaneous proxy for token importance at each decoder layer.
    Inherited from FastV-style training-free pruning; used throughout §3 as the base signal a_l before trajectory features are built.
  • domain assumption MLLM layers implement a hierarchical dispersion-to-concentration process so that token importance can meaningfully rise or fall across depth rather than stay fixed.
    Motivated via citations and Fig. 2 visualizations in §3.1; without this, cross-layer ‘momentum’ has no semantic reading.
  • ad hoc to paper Short-horizon sign consistency and volatility of attention deltas identify tokens worth reactivating (including downward-trending tokens) for downstream task performance.
    Core design hypothesis of §3.3–3.4 (Eqs. 4–8); supported only by ablations/benchmarks in this paper, not by an external theorem.
  • standard math Standard transformer FLOPs model C = 4Nd² + 2N²d + 2Ndm adequately represents inference cost for comparing methods.
    Used in §3.5 complexity analysis; routine but ignores memory movement and implementation constants.
invented entities (4)
  • Layer-wise Token Collector (sliding attention window W_l)
    purpose: Store recent per-layer attention vectors so trends can be computed instead of single-layer scores.
    Algorithmic memory structure introduced in §3.2; no existence claim beyond the implementation.
  • Token flow F_l / tendency scores (T_up, T_down, T_fluct)
    purpose: Quantify upward, downward, and volatile attention trajectories for adaptive selection.
    Defined in Eqs. 2–6; labels are paper-constructed features, not measured external quantities.
  • Flow Activation retention set S_final = S_rank ∪ S_up ∪ S_down ∪ S_fluct
    purpose: Merge static Top-k with trend-selected tokens so pruning is partially reversible across layers.
    Eq. 8 mechanism that operationalizes ‘reactivation’; success is only evidenced by this paper’s tables.
  • “Late-blooming” tokens
    purpose: Narrative category for tokens undervalued early but rising in importance later.
    Rhetorical/conceptual label in abstract and intro; not independently operationalized beyond upward-trend scores.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Capturing Token Tendencies for Training-Free Token Pruning in Multimodal Large Language Models." pith.science (2026). https://pith.science/paper/OQJ7MIPY

@misc{pith2026260728341,
  author       = {Pith},
  title        = {Pith review of: Capturing Token Tendencies for Training-Free Token Pruning in Multimodal Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OQJ7MIPY}},
  note         = {Machine review of arXiv:2607.28341}
}
read the original abstract

While visual token pruning is essential for efficient Multimodal Large Language Models (MLLMs), existing training-free methods suffer from a critical limitation: they rely on static, instantaneous heuristics to perform irreversible filtering. This approach ignores the hierarchical nature of MLLMs, where token importance often evolves dynamically rather than remaining fixed across layers. Consequently, tokens essential for deep-layer reasoning are often prematurely discarded by shallow-layer estimates. To address this, we propose Trend-aware Pruning, a novel framework that elevates pruning from a local snapshot decision to a temporal trajectory modeling problem. Instead of relying on isolated scores, our method captures the momentum of attention flow. This enables a dynamic rectification mechanism that selectively reactivates "late-blooming" tokens, those initially undervalued but exhibiting rising semantic importance, thereby preventing the loss of critical visual cues. Extensive experiments demonstrate that our approach achieves a superior efficiency-performance trade-off across diverse multimodal tasks. Notably, it reduces visual tokens by over 77.8%, retaining only approximately 23 tokens in the final layer while maintaining competitive performance, offering a robust and reversible solution for high-efficiency multimodal inference.

Figures

Figures reproduced from arXiv: 2607.28341 by the authors.

Figure 1
Figure 1. Token retention dynamics and performance com [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Visualization of layer-wise attention dynamics in LLaVA-v1.5-7B. We compare the attention heatmaps of visual tokens [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An overview of Trend-aware Pruning. The framework comprises three stages: (1) The Layer-wise Token Collector [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Visualization of trend-aware tokens via adaptive flow identification with instruction: [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Ablation study on hyperparameter sensitivity. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 11 linked inside Pith

  1. [1]

    Xiang An, Yin Xie, Kaicheng Yang, Wenkang Zhang, Xiuwei Zhao, Zheng Cheng, Yirui Wang, Songcen Xu, Changrui Chen, Chunsheng Wu, Huajie Tan, Chunyuan Li, Jing Yang, Jie Yu, Xiyao Wang, Bin Qin, Yumeng Wang, Zizhen Yan, Ziyong Feng, Ziwei Liu, Bo Li, and Jiankang Deng. 2025. LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training.CoRR...

    arXiv doi:10.48550/arxiv.2509.23661 2025

  2. [2]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Ming-Hsuan Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. 2025. Qwen2.5-VL Technical ...

    arXiv doi:10.48550/arxiv.2502.13923 2025

  3. [3]

    Bigham, Chandrika Jayant, Hanjie Ji, Greg Little, Andrew Miller, Robert C

    Jeffrey P. Bigham, Chandrika Jayant, Hanjie Ji, Greg Little, Andrew Miller, Robert C. Miller, Robin Miller, Aubrey Tatarowicz, Brandyn White, Samuel White, and Tom Yeh. 2010. VizWiz: nearly real-time answers to visual questions. In Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology, New York, NY, USA, October 3-6, 2010,...

    arXiv 2010

  4. [4]

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. 2024. An Image is Worth 1/2 Tokens After Layer 2: Plug- and-Play Inference Acceleration for Large Vision-Language Models. InEuropean Conference on Computer Vision

    2024

  5. [5]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yimin Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, Conghui He, Botian Shi, Xingcheng Zhang, Han Lv, Yi Wang, Wenqi Shao, Pei Chu, Zhongying Tu, Tong He, Zhiyong Wu, Huipeng Deng, Jia...

    arXiv doi:10.48550/arxiv.2412.05271 2024

  6. [6]

    Paul Cisek and John F. Kalaska. 2010. Neural mechanisms for interacting with a world full of action choices.Annual review of neuroscience33 (2010), 269–98. https://api.semanticscholar.org/CorpusID:3641138

    2010

  7. [7]

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, and et al. 2025. DeepSeek-V3 Technical Report. arXiv:2412.19437 [cs.CL]

    arXiv 2025

  8. [8]

    Fiebelkorn and Sabine Kastner

    Ian C. Fiebelkorn and Sabine Kastner. 2019. A Rhythmic Theory of Attention. Trends in Cognitive Sciences23, 2 (2019), 87–101. doi:10.1016/j.tics.2018.11.009

    doi:10.1016/j.tics.2018.11.009 2019

Show all 42 references
  1. [9]

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, and Rongrong Ji. 2023. MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models.CoRRabs/2306.13394 (2023). arXiv:2306...

    2023 doi arXiv

  2. [10]

    Mingyu Fu, Wei Suo, Ji Ma, Lin Yuanbo Wu, Peng Wang, and Yanning Zhang

  3. [11]

    Junyu Gao, Xuan Yao, Yong Rui, and Changsheng Xu. 2025. Building Embodied EvoAgent: A Brain-inspired Paradigm for Bridging Multimodal Large Models and World Models. InProceedings of the 33rd ACM International Conference on Multimedia(Dublin, Ireland)(MM ’25). Association for C...

    2025

  4. [12]

    Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxuan Zhang, Juanzi Li, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang. 2024. CogAgent: A Visual Language Model for GUI Agents. arXiv:2312.08914 [cs.CV] https://arxiv.org/abs/2312.08914

    2024 arXiv

  5. [13]

    Yihong Huang, Fei Ma, Yihua Shao, Jingcai Guo, Zitong Yu, Laizhong Cui, and Qi Tian. 2026. Nüwa: Mending the Spatial Integrity Torn by VLM Token Pruning. arXiv:2602.02951 [cs.CV] https://arxiv.org/abs/2602.02951

    2026

  6. [14]

    Hudson and Christopher D

    Drew A. Hudson and Christopher D. Manning. 2019. GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. Computer Vision Foundation...

    2019

  7. [15]

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Min Joon Seo, Hannaneh Ha- jishirzi, and Ali Farhadi. 2016. A Diagram is Worth a Dozen Images. InComputer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, Octo- ber 11-14, 2016, Proceedings, Part IV (Lectu...

    2016 doi

  8. [16]

    Youngeun Kim, Youjia Zhang, Huiling Liu, Aecheon Jung, Sunwoo Lee, and Sungeun Hong. 2026. ZOO-Prune: Training-Free Token Pruning via Zeroth- Order Gradient Estimation in Vision-Language Models. arXiv:2509.24837 [cs.CV] https://arxiv.org/abs/2509.24837

    2026

  9. [17]

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Evaluating Object Hallucination in Large Vision-Language Models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2...

    2023 doi

  10. [18]

    Tao Lin, Yilei Zhong, Yuxin Du, Jingjing Zhang, Jiting Liu, Yinxinyu Chen, Encheng Gu, Ziyan Liu, Hongyi Cai, Yanwen Zou, Lixing Zou, Zhaoye Zhou, Gen Li, and Bo Zhao. 2025. Evo-1: Lightweight Vision-Language-Action Model with Preserved Semantic Alignment.CoRRabs/2511.04555 (2...

    2025 doi

  11. [19]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023. Improved Baselines with Visual Instruction Tuning.2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)(2023), 26286–26296

    2023

  12. [20]

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024. LLaVA-NeXT: Improved reasoning, OCR, and world knowl- edge. https://llava-vl.github.io/blog/2024-01-30-llava-next/

    2024

  13. [21]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruc- tion Tuning.ArXivabs/2304.08485 (2023)

    2023 arXiv

  14. [22]

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin

  15. [23]

    Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. 2024. OCRBench: on the hidden mystery of OCR in large multimodal models.Sci. China Inf. Sci.67, 12 (2024). doi:10.1007/S11432-024-4235-6

    2024 doi

  16. [24]

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering. InAdvances in Neural Information Processing Systems 35:...

    2022

  17. [25]

    Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and C. V. Jawahar. 2022. InfographicVQA. InIEEE/CVF Winter Conference on Applications of Computer Vision, W ACV 2022, Waikoloa, HI, USA, January 3-8,

    2022

  18. [26]

    Qucheng Peng, Chen Bai, Guoxiang Zhang, Bo Xu, Xiaotong Liu, Xiaoyin Zheng, Chen Chen, and Cheng Lu. 2025. NavigScene: Bridging Local Perception and Global Navigation for Beyond-Visual-Range Autonomous Driving. InProceedings of the 33rd ACM International Conference on Multimed...

    2025

  19. [27]

    Zhaofan Qiu, Zijian Gong, Yingwei Pan, Ting Yao, and Tao Mei. 2025. Talk, Imagine, Evolve: A Unified Multimodal Agent for Seamless Visual Generation and Editing. InProceedings of the 33rd ACM International Conference on Multimedia (Dublin, Ireland)(MM ’25). Association for Com...

    2025

  20. [28]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. InProceedings ...

    2021

  21. [29]

    Cheng Shi, Yizhou Yu, and Sibei Yang. 2025. Vision Function Layer in Multimodal LLMs.CoRRabs/2509.24791 (2025). arXiv:2509.24791 doi:10.48550/ARXIV.2509. 24791

    2025 doi

  22. [30]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lam- ple. 2023. LLaMA: Open and Efficient Foundation ...

    2023 doi arXiv

  23. [31]

    Hanshi Wang, Yuhao Xu, Zekun Xu, Jin Gao, Yufan Liu, Weiming Hu, Ke Wang, and Zhipeng Zhang. 2025. AutoPrune: Each Complexity Deserves a Pruning Policy.CoRRabs/2509.23931 (2025). arXiv:2509.23931 doi:10.48550/ARXIV.2509. 23931

    2025 doi

  24. [32]

    Long Xing, Qidong Huang, Xiao wen Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, and Dahua Lin. 2025. Pyra- midDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction.Computer Vision and Pattern Recogn...

    2025 arXiv

  25. [33]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, and et al. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL]

    2025 arXiv

  26. [34]

    Yixiang Yang, Chuanrong Han, Runhao Mao, Hanshi Wang, Zhiwen Chen, Yantai Yang, Qianli Ma, Xuesong Chen, Shaoshuai Shi, and Zhipeng Zhang. 2025. Survey of General End-to-End Autonomous Driving: A Unified Perspective.Authorea Preprints(2025)

    2025

  27. [35]

    Zhuoran Yu and Yong Jae Lee. 2025. How multimodal llms solve image tasks: A lens on visual grounding, task reasoning, and answer decoding. (2025)

    2025

  28. [36]

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sig- moid Loss for Language Image Pre-Training. InIEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023. IEEE, 11941–11952. doi:10.1109/ICCV51070.2023.01100

    2023

  29. [37]

    Qizhe Zhang, Aosong Cheng, Ming Lu, Zhiyong Zhuo, Minqi Wang, Jiajun Cao, Shaobo Guo, Qi She, and Shanghang Zhang. 2024. [CLS] Attention is All You Need for Training-Free Visual Token Pruning: Make VLM Inference Faster.CoRR abs/2412.01818 (2024). arXiv:2412.01818 doi:10.48550/...

    2024 doi arXiv

  30. [38]

    Yuan Zhang, Chun-Kai Fan, Junpeng Ma, Wenzhao Zheng, Tao Huang, Kuan Cheng, Denis Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, et al

  31. [39]

    Highlighted Tokens

    Xin Zou, Di Lu, Yizhou Wang, Yibo Yan, Yuanhuiyi Lyu, Xu Zheng, Linfeng Zhang, and Xuming Hu. 2025. Don’t Just Chase "Highlighted Tokens" in MLLMs: Revisiting Visual Holistic Context Retention.CoRRabs/2510.02912 (2025). arXiv:2510.02912 doi:10.48550/ARXIV.2510.02912

    2025 doi

  32. [42]

    InInternational Conference on Machine Learning

    SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference. InInternational Conference on Machine Learning

  33. [2022]

    doi:10.1109/WACV51458.2022.00264

    IEEE, 2582–2591. doi:10.1109/WACV51458.2022.00264

    2022

  34. [2024]

    InComputer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part VI (Lecture Notes in Computer Science, Vol

    MMBench: Is Your Multi-modal Model an All-Around Player?. InComputer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part VI (Lecture Notes in Computer Science, Vol. 15064), Ales Leonardis, Elisa Ricci, Stefan Roth, Olga ...

    2024 doi

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.