Pith. sign in

REVIEW 4 major objections 7 minor 52 references

Short-LVLM: Compressing and Accelerating Large Vision-Language Models by Pruning Redundant Layers

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Layer pruning can compress vision-language models without retraining, provided the redundant layers are chosen by important tokens and the gaps they leave are bridged by a low-rank weight projection.

desk verdict A useful empirical extension of LLM layer pruning to LVLMs, with a genuine ablation result, but the SCP mechanism as written does not follow from its own equations. read the letter →

arxiv 2507.23362 v1 pith:RYLYEIRF submitted 2025-07-31 cs.CV

Ji Ma , Wei Suo , Peng Wang , Yanning Zhang This is my paper
classification cs.CV
keywords largevision-languagemodelslayerpruningmodelcompressiontraining-freetokenimportancesubspacecompensationsingularvaluedecompositionLLaVA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that large vision-language models (LVLMs) contain redundant transformer layers that can be removed without any fine-tuning, provided two problems are solved: finding the right layers, and repairing the damage. It argues that locating layers by all tokens, or by text or vision tokens alone, is biased and noisy; instead, one should score each token's importance through intra-modal and cross-modal attention and keep only the top ones. It further argues that pruning fails in LVLMs because adjacent layers have larger feature gaps than in pure language models, and that these gaps can be bridged by a low-rank subspace projection computed from calibration data. If both designs work, the method is training-free, works across architectures and scales, and can be combined with other accelerations.

What carries the argument

The load-bearing identity is the weight projection in Eq. (10)–(11): $W_{\text{proj}} = (I + V_k V_k^\top) W_{\text{org}}$, where $V_k$ holds the top-$k$ right singular vectors of the feature difference matrix $H_{p,r} = X_{\ell_p} - X_{\ell_r}$ between a pruned layer and its nearest retained layer. SVD is used because the authors assume, following prior work on residual architectures, that adjacent layers' feature spaces are aligned so that the gap is captured by a low-rank subspace; the projection term $x_{\ell_r}^{\text{org}} V_k V_k^\top W_{\text{org}}$ then reconstructs the pruned layer's features in the forward pass. It is supported by TIS, which computes per-token self-attention and cross-attention scores, averages them, and keeps the top-$p$ tokens (with $p=10\%$ in the default setting) to make the cosine-similarity localization less noisy.

What would settle it

Take a model where this assumption fails—for instance, a residual vision-language backbone with an abrupt architectural transition or a domain shift between calibration and evaluation—run Short-LVLM at 30–40% pruning, and measure whether SCP still restores performance. A cleaner control: replace $V_k$ in Eq. (10) with random orthogonal vectors of the same rank; if performance recovery is unchanged, the specific subspace content is not what carries the compensation.

Watch

Extended reading notes

Core claim

The central claim is that Short-LVLM (SVL) pairs Token Importance Scores (TIS) with Subspace-Compensated Pruning (SCP) to compress LVLMs without training, preserving most capacity at substantial pruning ratios. TIS identifies redundant layers by computing cosine similarity only over vision-language tokens judged important by their self- and cross-attention scores, discarding the noisy majority. SCP then computes the feature difference matrix $H_{p,r}$ between a pruned layer and its nearest retained layer, extracts its top-$k$ right singular vectors $V_k$ via singular value decomposition, and projects the retained layer's weights as $W_{\text{proj}} = (I + V_k V_k^\top) W_{\text{org}}$, so that the forward pass regains a "subspace feature" term that reconstructs the removed layer's output. On LLaVA-1.5-13B the combination preserves more than 96% of the original model's average benchmark capacity while pruning 40% of layers and speeding up inference by 1.34×; similar trade-offs are reported on LLaVA-1.5-7B, Qwen-VL-Chat-7B and mPLUG-Owl2-7B.

Load-bearing premise

SCP assumes the top singular vectors of the feature difference matrix computed from calibration data correctly span the subspace of the removed layer's features during the forward pass.

Editorial extensions

If this is right

  • If the central claim is right, LVLMs at 7B–13B scale can shed 20–40% of their transformer layers with single-digit percentage point losses and measurable speedups, entirely without fine-tuning.
  • The localization recipe—score tokens by attention, keep a small subset, then use cosine similarity—would carry over to any LVLM built on a residual LLM backbone, including those with different vision encoders.
  • The subspace-compensation step should combine with quantization and token pruning, so the speedups the paper reports could be compounded rather than replaced.
  • The ablations show both TIS and SCP contribute independently and additively, so either component could be reused separately in other pruning pipelines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the projection in Eq. (11) omits the residual connection, so SCP actually modifies only the sub-layer weights. One consequence the paper does not spell out is that the method's success may depend on whether the feature gap originates in the attention or the MLP block; testing the projection on each block separately could reveal where the low-rank compensation is really acting.
  • Inference: since SCP's correction is calibrated on a fixed calibration set (1k samples from LLaVA-665k), the method is vulnerable to distribution shift: if a deployment domain has features very different from the calibration data, the extracted subspace could point the wrong way. A natural stress test is to calibrate on one benchmark and evaluate on a disjoint domain.
  • Inference: the reported 1.34× speedup is modest because pruning is applied only to the LLM portion of the LVLM. Combining layer pruning with input-token pruning, which the paper says is compatible, could yield much larger end-to-end acceleration, since both attack different bottlenecks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes Short-LVLM (SVL), a training-free layer-pruning framework for large vision-language models. It first reports exploratory experiments suggesting that LLM layer-pruning heuristics transfer poorly to LVLMs because of redundant vision-language tokens and inter-layer feature gaps. It then introduces Token Importance Scores (TIS) to select important tokens for cosine-similarity-based layer localization, and Subspace-Compensated Pruning (SCP), which uses SVD on feature differences between retained and pruned layers to project the retained layer's weights and restore performance. Experiments on six benchmarks and four LVLMs claim 1.23-1.34x speedup with small accuracy loss, and ablations attribute gains to both TIS and SCP.

Significance. If validated, the work would provide a practical, training-free, model-agnostic compression baseline for LVLMs, with public code and potential compatibility with quantization and token pruning. The preliminary analyses and the two-component ablation design are useful and clearly presented. However, the central SCP mechanism as derived does not mathematically implement the claimed feature reconstruction, and the experimental evidence lacks variance reporting and uses calibration data drawn from the same instruction-tuning mixture used to train the evaluated LVLMs and some of the benchmarks. These issues need to be resolved before the central claims can be accepted.

major comments (4)
  1. [Sec. 4.2, Eqs. (7)-(11)] The derivation of SCP does not support the stated mechanism. The optimal rank-k approximation of H_{p,r} is H_{p,r} V_k V_k^T = U_k Sigma_k V_k^T, whose i-th row is (x_{l_p,i} - x_{l_r,i}) V_k V_k^T. Equation (11), however, adds x_org V_k V_k^T W_org to the retained layer's output. No assumption stated in Sec. 4.2 implies that x_org V_k V_k^T W_org equals or reconstructs this difference (or the pruned layer's features); the two expressions coincide only in degenerate cases. The paragraph after Eq. (11) therefore claims an SVD-based reconstruction that the equations do not provide. Since Table 2 shows SCP is the main component restoring performance, this is a load-bearing correctness issue; please either derive the correct forward-pass modification or state explicitly that SCP is an empirical heuristic whose operation is not the SVD reconstruction claimed.
  2. [Sec. 4.2, Eq. (11)] The forward pass is written as x_proj = x_org (I + V_k V_k^T) W_org, which treats a transformer layer as a single linear map without a residual connection or nonlinearities. The models in Table 1 use standard transformer blocks whose output is x + f(x; theta), with f containing attention and MLP. As written, Eq. (11) does not describe the actual forward process of the pruned model, so the claim that the subspace feature term bridges the feature gap in the real residual stream is not established. Please specify the precise modification to the residual stream and derive the resulting output.
  3. [Sec. 5.1.2 and Table 1] Calibration data are 1k random samples from LLaVA-665k, which is the instruction-tuning mixture used to train LLaVA-1.5 [23] and includes data from benchmarks such as A-OKVQA and ScienceQA-Img evaluated in Table 1. The paper does not report whether these 1k samples overlap with the evaluation examples or come from the same distribution. Because both TIS layer selection and the SVD subspace in SCP are fitted to these calibration features, the reported gains may partly reflect adaptation to the test distribution rather than a general pruning principle. Please rerun with calibration data explicitly disjoint from the evaluation benchmarks and report whether the performance differences persist.
  4. [Tables 1-3] All results are single runs with no error bars or significance measures. Several reported differences are small (e.g., LLaVA-1.5-7B average 69.9 vs 70.1 at 10% pruning; MMBench 65.1 vs 64.3 at 10% with SCP in Table 2), which is within the typical seed-to-seed variability of these benchmarks. The claim of a superior trade-off needs support from multiple seeds or variance estimates, particularly for the ablation comparisons in Table 3 where differences of 1-2 points are used to rank methods.
minor comments (7)
  1. [Eq. (8)] Standard SVD should be H = U Sigma V^T; the missing transpose makes the subsequent projection V_k V_k^T notationally inconsistent.
  2. [Sec. 4.1] Calling the token-selection ratio 'Top-p sampling [14]' is misleading; nucleus sampling is a distribution-based decoding method, whereas here a fixed proportion of tokens is discarded. Please rename the method to avoid confusion.
  3. [Sec. 5.2] 'scaling lows' should be 'scaling laws'.
  4. [Table 3] 'Spatial Samping' should be 'Spatial Sampling'.
  5. [References] References [10] and [11] are the same paper and should be merged; please also check for other duplicated entries.
  6. [Eq. (2)] The summation uses index i while the argument uses ell; align the indices for readability.
  7. [Abstract and Sec. 1] The phrase 'empirically prove' is stronger than the evidence supports; 'demonstrate' or 'show' is more appropriate given the limited number of models and the absence of significance testing.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: SCP's reconstruction gap is a derivation error, not an input-output circle; self-citations are non-load-bearing.

full rationale

After walking the derivation chain, I find no circular step. The two proposed components are validated against external benchmarks, not against their own fitting targets. TIS (Sec. 4.1) uses attention-derived importance scores to select a Top-p subset of tokens and then applies the existing cosine-similarity redundancy criterion to those tokens; the resulting pruning decisions are scored on AOKVQA, ScienceQA-Img, MME, POPE, MMBench, and SEED-Bench-Img (Tables 1-3), so the localization output is not defined by the performance it is said to predict. SCP (Sec. 4.2) fits a rank-64 SVD subspace to the calibration feature-difference matrix H_{p,r} (Eq. 7-8) and modifies weights with I+V_kV_k^T (Eq. 10); its success is measured by held-out benchmark accuracy and by feature-gap visualizations on 200 fresh samples (Fig. 6), not by reconstruction error on the calibrated H itself. The low-rank-alignment premise is attributed to [1,19], which are external papers with no author overlap, so no load-bearing argument reduces to a self-citation. The paper's two self-citations ([29] in the related-work list and [40] in the sparse-computation list) are contextual and not load-bearing. One genuine internal issue exists in Sec. 4.2: the text claims "the features of pruned layer ℓ_p can be effectively reconstructed using the subspace feature term," but Eq. 11's added term is x_org V_k V_k^T W_org, whereas the SVD reconstruction of H_{p,r} would be (x_p - x_r) V_k V_k^T (row-wise); the paper supplies no equation implying these coincide. This is a derivation/correctness gap that a reader may want addressed, but it is not a circular equivalence of output to input, so it does not change the circularity score.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central method depends on a small number of hand-set hyperparameters (p, k, calibration set size) and on empirical properties borrowed from prior work (attention-based importance, residual alignment). No new entities are introduced. The main burden is the calibration-data dependence of SCP.

free parameters (4)
  • p (Top-p token sampling ratio) = 0.1 (10%)
    Chosen by hand; determines how many tokens are discarded as non-essential in TIS. No sensitivity analysis is provided.
  • k (subspace rank) = 64
    Chosen by hand for SVD low-rank approximation of feature differences. No sensitivity analysis is provided.
  • Calibration data size = 1000 samples
    Randomly sampled from LLaVA-665k; no analysis of how this size affects results.
  • Pruning search range = last 16 of 32 layers
    The method restricts pruning to the deeper half of the model; rationale from prior work but not a model hyperparameter.
assumptions (4)
  • domain assumption Attention scores are strong indicators of token importance.
    Invoked in Sec. 4.1 to justify TIS; based on prior token pruning works [4,48].
  • domain assumption Top singular vectors of Jacobian matrices across layers are aligned in residual architectures.
    Invoked in Sec. 4.2, cited from [1,19]; supports the claim that feature gaps can be captured by low-rank subspaces.
  • domain assumption Layers closer in position exhibit better feature alignment.
    Used in Sec. 4.2 to choose the candidate retained layer l_r; not proven in this paper.
  • domain assumption The latter half of layers is more redundant than the earlier half.
    Used in Sec. 5.1.2 to restrict pruning to deeper layers; cited from previous LLM pruning works [12,22].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Short-LVLM: Compressing and Accelerating Large Vision-Language Models by Pruning Redundant Layers." pith.science (2026). https://pith.science/paper/RYLYEIRF

@misc{pith2026250723362,
  author       = {Pith},
  title        = {Pith review of: Short-LVLM: Compressing and Accelerating Large Vision-Language Models by Pruning Redundant Layers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RYLYEIRF}},
  note         = {Machine review of arXiv:2507.23362}
}
read the original abstract

Although large vision-language models (LVLMs) have demonstrated impressive capabilities in multi-modal understanding and reasoning, their practical applications are still limited by massive model parameters and high computational costs. Recent efforts from natural language processing (NLP) have shown the effectiveness of layer pruning, offering a plausible training-free compression solution. However, due to the modality divergence between vision and language, it is unclear whether these NLP techniques are still effective in LVLMs. In this paper, we empirically prove that directly applying these layer pruning methods to LVLMs is ineffective. Through extensive experiments, we find that non-essential vision-language (VL) tokens and inter-layer feature gaps pose critical challenges to pruning layers in LVLMs. Based on these insights, we propose a novel framework Short-LVLM (SVL) that can utilize important VL tokens and mitigate the layer-wise feature gaps. Notably, Short-LVLM not only achieves a superior trade-off between performance and efficiency but also exhibits several potential advantages, i.e., training-free, model-agnostic, and highly compatible. The code for this work is publicly available at https://github.com/ASGO-MM/Short-LVLM.

Figures

Figures reproduced from arXiv: 2507.23362 by the authors.

Figure 1
Figure 1. Comparison of different model pruning methods. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Preliminary experiments using different tokens [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Preliminary experiments exploring the effective [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Visualization of feature similarity across layers. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Overview of our Short-LVLM (SVL). Our Short-LVLM consists of two key designs: (a) Localizing redundant layers with [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Visualization results of Subspace-Compensated [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Qualitative results of Short-LVLM. The retained [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 24 canonical work pages

  1. [23]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 26296–26306

    2024

  2. [1]

    Murdock Aubry, Haoming Meng, Anton Sugolov, and Vardan Papyan. 2024. Transformer block coupling and its correlation with generalization in llms. arXiv preprint arXiv:2407.07810 (2024)

    arXiv 2024

  3. [2]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609 (2023)

    arXiv 2023

  4. [3]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond

    2023

  5. [4]

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. 2024. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. In European Conference on Computer Vision. Springer, 19–35

    2024

  6. [5]

    Xiaodong Chen, Yuxuan Hu, Jing Zhang, Yanling Wang, Cuiping Li, and Hong Chen. 2024. Streamlining redundant layers to compress large language models. arXiv preprint arXiv:2403.19135 (2024)

    arXiv 2024

  7. [6]

    Matteo Farina, Massimiliano Mancini, Elia Cunegatti, Gaowen Liu, Giovanni Iacca, and Elisa Ricci. 2024. MULTIFLOW: Shifting Towards Task-Agnostic Vision-Language Pruning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 16185–16195

    work page 2024

  8. [7]

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323 (2022)

    arXiv 2022

Show all 52 references
  1. [8]

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. 2024. MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models. (2024)

    2024

  2. [9]

    Advait Harshal Gadhikar, Sohom Mukherjee, and Rebekka Burkholz. 2023. Why random pruning is all we need to start sparse. In International Conference on Machine Learning. PMLR, 10542–10570

    2023

  3. [11]

    Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, and Zicheng Liu. 2022. Playing lottery tickets with vi- sion and language. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 36. 652–660

    2022

  4. [12]

    Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, and Daniel A Roberts. 2024. The unreasonable ineffectiveness of the deeper layers. arXiv preprint arXiv:2403.17887 (2024)

    2024 arXiv

  5. [13]

    Song Han, Jeff Pool, John Tran, and William Dally. 2015. Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015)

    2015

  6. [14]

    Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. [n. d.]. The Curious Case of Neural Text Degeneration. InInternational Conference on Learning Representations

  7. [15]

    Chaoya Jiang, Haiyang Xu, Mengfan Dong, Jiaxing Chen, Wei Ye, Ming Yan, Qing- hao Ye, Ji Zhang, Fei Huang, and Shikun Zhang. 2024. Hallucination augmented contrastive learning for multimodal large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and...

    2024

  8. [16]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 (2020)

    2020 arXiv

  9. [17]

    Virginia Klema and Alan Laub. 1980. The singular value decomposition: Its computation and some applications. IEEE Transactions on automatic control 25, 2 (1980), 164–176

    1980

  10. [18]

    Bohao Li, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. 2024. Seed-bench: Benchmarking multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13299–13308

    2024

  11. [19]

    Jianing Li and Vardan Papyan. 2023. Residual alignment: uncovering the mecha- nisms of residual networks. Advances in Neural Information Processing Systems 36 (2023), 57660–57712

    2023

  12. [20]

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen

  13. [21]

    Haokun Lin, Haoli Bai, Zhili Liu, Lu Hou, Muyi Sun, Linqi Song, Ying Wei, and Zhenan Sun. 2024. Mope-clip: Structured pruning for efficient vision-language models with module-wise pruning error metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...

    2024

  14. [22]

    Zhihang Lin, Mingbao Lin, Luxi Lin, and Rongrong Ji. 2024. Boosting multimodal large language models with visual tokens withdrawal for rapid inference. arXiv preprint arXiv:2405.05803 (2024)

    2024 arXiv

  15. [24]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruc- tion Tuning

    2023

  16. [25]

    Shiwei Liu, Tianlong Chen, Zhenyu Zhang, Xuxi Chen, Tianjin Huang, AJAY KU- MAR JAISWAL, and Zhangyang Wang. [n. d.]. Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!. In The Eleventh International Con- ference on Learning Representations

  17. [26]

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al . 2024. Mmbench: Is your multi-modal model an all-around player?. In European conference on computer vision. Springer, 216–233

    2024

  18. [27]

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to explain: Multi- modal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems 35 ...

    2022

  19. [28]

    2024.𝑔𝑎𝑚𝑚𝑎− MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models

    Yaxin Luo, Gen Luo, Jiayi Ji, Yiyi Zhou, Xiaoshuai Sun, Zhiqiang Shen, and Rongrong Ji. 2024.𝑔𝑎𝑚𝑚𝑎− MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models. arXiv preprint arXiv:2410.13859 (2024)

    2024 arXiv

  20. [29]

    Ji Ma, Wei Suo, Peng Wang, and Yanning Zhang. 2024. C3 L: content correlated vision-language instruction tuning data generation via contrastive learning. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelli- gence. 1155–1163

    2024

  21. [30]

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023. Llm-pruner: On the struc- tural pruning of large language models.Advances in neural information processing systems 36 (2023), 21702–21720

    2023

  22. [31]

    Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. 2024. Shortgpt: Layers in large language models are more redundant than you expect. arXiv preprint arXiv:2403.03853 (2024)

    2024 arXiv

  23. [32]

    Clement Neo, Luke Ong, Philip Torr, Mor Geva, David Krueger, and Fazl Barez

  24. [33]

    Azade Nova, Hanjun Dai, and Dale Schuurmans. 2023. Gradient-free structured pruning with unlabeled data. In International Conference on Machine Learning . PMLR, 26326–26341

    2023

  25. [34]

    Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277 (2023)

    2023 arXiv

  26. [35]

    Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. 2022. A-okvqa: A benchmark for visual question answering using world knowledge. In European conference on computer vision . Springer, 146–162

    2022

  27. [36]

    Yuzhang Shang, Mu Cai, Bingxin Xu, Yong Jae Lee, and Yan Yan. 2024. Llava- prumerge: Adaptive token reduction for efficient large multimodal models. arXiv preprint arXiv:2403.15388 (2024)

    2024

  28. [37]

    Dachuan Shi, Chaofan Tao, Ying Jin, Zhendong Yang, Chun Yuan, and Jiaqi Wang. 2023. UPop: Unified and progressive pruning for compressing vision- language transformers. In International Conference on Machine Learning . PMLR, 31292–31311

    2023

  29. [38]

    Jiwon Song, Kyungseok Oh, Taesu Kim, Hyungjun Kim, Yulhwa Kim, and Jae- Joon Kim. 2024. Sleb: Streamlining llms through redundancy verification and elimination of transformer blocks. arXiv preprint arXiv:2402.09025 (2024)

    2024 arXiv

  30. [39]

    Wei Suo, Lanqing Lai, Mengyang Sun, Hanwang Zhang, Peng Wang, and Yanning Zhang. 2024. Rethinking and improving visual prompt selection for in-context learning segmentation. In European Conference on Computer Vision . Springer, 18–35

    2024

  31. [40]

    Wei Suo, Ji Ma, Mengyang Sun, Lin Yuanbo Wu, Peng Wang, and Yanning Zhang

  32. [41]

    Wei Suo, Mengyang Sun, Weisong Liu, Yiqi Gao, Peng Wang, Yanning Zhang, and Qi Wu. 2023. S3c: Semi-supervised vqa natural language explanation via self-critical learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2646–2656

    2023

  33. [42]

    Wei Suo, Lijun Zhang, Mengyang Sun, Lin Yuanbo Wu, Peng Wang, and Yanning Zhang. 2025. Octopus: Alleviating hallucination via dynamic contrastive decod- ing. In Proceedings of the Computer Vision and Pattern Recognition Conference . 29904–29914

    2025

  34. [43]

    arXiv preprint arXiv:2412.06458 (2024)

    Pruning All-Rounder: Rethinking and Improving Inference Efficiency for Large Vision Language Models. arXiv preprint arXiv:2412.06458 (2024)

    2024 arXiv

  35. [44]

    Qiong Wu, Zhaoxi Ke, Yiyi Zhou, Xiaoshuai Sun, and Rongrong Ji. [n. d.]. Rout- ing Experts: Learning to Route Dynamic Experts in Existing Multi-modal Large Language Models. In The Thirteenth International Conference on Learning Repre- sentations

  36. [45]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al . 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115 (2024)

    2024 arXiv

  37. [46]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  38. [47]

    Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. 2023. mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration

    2023

  39. [48]

    Xubing Ye, Yukang Gan, Yixiao Ge, Xiao-Ping Zhang, and Yansong Tang. 2024. ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models. arXiv preprint arXiv:2412.00447 (2024)

    2024 arXiv

  40. [49]

    Le Yang, Ziwei Zheng, Boxu Chen, Zhengyu Zhao, Chenhao Lin, and Chao Shen

  41. [53]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models

    2023

  42. [2023]

    arXiv preprint arXiv:2305.10355 (2023)

    Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355 (2023)

    2023 arXiv

  43. [2024]

    arXiv preprint arXiv:2410.07149 (2024)

    Towards interpreting visual information processing in vision-language models. arXiv preprint arXiv:2410.07149 (2024)

    2024 arXiv

  44. [2025]

    In Proceedings of the Computer Vision and Pattern MM ’25, October 27–31, 2025, Dublin, Ireland Ji Ma, Wei Suo, Peng Wang, and Yanning Zhang Recognition Conference

    Nullu: Mitigating object hallucinations in large vision-language models via halluspace projection. In Proceedings of the Computer Vision and Pattern MM ’25, October 27–31, 2025, Dublin, Ireland Ji Ma, Wei Suo, Peng Wang, and Yanning Zhang Recognition Conference. 14635–14645

    2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.