Pith. sign in

REVIEW 3 major objections 5 minor 28 references

Streamline Without Sacrifice -- Squeeze out Computation Redundancy in LMM

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that decoder-only large multimodal models perform redundant computation on vision tokens in their middle and later layers, and that replacing that computation with lightweight proxy-token updates can cut prefill cost by…

desk verdict Useful new efficiency lever for LMM prefill, but the 'no sacrifice' claim is tuned to the same benchmarks it's measured on. read the letter →

arxiv 2505.15816 v1 pith:NJKVGJG7 submitted 2025-05-21 cs.CV

Penghao Wu , Lewei Lu , Ziwei Liu This is my paper
classification cs.CV
keywords largemultimodalmodelscomputation-levelredundancyvisiontokenefficiencyproxytokensdecoder-onlyLMMsvisualinformationpreservationmodelaccelerationfine-grainedunderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the expensive part of a large multimodal model is not only the number of visual tokens but the heavy computation spent on each one inside the language model. It claims that in the middle and later decoder layers, the self-attention and feed-forward operations on vision tokens are largely redundant, because the vision tokens already carry semantic information from the pretrained image encoder. To exploit that redundancy, the paper proposes ProxyV: a small set of proxy tokens absorbs the heavy computation, and a lightweight MLP updates each full visual token from its spatially matching proxy token, so no visual token is ever discarded. Reported results show prefill FLOPs and time cut by 36--46% and 31--41% while fine-grained benchmark scores stay at 101--102% of baseline on Vicuna1.5-7B, with similar no-loss or slight-gain behavior on five other LLMs. If this holds, LMMs can run faster without the information loss that token pruning and merging risk on dense documents and grounding tasks.

What carries the argument

The load-bearing object is the proxy vision token with a guided-update module. Given $N\times N$ full vision tokens, ProxyV downsamples by a factor $r=4$ to $M\times M=N/r$ proxy tokens; in each decoder layer only the proxy tokens and text tokens serve as queries and pass through self-attention and FFNs, while full vision tokens are excluded from those heavy operations. Each proxy token then updates its spatially corresponding $r\times r$ block of full vision tokens through a two-layer MLP that concatenates the down-projected full token with its proxy token. This machinery converts the paper's finding -- that vision attention and FFNs are redundant in middle and later layers -- into an architecture that removes the heavy per-token computation while keeping a cheap information conduit. A non-spatial variant replaces the 2D grid prior with learnable queries $Q\in\mathbb{R}^{m\times d}$, keys $K\in\mathbb{R}^{n\times d}$ from the full tokens, and values equal to the full tokens, reusing the attention logits to splat proxy information back onto every full token.

What would settle it

Take a dense-document benchmark and construct image pairs that differ only in one small critical detail, such as a digit in a table or the location of a small object; if a ProxyV-finetuned model answers the pair less accurately than the no-acceleration baseline while a matched token-reduction model does not, the proxy-to-token update has lost information, contradicting the paper's core claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that decoder-only large multimodal models spend a large portion of their compute redundantly on vision tokens: in the middle and later layers, the self-attention and feed-forward operations applied to vision tokens can be skipped or replaced without harming, and sometimes improving, fine-grained visual task performance. The evidence comes in three steps: masking vision-to-vision attention at different depths shows minimal or no performance loss when masking starts in middle or later layers across six LLMs; finetuning with vision attention skipped narrows the gap while saving limited FLOPs; replacing both attention and FFN on vision tokens with lightweight vision-specific MLPs cuts FLOPs sharply and adds a performance gain. On that basis ProxyV downsamples the full vision tokens into proxy tokens, runs the heavy operations only on proxy and text tokens, and updates each full vision token from its spatially corresponding proxy token through a lightweight two-layer MLP. In the paper's reported numbers, ProxyV on Vicuna1.5-7B reaches 101% of baseline fine-grained score with 46% fewer prefill FLOPs (41% less time) when applied from layer 12, and 102.4% with 36% fewer FLOPs (31% less time) from layer 16; similar no-loss or modest-gain results are shown for five other LLMs.

Load-bearing premise

The whole result depends on the assumption that the lightweight guided-update module can transfer all task-relevant visual information from proxy tokens back to the full vision tokens, even for dense documents and grounding tasks where a single missing detail can change the answer.

Editorial extensions

If this is right

  • LMMs can keep every vision token while removing a large share of vision-token compute, so dense documents, charts, and grounding images retain all fine-grained information.
  • Applying ProxyV from middle layers preserves or improves fine-grained scores across Vicuna-7B/13B, Llama3-8B, Qwen2-7B, Phi3-3B, and InternLM2.5-7B, meaning the effect is not tied to one backbone.
  • A non-spatial variant removes the 2D-grid requirement and can be combined with token-reduction methods such as VisionZip, reaching about 62% FLOPs and 65% time reduction with near-baseline scores.
  • Under matched efficiency, token-reduction baselines lose substantially on document parsing and RefCOCO grounding, whereas ProxyV does not, supporting the paper's claim that avoiding token loss protects fine-grained understanding.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's evidence is limited to single-image, high-resolution tasks; extending ProxyV to video or interleaved multi-image LMMs, where vision tokens are far more numerous, is a natural next test that the authors do not run.
  • The varying layer-wise redundancy across LLMs suggests a learned or calibrated per-model switch point could squeeze out additional compute beyond the fixed layer-12/16 choices tested here.
  • The authors explicitly leave local attention or convolution inside the guided-update module for future work; replacing the two-layer MLP with a local-window mechanism is a concrete way to test whether fine-grained inter-token interactions are the bottleneck.
  • The reported drop in Modality Integration Rate under ProxyV implies the gains come partly from better text-vision alignment; a targeted probe on alignment-sensitive tasks could predict where ProxyV will help most.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper identifies computation-level redundancy in the processing of vision tokens within decoder-only large multimodal models (LMMs). Rather than removing vision tokens, it proposes ProxyV, which downsamples full vision tokens into a smaller set of proxy tokens that participate in the original self-attention and FFN operations, while the full vision tokens are updated by lightweight MLPs guided by their spatially corresponding proxy tokens. The authors report that ProxyV maintains or slightly improves performance on fine-grained OCR-heavy benchmarks across six LLM backbones while reducing prefill FLOPs by 25-46%, and they introduce a non-spatial variant that can be combined with token-reduction methods. The paper concludes that attention-related computation redundancy exists in middle and later layers of LMMs, with varying degrees across backbones.

Significance. If the central claims hold, ProxyV offers a general and token-preserving approach to reducing LMM inference cost that is orthogonal to token reduction, and the redundancy analysis across multiple LLMs is a useful empirical contribution. The paper's strengths include experiments across six LLM backbones, evaluation on fine-grained OCR benchmarks, comparison with token-reduction methods on grounding and document parsing tasks, and a non-spatial variant that integrates with existing token pruning. The authors also provide detailed benchmark tables and specify their evaluation configuration. However, the 'without sacrifice' claim is weakened by the apparent selection of the ProxyV start layer on the same benchmarks used for evaluation and by the absence of repeated runs or error bars.

major comments (3)
  1. [Section 4, Table 4] The choice of ProxyV start layer (12, 16, or 20) for each backbone is not governed by a stated rule, and the layers appear to be selected based on the redundancy patterns in Figure 2, which are measured on the same Score_fine benchmark suite (DocVQA, ChartQA, InfoVQA, OCRBench, TextVQA) used to report final performance. Selecting the start layer on the evaluation benchmarks and then reporting performance on those same benchmarks constitutes test-set tuning, which directly threatens the claim that ProxyV reduces computation 'without sacrifice.' The authors should provide an independent layer-selection protocol (e.g., a validation split or a fixed rule based on model depth) and report results under that protocol.
  2. [Table 5] The claim of 'no sacrifice' is contradicted by the authors' own document parsing numbers: ProxyV-L12 has DocParsing-BLEU 0.7923 versus 0.7991 for the baseline and DocParsing-ED 0.1566 versus 0.1551, both slightly worse. The paper does not discuss this degradation, and given that all results are single runs without error bars, the small gains in Score_fine (0.5-1.5 points) and the small losses on document parsing are within the range of run-to-run noise. The authors should report multiple seeds with standard deviations or at least acknowledge these discrepancies in the text.
  3. [Section 3.1] The guided-update module assumes that a lightweight two-layer MLP, fed with each full vision token concatenated with its spatially corresponding proxy token, can transfer all task-relevant information that the original self-attention and FFN would have provided. The paper itself notes that local attention or convolution layers could further promote fine-grained inter-token interactions and leaves this for future work, which is an implicit admission that the current design may not fully capture local fine-grained interactions. The empirical results on fine-grained benchmarks are positive, but the strong claim of 'no information loss' is not fully supported without testing the module's behavior on more diverse dense-image tasks or incorporating such local interactions.
minor comments (5)
  1. [Abstract] The word 'prefling' appears in the abstract and should be 'prefilling.'
  2. [Table 4] The entry 'ProxyV - Layer 1655.94' is missing a space between 'Layer 16' and the score '55.94.'
  3. [Section 3.3] The text writes 'VisioZip' in one place; this should be 'VisionZip' for consistency with the rest of the paper.
  4. [Section 2] The notation for the fine-grained average is inconsistent: 'Score_fine', 'Scorefine', and 'Score fine' are all used. Please unify.
  5. [Section 4] The reported FLOPs and time are for a fixed configuration (five image grids and 50 text tokens) with eager attention; the paper should state more explicitly that the efficiency gains are configuration-dependent and may differ with other text-to-vision token ratios or with FlashAttention-style implementations.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline 'without sacrifice' is partially selected: the start layers in Table 4 are chosen per backbone on the same Score_fine benchmarks used for the redundancy discovery in Figure 2.

  1. fitted input called prediction [Section 2 (Figure 2, Score_fine definition); Section 3.1 (Table 3); Section 4 (Table 4)]
    "To evaluate the impact of this masking on the performance of LMMs, we select a set of OCR-extensive benchmarks ... that require fine-grained visual information, thus being highly sensitive to potential information loss in vision tokens, and denote the average performance on them asScore fine. ... The results indicate that applying ProxyV from the middle layers can achieve no performance loss or a small performance gain (100% - 101%) with moderate efficiency improvement."

    The paper first measures, on the same five fine-grained benchmarks that define Score_fine, where attention masking can be applied with no drop (Figure 2). It then presents ProxyV's headline 101%-102.4% relative scores on that same Score_fine aggregate, using per-backbone start layers (L12/L16/L20) that vary across models. No independent rule for choosing the start layer is given: Section 4 fixes r=4 but says nothing about layer selection. Choosing the layer after observing the redundancy curve on the evaluation benchmarks makes the 'no sacrifice' result a selected outcome rather than a prediction: the start layer is effectively fit to Score_fine, and the relative Score_fine is then reported as the measured consequence.

full rationale

Aside from the selection step, the paper contains no equation-level circularity and no load-bearing self-citation chain: ProxyV is an empirical architecture, the FLOPs/time reductions are direct arithmetic consequences of replacing full-token attention and FFN processing with r=4 proxy tokens, and comparisons to VisionZip, PyramidDrop, RefCOCO, and DocParsing are external controls. The guided-update module's acknowledged lack of local fine-grained interactions is a completeness or robustness limitation, not circularity. The main circularity concern is that the discovery experiments (Section 2) and the final demonstration (Tables 3 and 4) are both on Score_fine, and the start layer is selected per backbone after inspecting the discovery curve. If the layer were chosen by a held-out rule or fixed protocol, the headline numbers would be independent predictions; as written, the 100%-102.4% figures are partially selected outcomes. The score of 6 reflects partial circularity of the central 'no sacrifice' claim, not of the efficiency arithmetic.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claims rest on several hand-chosen hyperparameters (r, MLP hidden dim, per-model start layer) and domain assumptions about the semantic richness of pretrained vision tokens and the sufficiency of the lightweight guided-update MLP. The proxy tokens themselves are internal architectural constructs with no external falsifiable handle.

free parameters (4)
  • Downsampling factor r = 4
    Chosen by hand in Section 4; determines proxy count (36 proxies from 576 full tokens). No ablation or external justification is provided.
  • Guided-update MLP hidden dim = 1/4 of LLM hidden dim
    Set to 1/4 of the LLM hidden dimension in Section 4; the paper does not report a sweep over this ratio.
  • ProxyV start layer per backbone = Vicuna-7B: L12/L16; Vicuna-13B: L16/L20; Llama3-8B: L16/L20; Qwen2-7B: L16/L20; Phi3-3B: L16/L20; InternLM2.5-7B…
    Reported in Table 4. The start layer is selected per model based on fine-grained benchmark results from the exploratory experiments, making it a fitted choice rather than a fixed rule.
  • Number of learnable queries in non-spatial variant = same as spatial (36)
    Section 3.3 sets m to match the spatial proxy count; no ablation is reported.
assumptions (4)
  • domain assumption Vision tokens from the pretrained encoder are already highly semantic and therefore do not require all heavy attention/FFN operations in the LLM.
    Invoked in the Introduction and Section 2 to motivate the search for computation-level redundancy. If wrong, skipping operations would lose information; the paper provides only indirect empirical support.
  • ad hoc to paper A lightweight two-layer MLP fed with each full token concatenated with its spatially corresponding proxy token can replace the function of self-attention and FFN for vision-token updates.
    Section 3.1. The paper acknowledges it could add local attention or convolution but leaves that out, so the sufficiency of the MLP is assumed.
  • domain assumption Standard two-stage LMM training (ShareGPT4V pretraining + LLaVA-Next finetuning) is sufficient to train the new modules without special adaptation.
    Section 4. The method inherits all assumptions of the LLaVA-Next recipe, including the quality and generality of those training sets.
  • domain assumption The average of five OCR-heavy benchmarks (DocVQA, ChartQA, InfoVQA, OCRBench, TextVQA) is a sufficient proxy for "no information loss" in vision tokens.
    The paper's main claim of no sacrifice is based on Score_fine (Section 2). General benchmarks in Table 7 show mixed results, so this benchmark choice is load-bearing.
invented entities (2)
  • Proxy vision tokens (spatial variant)
    purpose: Downsampled MxM tokens that replace full vision tokens in self-attention and FFNs, then guide updates of full tokens via MLP.
    An internal architectural device. Its utility is demonstrated only by the paper's own benchmark comparisons; no external falsifiable prediction is made.
  • Learnable proxy queries Q in non-spatial variant
    purpose: m learnable embeddings that pool full vision tokens via a learned attention operation, removing the spatial prior.
    A learned parameter set introduced in Section 3.3; no evidence outside the paper's experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Streamline Without Sacrifice -- Squeeze out Computation Redundancy in LMM." pith.science (2026). https://pith.science/paper/NJKVGJG7

@misc{pith2026250515816,
  author       = {Pith},
  title        = {Pith review of: Streamline Without Sacrifice -- Squeeze out Computation Redundancy in LMM},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NJKVGJG7}},
  note         = {Machine review of arXiv:2505.15816}
}
read the original abstract

Large multimodal models excel in multimodal tasks but face significant computational challenges due to excessive computation on visual tokens. Unlike token reduction methods that focus on token-level redundancy, we identify and study the computation-level redundancy on vision tokens to ensure no information loss. Our key insight is that vision tokens from the pretrained vision encoder do not necessarily require all the heavy operations (e.g., self-attention, FFNs) in decoder-only LMMs and could be processed more lightly with proper designs. We designed a series of experiments to discover and progressively squeeze out the vision-related computation redundancy. Based on our findings, we propose ProxyV, a novel approach that utilizes proxy vision tokens to alleviate the computational burden on original vision tokens. ProxyV enhances efficiency without compromising performance and can even yield notable performance gains in scenarios with more moderate efficiency improvements. Furthermore, the flexibility of ProxyV is demonstrated through its combination with token reduction methods to boost efficiency further. The code will be made public at this https://github.com/penghao-wu/ProxyV URL.

Figures

Figures reproduced from arXiv: 2505.15816 by the authors.

Figure 1
Figure 1. ProxyV retains or increases the fine-grained benchmark performance while effectively reducing the computational cost. ProxyV-L12 and ProxyV-L16 denote applying ProxyV from layers 12 and 16, respectively. computation-level redundancy exists in the processing of vi￾sion tokens and can be squeezed out through proper designs without sacrificing performance. Current mainstream LMMs adopt a typical decoder-only architectu… view at source ↗
Figure 2
Figure 2. The relative Scorefine with different vision attention masking ratios for different LLMs. The computation redundancy begins in the middle to rear part of the LLMs as masking the vision attention does not affect the performance. to potential information loss in vision tokens, and denote the average performance on them as Scorefine. As shown in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Left: the vanilla LMM structure where full vision tokens cause significant computation. Right: the overall pipeline of the proposed ProxyV algorithm. The full vision tokens are first downsampled to obtain a much smaller version that works as proxy vision tokens. The proxy vision tokens participate in the original operations in the decoder layer including the self-attention and the FFNs to obtain useful information a… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The illustration of the non-spatial ProxyV. Upper part: proxy vision tokens are generated as a weighted combination of full vision tokens through a simple attention operation. Lower part: The previous attention score is reused to splat the proxy vision tokens into guid…
Figure 5
Figure 5. Figure 5: Cases where token reduction methods fail. Left: Token reduction methods fail to extract the complete dense information accurately. Right: Token reduction methods fail to retain critical visual information when the image contains diverse and dense visual details. In the…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 2 canonical work pages

  1. [1]

    A., Bach, N., Bahree, A., Bakhtiari, A., Bao, J., Behl, H., et al

    Abdin, M., Aneja, J., Awadalla, H., Awadallah, A., Awan, A. A., Bach, N., Bahree, A., Bakhtiari, A., Bao, J., Behl, H., et al. Phi-3 technical report: A highly capable lan- guage model locally on your phone.arXiv preprint arXiv:2404.14219,

    arXiv

  2. [5]

    Flashattention-2: Faster attention with bet- ter parallelism and work partitioning.arXiv preprint arXiv:2307.08691,

    Dao, T. Flashattention-2: Faster attention with bet- ter parallelism and work partitioning.arXiv preprint arXiv:2307.08691,

    arXiv

  3. [6]

    Benchmarking and improving detail image caption

    Dong, H., Li, J., Wu, B., Wang, J., Zhang, Y ., and Guo, H. Benchmarking and improving detail image caption. arXiv preprint arXiv:2405.19092,

    arXiv

  4. [7]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    arXiv

  5. [8]

    Feather the throt- tle: Revisiting visual token pruning for vision-language model acceleration.arXiv preprint arXiv:2412.13180,

    Endo, M., Wang, X., and Yeung-Levy, S. Feather the throt- tle: Revisiting visual token pruning for vision-language model acceleration.arXiv preprint arXiv:2412.13180,

    arXiv

  6. [9]

    Mme: A comprehen- sive evaluation benchmark for multimodal large language models.arXiv preprint arXiv:2306.13394,

    Fu, C., Chen, P., Shen, Y ., Qin, Y ., Zhang, M., Lin, X., Yang, J., Zheng, X., Li, K., Sun, X., et al. Mme: A comprehen- sive evaluation benchmark for multimodal large language models.arXiv preprint arXiv:2306.13394,

    arXiv

  7. [10]

    Model tells you what to discard: Adaptive kv cache compression for llms.arXiv preprint arXiv:2310.01801,

    Ge, S., Zhang, Y ., Liu, L., Zhang, M., Han, J., and Gao, J. Model tells you what to discard: Adaptive kv cache compression for llms.arXiv preprint arXiv:2310.01801,

    arXiv

  8. [11]

    Zipvl: Efficient large vision-language models with dynamic token sparsification and kv cache compression.arXiv preprint arXiv:2410.08584,

    He, Y ., Chen, F., Liu, J., Shao, W., Zhou, H., Zhang, K., and Zhuang, B. Zipvl: Efficient large vision-language models with dynamic token sparsification and kv cache compression.arXiv preprint arXiv:2410.08584,

    arXiv

Show all 28 references
  1. [12]

    mplug-docowl 1.5: Unified structure learning for ocr-free document under- standing.arXiv preprint arXiv:2403.12895,

    Hu, A., Xu, H., Ye, J., Yan, M., Zhang, L., Zhang, B., Li, C., Zhang, J., Jin, Q., Huang, F., et al. mplug-docowl 1.5: Unified structure learning for ocr-free document under- standing.arXiv preprint arXiv:2403.12895,

    arXiv

  2. [15]

    Seed-bench: Benchmarking multimodal llms with gener- ative comprehension.arXiv preprint arXiv:2307.16125, 2023a

    Li, B., Wang, R., Wang, G., Ge, Y ., Ge, Y ., and Shan, Y . Seed-bench: Benchmarking multimodal llms with gener- ative comprehension.arXiv preprint arXiv:2307.16125, 2023a. Li, B., Zhang, P., Zhang, K., Pu, F., Du, X., Dong, Y ., Liu, H., Zhang, Y ., Zhang, G., Li, C., and Liu...

    2024 arXiv

  3. [16]

    Ee-mllm: A data-efficient and compute- efficient multimodal large language model.arXiv preprint arXiv:2408.11795,

    Ma, F., Zhou, Y ., Li, H., He, Z., Wu, S., Rao, F., Zhang, Y ., and Sun, X. Ee-mllm: A data-efficient and compute- efficient multimodal large language model.arXiv preprint arXiv:2408.11795,

    arXiv

  4. [19]

    Look-m: Look-once optimization in kv cache for efficient multimodal long-context inference

    Wan, Z., Wu, Z., Liu, C., Huang, J., Zhu, Z., Jin, P., Wang, L., and Yuan, L. Look-m: Look-once optimization in kv cache for efficient multimodal long-context inference. arXiv preprint arXiv:2406.18139,

    arXiv

  5. [20]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191,

    Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191,

    arXiv

  6. [21]

    Pyramid- drop: Accelerating your large vision-language models via pyramid visual redundancy reduction.arXiv preprint arXiv:2410.17247,

    Xing, L., Huang, Q., Dong, X., Lu, J., Zhang, P., Zang, Y ., Cao, Y ., He, C., Wang, J., Wu, F., et al. Pyramid- drop: Accelerating your large vision-language models via pyramid visual redundancy reduction.arXiv preprint arXiv:2410.17247,

    arXiv

  7. [22]

    Longvila: Scaling long- context visual language models for long videos.arXiv preprint arXiv:2408.10188,

    Xue, F., Chen, Y ., Li, D., Hu, Q., Zhu, L., Li, X., Fang, Y ., Tang, H., Yang, S., Liu, Z., et al. Longvila: Scaling long- context visual language models for long videos.arXiv preprint arXiv:2408.10188,

    arXiv

  8. [23]

    Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2024a

    Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., et al. Qwen2 technical report.arXiv preprint arXiv:2407.10671, 2024a. Yang, S., Chen, Y ., Tian, Z., Wang, C., Li, J., Yu, B., and Jia, J. Visionzip: Longer is better but not necessary in vi- si...

    arXiv

  9. [24]

    Long con- text transfer from language to vision.arXiv preprint arXiv:2406.16852, 2024a

    Zhang, P., Zhang, K., Li, B., Zeng, G., Yang, J., Zhang, Y ., Wang, Z., Tan, H., Li, C., and Liu, Z. Long con- text transfer from language to vision.arXiv preprint arXiv:2406.16852, 2024a. Zhang, Y ., Fan, C.-K., Ma, J., Zheng, W., Huang, T., Cheng, K., Gudovskiy, D., Okuno, T...

    arXiv

  10. [25]

    N., and Yu, L

    Zhao, S., Wang, Z., Juefei-Xu, F., Xia, X., Liu, M., Wang, X., Liang, M., Zhang, N., Metaxas, D. N., and Yu, L. Accelerating multimodel large language models by searching optimal vision token reduction.arXiv preprint arXiv:2412.00556,

    arXiv

  11. [26]

    Aim: Adaptive inference of multi-modal llms via token merging and pruning.arXiv preprint arXiv:2412.03248,

    Zhong, Y ., Liu, Z., Li, Y ., and Wang, L. Aim: Adaptive inference of multi-modal llms via token merging and pruning.arXiv preprint arXiv:2412.03248,

    arXiv

  12. [27]

    12 Streamline Without Sacrifice - Squeeze out Computation Redundancy in LMM A. Evaluation on More Benchmarks Besides the fine-grained benchmarks, here we evaluate models on a wide range of general multimodal benchmarks including MMBench (Liu et al., 2025), SEED-Bench (Li et al...

    2025

  13. [28]

    Table 7.Evaluation on a comprehensive set of general multimodal benchmarks

    We can observe that ProxyV consistently achieves no performance loss or even improvements on general multimodal benchmarks. Table 7.Evaluation on a comprehensive set of general multimodal benchmarks. MMBench SEED-Img RefCOCO MMStar GQA MME-P MMMU POPE SQA AI2D RealWorldQA Vicu...

    2021

  14. [2002]

    Cross-self kv cache pruning for efficient vision-language inference.arXiv preprint arXiv:2412.04652,

    Pei, X., Huang, T., and Xu, C. Cross-self kv cache pruning for efficient vision-language inference.arXiv preprint arXiv:2412.04652,

    arXiv

  15. [2019]

    Mantis: Interleaved multi-image instruction tuning.arXiv preprint arXiv:2405.01483, 2024a

    Jiang, D., He, X., Zeng, H., Wei, C., Ku, M., Liu, Q., and Chen, W. Mantis: Interleaved multi-image instruction tuning.arXiv preprint arXiv:2405.01483, 2024a. Jiang, L., Huang, W., Liu, T., Zeng, Y ., Li, J., Cheng, L., and Xu, X. Fopru: Focal pruning for efficient large visio...

    arXiv

  16. [2021]

    J., and Yan, Y

    Shang, Y ., Cai, M., Xu, B., Lee, Y . J., and Yan, Y . Llava- prumerge: Adaptive token reduction for efficient large multimodal models.arXiv preprint arXiv:2403.15388,

  17. [2022]

    Internlm2 technical report.arXiv preprint arXiv:2403.17297,

    Cai, Z., Cao, M., Chen, H., Chen, K., Chen, K., Chen, X., Chen, X., Chen, Z., Chen, Z., Chu, P., et al. Internlm2 technical report.arXiv preprint arXiv:2403.17297,

    arXiv

  18. [2023]

    Nvlm: Open frontier-class multimodal llms.arXiv preprint arXiv:2409.11402,

    Dai, W., Lee, N., Wang, B., Yang, Z., Liu, Z., Barker, J., Rintamaki, T., Shoeybi, M., Catanzaro, B., and Ping, W. Nvlm: Open frontier-class multimodal llms.arXiv preprint arXiv:2409.11402,

    arXiv

  19. [2024]

    Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024a

    Chen, L., Li, J., Dong, X., Zhang, P., Zang, Y ., Chen, Z., Duan, H., Wang, J., Qiao, Y ., Lin, D., et al. Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024a. Chen, L., Li, J., Dong, X., Zhang, P., He, C., Wang, J., Zhao...

    arXiv

  20. [2025]

    Deciphering cross-modal alignment in large vision-language models with modality integration rate.arXiv preprint arXiv:2410.07167,

    Huang, Q., Dong, X., Zhang, P., Zang, Y ., Cao, Y ., Wang, J., Lin, D., Zhang, W., and Yu, N. Deciphering cross-modal alignment in large vision-language models with modality integration rate.arXiv preprint arXiv:2410.07167,

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.