Pith. sign in

REVIEW 3 major objections 5 minor 39 references

Vision token pruning becomes a reconstruction problem, with a guarantee of keeping the most representative tokens.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

OC-VTP uses Slot Attention to select a few 'object-representative' vision tokens, reporting high accuracy retention at extreme pruning ratios, but the claimed guarantee is undercut by a non-differentiable selection and inconsistent 'best at every ratio' results.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection New object-centric pruning idea, solid experiments, but the guarantee claim doesn't hold up and the training path is unclear. the 3 major comments →

arxiv 2511.20439 v2 pith:5XHRS2NC submitted 2025-11-25 cs.CV cs.AI

Object-Centric Vision Token Pruning for Vision Language Models

Guangyuan Li , Rongzhen Zhao , Jinhong Deng , Yanbo Wang , Joni Pajarinen This is my paper
classification cs.CV cs.AI
keywords vision token pruningvision language modelsobject-centric learningslot attentionreconstruction-based selectioninference accelerationtoken efficiencyinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that vision token pruning for vision-language models can be turned from a heuristic art into a problem with an optimality guarantee. It proposes OC-VTP, which uses object-centric learning (Slot Attention) to group tokens into object-level slots, keeps the single most-attended token per slot, and trains a lightweight decoder to reconstruct the full token set from that kept subset. The authors argue that minimizing this reconstruction error is a direct way to keep the most representative tokens, and that the resulting pruner, pretrained once on generic images, plugs into existing VLMs without any fine-tuning. Across three VLM families and pruning ratios from 33% down to about 5% of tokens, it reports the highest retained accuracy compared with all prior attention- and similarity-based methods, with substantial FLOP and latency savings.

Core claim

The paper's central claim: the token subset that best reconstructs the original unpruned tokens is the most representative subset, and it can be found by a Slot Attention-based selector trained through reconstruction. The OC-pruner aggregates tokens into object slots equal to the budget, picks the argmax-attention token per slot, and reconstructs the full sequence with a random autoregressive Transformer; training minimizes an area-weighted MSE that up-weights small informative regions. The authors state this is the first guaranteed VTP method, that it beats FastV, SparseVLM, VisionZip, PyramidDrop, and HiPrune at every pruning ratio on LLaVA-1.5, LLaVA-NeXT, and Qwen2.5-VL, and that retaine

What carries the argument

The OC-pruner: a Slot Attention module (a learned clustering that groups input tokens into a fixed number of object-level slots through competitive attention) producing an attention map over vision tokens; an argmax over that map selecting one token per slot as the kept token; and a random autoregressive Transformer decoder reconstructing the original sequence from those kept tokens. Training uses an Area-Weighted Mean-Squared Error, re-weighting each slot's reconstruction error by the inverse of its mask area so small informative regions are preserved. The claimed guarantee combines Slot Attention's exclusiveness (competition makes slots cover distinct objects) with the completeness enforce

Load-bearing premise

The entire guarantee rests on the assumption that the hard argmax token-selection step is effectively trained by the reconstruction loss, even though argmax is non-differentiable and the paper does not describe any surrogate gradient or straight-through estimator—if that assumption fails, the selection is never actually optimized, and the 'most representative' claim collapses.

What would settle it

Measure the reconstruction loss (with the paper's own decoder and AW-MSE) on the token subsets selected by the trained OC-pruner versus equally sized random subsets on held-out images; if the selected subsets do not achieve notably lower reconstruction error than random, the training signal is not reaching the selection mechanism. Alternatively, fine-tune the same architecture with a differentiable relaxation of selection (e.g., Gumbel-top-k) and show it significantly outperforms the published argmax version, which would demonstrate that the hard selection is not being optimized.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If correct, this is the first provably reconstruction-optimal vision token pruner, replacing handcrafted attention/similarity heuristics with an optimized selection criterion.
  • A single pretrained OC-pruner transfers across VLM architectures without fine-tuning, so token efficiency can be added to existing models at near-zero deployment cost.
  • The 5–17x FLOP reduction at high pruning ratios, with comparable latency gains, could make high-resolution or multi-image VLM inference practical on resource-limited hardware.
  • The object-level interpretability of the kept tokens opens the door to explainable or controllable VLM inference, where users can see which objects the model is attending to.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The strength of the 'guarantee' depends on how faithfully the random autoregressive decoder approximates the true information content of the original tokens; with a weak decoder, low reconstruction error may not imply high task-relevant information, so the guarantee is likely looser than stated.
  • The published training description does not show how the non-differentiable argmax selection is optimized; if gradients only flow through the decoder and slot attention's soft assignment, the hard selection indices may not be the ones learned, and the reported gains could stem largely from the slot-grouping itself rather than the final selection.
  • A natural test: replace the hard argmax with a Gumbel-softmax or straight-through estimator and compare reconstruction loss and downstream accuracy; a large gap would indicate that selection is not actually trained in the current formulation.
  • The weaker results on Qwen2.5-VL (which uses dynamic-resolution token counts) suggest the 'any budget, train once' claim may be specific to fixed-token-count encoders; variable token counts break the learned mapping from budget to number of slots.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces OC-VTP, a vision-token pruning method for VLMs. A Slot-Attention-based 'OC-pruner' is pre-trained on 40k COCO images and then inserted between a VLM's vision encoder and projector without fine-tuning the VLM. Given a token budget, the pruner maps vision tokens to slots, selects one token per slot via argmax attention, and prunes the rest. The pruner is trained with a reconstruction loss, including a proposed area-weighted MSE (AW-MSE), that reconstructs the original unpruned tokens from the selected tokens. The authors claim this provides a 'guaranteed' selection of the most representative vision tokens and that the method 'consistently' preserves the highest inference accuracy of mainstream VLMs at any pruning ratio. Experiments are reported on LLaVA-1.5, LLaVA-NeXT, and Qwen2.5-VL over multiple benchmarks and budgets.

Significance. The idea of using object-centric slot attention for token pruning is appealing, and the empirical comparison is broad, with code released. If the method worked as described, the plug-and-play, train-once property and the object-level interpretability would be useful contributions. However, the central 'guaranteed' claim is not established by the presented training formulation, and the uniformly-best accuracy claim is contradicted by one of the paper's own tables. The contribution is therefore conditional on substantial revision of both the method description and the claims.

major comments (3)
  1. [§3.3, Eqs. (7)–(10)] The training objective cannot train the selector as described. Eq. (7) defines I = argmax_n(A), and Eq. (8) gathers Vp = V[I,:]. Since argmax is piecewise constant and the gathered rows are copies of the input tokens, the reconstruction loss in Eq. (10) has zero gradient with respect to the Slot Attention parameters. The text says the pruner is trained 'by reconstructing ... from the slots,' but Eq. (9) uses Vp, not S. No straight-through estimator, Gumbel relaxation, or surrogate gradient is described. Thus the paper does not establish that the selection mechanism learns anything; the Comment in §3.3 asserting that Eq. (10) 'ensures completeness' is unsupported. If training intentionally reconstructs from the slots S rather than the selected tokens Vp, the paper must say so and reconcile Eq. (9); if it truly reconstructs from Vp, the training scheme needs to be revised.
  2. [§1 and §3.3 Comment] The 'guaranteed' claim is not proved. Minimizing a reconstruction loss on a training set does not imply that the greedy per-slot argmax yields the subset of size s with minimal reconstruction error on arbitrary inputs. Slot Attention's 'exclusiveness and completeness' properties are properties of the slot-attention mechanism, not formal optimality guarantees for the resulting token subset. Moreover, the target quantity in Eq. (2) is VLM accuracy, while Eq. (10) optimizes a proxy; no bound or transfer argument connects the two. The paper should either provide a precise formal statement with proof or remove 'guaranteed' and 'first guaranteed method' from the abstract and introduction.
  3. [Abstract and Table 3] The claim that OC-VTP 'consistently helps mainstream VLMs to preserve the highest inference accuracy' across any pruning ratio is contradicted by Table 3: at 33.3% retained tokens on Qwen2.5-VL, OC-VTP's average relative accuracy is 98.7% versus HiPrune's 98.8%. Similarly, in Table 1 (LLaVA-1.5, 33.3% retained), reading the last column as the average relative accuracy, HiPrune is 109% while OC-VTP is 108%. The headline claim should be corrected to reflect the actual rankings, e.g., 'competitive' or 'best in most settings.'
minor comments (5)
  1. [Abstract] Typo: 'gauranteed' should be 'guaranteed.'
  2. [§3.4] 'Beside, the vision tokens...' should be 'Besides, the vision tokens...'
  3. [§3.3 / §4] Training details are incomplete: report Slot Attention iterations, learning rate, batch size, optimizer, and the architecture/hyperparameters of the random autoregressive decoder. Currently only '40,000 images' is stated.
  4. [Tables 1–3] The rightmost 'average proportion' column contains values over 100% and some missing benchmarks; clarify how averages are computed when a benchmark is unavailable for a method.
  5. [Eq. (2)] The notation for the ideal pruning objective is malformed; define the expectations and the indicator notation clearly.

Circularity Check

1 steps flagged

The 'guaranteed most representative token' claim is the reconstruction objective restated as a theorem; benchmark comparisons are independent, so circularity is partial.

specific steps
  1. self definitional [Abstract; Section 3.1 Eq. (4); Section 3.3 Eqs. (9)-(10) and Comment]
    "It is gauranteed that the most representative vision tokens are kept by minimizing the error in reconstructing the original unpruned tokens from the selected ones. ... Comment. ... our training objective Equation (10) ensures the union of the kept tokens preserve the most information from the original unpruned tokens, i.e., completeness. These two aspects guarantee the most representative vision tokens are kept."

    Section 3.1 defines 'most representative' operationally as the tokens that minimize the distance d(V,Vp) in Eq. (4). Section 3.3 implements d as reconstruction from the selected tokens and trains with exactly that loss in Eq. (10). The Comment's assertion that Eq. (10) 'ensures ... completeness' and therefore 'guarantees' representativeness merely restates the objective used to fit the pruner; it is not an independent derivation. No proof is offered that the argmax selector attains the global minimum of Eq. (10) per image, nor that minimal reconstruction error implies higher VLM accuracy.

full rationale

The load-bearing theoretical claim—'guaranteed' selection of the most representative tokens—reduces to the reconstruction objective by construction: representativeness is defined as low reconstruction error and then certified by the same error. This is a self-definitional step rather than a derived optimality result. The empirical accuracy comparisons (Tables 1-3) are independent external evaluations; they are not circular, although Table 3 (OC-VTP 98.7 vs HiPrune 98.8 at 33.3% retained tokens on Qwen2.5-VL) tensions the abstract's 'consistently highest' accuracy claim, which is a correctness matter. The argmax/gather in Eqs. (7)-(8) is non-differentiable and no straight-through estimator/Gumbel surrogate is described, so the published training description does not establish that the selector is optimized; this is a trainability gap, not circularity itself. The self-citations [35]-[38] support object-centric decoder choices but are not invoked as a uniqueness theorem, so they are not load-bearing for the guarantee. Overall, because the central guarantee is the training objective restated, but the empirical content is independent, partial circularity (6) is appropriate.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 2 invented entities

The central result rests on several unproven assumptions: that reconstruction error transfers to downstream VLM accuracy, that Slot Attention's theoretical properties apply to hard token selection, that the non-differentiable argmax is trainable, and that COCO pretraining generalizes to all evaluation distributions. The slot budget and insertion layer are hand-picked or ablation-selected, and the learned query distribution plus decoder details are unreported.

free parameters (5)
  • Slot budget s = 32, 64, 128, 192 (sampled during training)
    The number of slots equals the target number of retained tokens and is sampled from a small set of budget values; the choice directly determines the pruning ratio.
  • Learned Gaussian query parameters = Not reported
    Slot Attention queries are sampled from a learned Gaussian distribution; the mean and variance are learned from the 40k COCO training set and are not reported.
  • Encoder insertion layer = Layer 9
    The middle-layer reference is chosen by ablation over Layer 9, Layer 10, and Layer -2 (Table 6), i.e., selected because it performed best on the evaluated benchmarks.
  • AW-MSE area weighting = Inverse mask area
    The area-weighted loss is introduced to preserve small objects and is adopted because Table 7 shows it outperforms plain MSE on the evaluated benchmarks.
  • Random autoregressive decoder architecture and hyperparameters = Not reported
    The decoder used for reconstruction is described only as a random auto-regressive Transformer; no architecture details, training schedule, or hyperparameters are provided.
axioms (4)
  • domain assumption Reconstruction error of vision tokens is a valid proxy for preserving VLM inference accuracy.
    Section 3.1 replaces the ideal objective in Eq. (2) with the practical reconstruction objective in Eq. (4) without establishing a formal link between reconstruction fidelity and downstream VLM accuracy.
  • domain assumption Slot Attention's exclusiveness and completeness guarantee that the selected tokens carry least information loss.
    Section 3.3's Comment asserts that the competition mechanism in Slot Attention ensures exclusiveness and the training objective ensures completeness, but this is an inherited property claim from prior OCL work, not a proof for the token-selection setting.
  • ad hoc to paper The hard argmax token selection is trainable through the reconstruction loss.
    Eqs. (7)-(10) describe a pipeline whose selection step is argmax, which is non-differentiable. The paper does not describe a straight-through estimator, soft relaxation, or alternative training signal, so this assumption is needed for the claimed learning to happen.
  • domain assumption A pruner trained on 40k COCO images generalizes zero-shot to all evaluation benchmarks and VLMs.
    Section 3.3 states the model is 'naively pre-trained' on COCO and then applied to diverse benchmarks; the paper treats this as a given rather than testing the distribution shift explicitly.
invented entities (2)
  • OC-pruner (object-centric token pruner) no independent evidence
    purpose: Selects representative vision tokens via Slot Attention followed by argmax over attention maps.
    The module is new and its effectiveness is only demonstrated in this paper's experiments; no external validation is provided.
  • AW-MSE (area-weighted mean-squared error) no independent evidence
    purpose: Re-weights the reconstruction loss by inverse mask area to preserve small but important image regions.
    This is a new loss term introduced to fix a failure mode of plain MSE; its benefit is shown only in the paper's Table 7 ablation.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Object-Centric Vision Token Pruning for Vision Language Models." pith.science (2026). https://pith.science/paper/5XHRS2NC

@misc{pith2026251120439,
  author       = {Pith},
  title        = {Pith review of: Object-Centric Vision Token Pruning for Vision Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5XHRS2NC}},
  note         = {Machine review of arXiv:2511.20439}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

In Vision Language Models (VLMs), vision tokens are quantity-heavy yet information-dispersed compared with language tokens, thus consume too much unnecessary computation. Pruning redundant vision tokens for high VLM inference efficiency has been continuously studied but all existing methods resort to indirect and non-guaranteed ways. We propose OC-VTP, a direct and guaranteed approach to select the most representative vision tokens for high-efficiency yet accuracy-preserving VLM inference. Our OC-VTP requires merely light-weight pre-training of a small object-centric vision token pruner, which can then be inserted into existing VLMs, without fine-tuning of any models on any datasets. It is gauranteed that the most representative vision tokens are kept by minimizing the error in reconstructing the original unpruned tokens from the selected ones. Across any vision pruning ratios, i.e., inference efficiency, our OC-VTP consistently helps mainstream VLMs to preserve the highest inference accuracy. Our pruning also demonstrates interesting interpretability. Our codes are available at https://github.com/GarryLarry010131/OC-VTP.

Figures

Figures reproduced from arXiv: 2511.20439 by Guangyuan Li, Jinhong Deng, Joni Pajarinen, Rongzhen Zhao, Yanbo Wang.

Figure 1
Figure 1. Figure 1: (left) Our OC-VTP consistently outperforms prior SotA methods, retaining over 95% of accuracy with only 11.1% of vi￾sual tokens on LLaVA-1.5. (right) Our OC-VTP reduces FLOPs by nearly 85% on LLaVA-1.5-7B when retaining 11.1% vision tokens, and by 95% on LLaVA-NeXT-7B when retaining 5.6% vision tokens, at a text length of 32, assuming MAC=2. the whole. This motivates Vision Token Pruning (VTP) to reduce VL… view at source ↗
Figure 2
Figure 2. Figure 2: (left) Structure of typical Vision-Language Models (VLMs), and three Vision Token Pruning (VTP) places: encoder methods like ToMe and FastV [3, 4]; decoder methods like SparseVLM and PyramidDrop [29, 34]; and in-between methods like TRIM, VisionZip and HiPrune [17, 25, 30]. Our OC-VTP operates in-between. (right) OC-pruner is the core of our OC-VTP. It takes the middle layer tokens from the vision encoder … view at source ↗
Figure 3
Figure 3. Figure 3: Inference time per image (ms). Average inference [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: OC-VTP visualization results. OC-Pruner retains one token each slot, and the slots represent different objects in the scene. The retaining budget is 64, and the test is conducted on LLaVA-1.5. slots top-k GQA MMB MME POPE VQAText MMMU SEED Avg Vanilla, 576 Tokens (100%) Vanilla 61.9 64.7 1862 85.9 58.2 36.3 58.6 100% 100% 100% 100% 100% 100% 100% 100% Retain 192 Tokens (33.3%) 32 Slots - Top 6 56.4 60.9 17… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 7 linked inside Pith

  1. [1]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 1, 2, 4, 6

    Pith/arXiv arXiv 2025

  2. [2]

    Invariant Slot Attention: Object Discovery with Slot- Centric Reference Frames

    Ondrej Biza, Sjoerd van Steenkiste, Mehdi SM Sajjadi, Gamaleldin Elsayed, Aravindh Mahendran, and Thomas Kipf. Invariant Slot Attention: Object Discovery with Slot- Centric Reference Frames. InInternational Conference on Machine Learning, pages 2507–2527, 2023. 2

    2023

  3. [3]

    Token merging: Your vit but faster

    Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your vit but faster. InICLR, 2023. 1, 3, 4

    2023

  4. [4]

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models

    Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. InEuropean Conference on Computer Vision, pages 19–35. Springer, 2024. 1, 2, 3, 4

    2024

  5. [5]

    Mme: A comprehensive evaluation bench- mark for multimodal large language models

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al. Mme: A comprehensive evaluation bench- mark for multimodal large language models. InThe Thirty- ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2025. 4

    2025

  6. [6]

    Making the v in vqa matter: Elevating the role of image understanding in visual question answer- ing

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Ba- tra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answer- ing. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 6904–6913, 2017

    2017

  7. [7]

    Vizwiz grand challenge: Answering visual questions from blind people

    Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. Vizwiz grand challenge: Answering visual questions from blind people. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 3608–3617, 2018

    2018

  8. [8]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 6700–6709, 2019. 2, 4

    2019

  9. [9]

    Improving Object-centric Learning with Query Optimization

    Baoxiong Jia, Yu Liu, and Siyuan Huang. Improving Object-centric Learning with Query Optimization. InThe Eleventh International Conference on Learning Representa- tions, 2023. 2

    2023

  10. [10]

    Spot: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformers

    Ioannis Kakogeorgiou, Spyros Gidaris, Konstantinos Karantzalos, and Nikos Komodakis. Spot: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformers. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22776–22786, 2024. 2

    2024

  11. [11]

    Seed-bench: Bench- marking multimodal large language models

    Bohao Li, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. Seed-bench: Bench- marking multimodal large language models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 13299–13308, 2024. 4

    2024

  12. [12]

    Evaluating object hallucina- tion in large vision-language models.arXiv preprint arXiv:2305.10355, 2023

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucina- tion in large vision-language models.arXiv preprint arXiv:2305.10355, 2023. 2, 4

    Pith/arXiv arXiv 2023

  13. [13]

    Mini-gemini: Mining the potential of multi-modality vision language models.arXiv preprint arXiv:2403.18814,

    Yanwei Li, Yuechen Zhang, Chengyao Wang, Zhisheng Zhong, Yixin Chen, Ruihang Chu, Shaoteng Liu, and Jiaya Jia. Mini-gemini: Mining the potential of multi-modality vision language models.arXiv preprint arXiv:2403.18814,

    Pith/arXiv arXiv

  14. [14]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer, 2014. 4

    2014

  15. [15]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023. 1, 2, 4

    2023

  16. [16]

    Llavanext: Improved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llavanext: Improved reasoning, ocr, and world knowledge, 2024. 1, 2, 4

    2024

  17. [17]

    Hiprune: Training-free visual token pruning via hierarchical attention in vision-language models.arXiv preprint arXiv:2508.00553, 2025

    Jizhihui Liu, Feiyi Du, Guangdao Zhu, Niu Lian, Jun Li, and Bin Chen. Hiprune: Training-free visual token pruning via hierarchical attention in vision-language models.arXiv preprint arXiv:2508.00553, 2025. 1, 2, 3, 4, 8

    Pith/arXiv arXiv 2025

  18. [18]

    Mmbench: Is your multi-modal model an all-around player? InEuropean conference on computer vi- sion, pages 216–233

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? InEuropean conference on computer vi- sion, pages 216–233. Springer, 2024. 2, 4

    2024

  19. [19]

    Object- centric learning with slot attention.Advances in neural in- formation processing systems, 33:11525–11538, 2020

    Francesco Locatello, Dirk Weissenborn, Thomas Un- terthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object- centric learning with slot attention.Advances in neural in- formation processing systems, 33:11525–11538, 2020. 2, 3, 4

    2020

  20. [20]

    Learn to explain: Multimodal reasoning via thought chains for science question answering.Advances in Neural Information Processing Systems, 35:2507–2521,

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering.Advances in Neural Information Processing Systems, 35:2507–2521,

  21. [21]

    Temporally consistent object-centric learning by contrasting slots

    Anna Manasyan, Maximilian Seitzer, Filip Radovic, Georg Martius, and Andrii Zadaianchuk. Temporally consistent object-centric learning by contrasting slots. InProceedings 9 of the Computer Vision and Pattern Recognition Conference, pages 5401–5411, 2025. 2

    2025

  22. [22]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021. 2

    2021

  23. [23]

    Bridging the gap to real-world object-centric learning

    Maximilian Seitzer, Max Horn, Andrii Zadaianchuk, Do- minik Zietlow, Tianjun Xiao, Carl-Johann Simon-Gabriel, Tong He, Zheng Zhang, Bernhard Sch¨olkopf, Thomas Brox, et al. Bridging the gap to real-world object-centric learning. InThe Eleventh International Conference on Learning Rep- resentations (ICLR 2023). OpenReview, 2023. 2, 4

    2023

  24. [24]

    Towards vqa models that can read

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8317–8326, 2019. 2, 4

    2019

  25. [25]

    Less is more: A sim- ple yet effective token reduction method for efficient multi- modal llms

    Dingjie Song, Wenjun Wang, Shunian Chen, Xidong Wang, Michael X Guan, and Benyou Wang. Less is more: A sim- ple yet effective token reduction method for efficient multi- modal llms. InProceedings of the 31st International Confer- ence on Computational Linguistics, pages 7614–7623, 2025. 1, 2, 3

    2025

  26. [26]

    Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063,

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063,

  27. [27]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024. 2

    Pith/arXiv arXiv 2024

  28. [28]

    SlotDiffusion: Object-Centric Generative Mod- eling with Diffusion Models.Advances in Neural Informa- tion Processing Systems, 36:50932–50958, 2023

    Ziyi Wu, Jingyu Hu, Wuyue Lu, Igor Gilitschenski, and Ani- mesh Garg. SlotDiffusion: Object-Centric Generative Mod- eling with Diffusion Models.Advances in Neural Informa- tion Processing Systems, 36:50932–50958, 2023. 2

    2023

  29. [29]

    Conical visual concentration for efficient large vision-language models

    Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, and Dahua Lin. Conical visual concentration for efficient large vision-language models. InProceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pages 14593–14603, 2025. 1, 2, 3, 4

    2025

  30. [30]

    Visionzip: Longer is better but not necessary in vision language models

    Senqiao Yang, Yukang Chen, Zhuotao Tian, Chengyao Wang, Jingyao Li, Bei Yu, and Jiaya Jia. Visionzip: Longer is better but not necessary in vision language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 19792–19802, 2025. 1, 2, 3, 4

    2025

  31. [31]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for ex- pert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for ex- pert agi. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9556– 9567, 2024. 4

    2024

  32. [32]

    Object-Centric Learning for Real-World Videos by Pre- dicting Temporal Feature Similarities.Advances in Neural Information Processing Systems, 36, 2024

    Andrii Zadaianchuk, Maximilian Seitzer, and Georg Mar- tius. Object-Centric Learning for Real-World Videos by Pre- dicting Temporal Feature Similarities.Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  33. [33]

    Lmms-eval: Re- ality check on the evaluation of large multimodal models

    Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, et al. Lmms-eval: Re- ality check on the evaluation of large multimodal models. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 881–916, 2025. 4

    2025

  34. [34]

    Sparsevlm: Vi- sual token sparsification for efficient vision-language model inference

    Yuan Zhang, Chun-Kai Fan, Junpeng Ma, Wenzhao Zheng, Tao Huang, Kuan Cheng, Denis A Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, et al. Sparsevlm: Vi- sual token sparsification for efficient vision-language model inference. InForty-second International Conference on Ma- chine Learning, 2025. 1, 2, 3, 4

    2025

  35. [35]

    Predicting video slot attention queries from random slot-feature pairs.arXiv preprint arXiv:2508.22772, 2025

    Rongzhen Zhao, Jian Li, Juho Kannala, and Joni Pajari- nen. Predicting video slot attention queries from random slot-feature pairs.arXiv preprint arXiv:2508.22772, 2025. 2

    arXiv 2025

  36. [36]

    Vector-Quantized Vision Foundation Model for Object-Centric Learning

    Rongzhen Zhao, Vivienne Wang, Juho Kannala, and Joni Pajarinen. Vector-Quantized Vision Foundation Model for Object-Centric Learning. InACM Multimedia, 2025. 2

    2025

  37. [37]

    Smoothing Slot Attention Iterations and Recur- rences.arXiv:2508.05417, 2025

    Rongzhen Zhao, Wenyan Yang, Juho Kannala, and Joni Pa- jarinen. Smoothing Slot Attention Iterations and Recur- rences.arXiv:2508.05417, 2025. 2

    Pith/arXiv arXiv 2025

  38. [38]

    Slot Attention with Re-Initialization and Self-Distillation

    Rongzhen Zhao, Yi Zhao, Juho Kannala, and Joni Pajarinen. Slot Attention with Re-Initialization and Self-Distillation. In ACM Multimedia, 2025. 2, 4

    2025

  39. [39]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models.arXiv preprint arXiv:2304.10592, 2023

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models.arXiv preprint arXiv:2304.10592, 2023. 1 10

    Pith/arXiv arXiv 2023

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.