{"id":"6abf1ba9-de9d-4a28-badd-0c0d4c9e3fdd","arxiv_id":"2511.20439","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"OC-VTP uses Slot Attention to select a few 'object-representative' vision tokens, reporting high accuracy retention at extreme pruning ratios, but the claimed guarantee is undercut by a non-differentiable selection and inconsistent 'best at every ratio' results.","lead":"This paper introduces OC-VTP, a vision-token pruning module that uses object-centric Slot Attention to keep a small subset of image tokens before they enter a vision-language model's language decoder, targeting cheaper inference with little accuracy loss. It reports strong accuracy retention at extreme pruning ratios, but the 'guaranteed' optimality claim is not supported by the training description and the 'best at every ratio' claim is contradicted by the paper's own result","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Training signal cannot reach the argmax-based token selector through Eq. (10), so the paper's central 'guaranteed' claim is not established; own Table 3 also contradicts 'consistently highest accuracy.'","rationale":"The reader's weakest assumption is exactly the load-bearing issue: the hard argmax in Eqs. (7)-(8) is not differentiable, and Section 3.3 describes no mechanism to train the selector through Eq. (10). This is not a minor implementation detail; it is the only route by which the Slot Attention module could learn to choose representative tokens. Without it, the 'guaranteed representative token' claim collapses, because the trained component (the random autoregressive decoder) does not influence which tokens are kept. The paper's own text provides no proof of gradient flow, and the absence of any mention of STE/Gumbel/soft-top-k in Section 3.3 makes the burden on the released code high. The secondary contradiction in Table 3 further weakens the abstract's consistency claim, but the trainability issue is more fundamental because it undermines the method's claimed novelty and guarantee. The reader's REJECT verdict therefore remains appropriate; no adjustment is needed.","tokens_in":15900,"tokens_out":4511,"duration_ms":51767,"concrete_test":"Inspect the released training code at https://github.com/GarryLarry010131/OC-VTP for any differentiable relaxation of Eq. (7) (straight-through estimator, Gumbel-top-k, or soft argmax). If none exists, instrument the training graph to record the gradient norm of the Slot Attention parameters and the sampled query parameters after one backward pass of Eq. (10). A zero gradient norm confirms that the selector is not trained and the central guarantee is unsupported. If a relaxation exists, re-run the Qwen2.5-VL 33.3% budget comparison to verify whether OC-VTP actually exceeds HiPrune as the abstract claims.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that OC-VTP provably keeps the most representative vision tokens by minimizing reconstruction error. This requires the Slot Attention parameters in Eq. (6) to be optimized by the loss in Eq. (10). But the selection is I = argmax_n(A) in Eq. (7), followed by the non-differentiable gather Vp = V[I,:] in Eq. (8). Section 3.3 describes no straight-through estimator, Gumbel relaxation, or other surrogate gradient. Consequently, the gradient of the reconstruction loss with respect to Slot Attention's parameters is zero: only the random autoregressive decoder φ_RAR is trained. The paper's Comment in Section 3.3 asserts that Eq. (10) 'ensures completeness' of the selected tokens, but with a broken gradient path this assertion is unsupported. Even if a relaxation were added, the paper provides no formal proof that greedy per-slot argmax minimizes the global reconstruction error or that minimizing reconstruction error guarantees downstream accuracy. Separately, the abstract claims OC-VTP 'consistently helps mainstream VLMs to preserve the highest inference accuracy' across any pruning ratio, yet Table 3 (Qwen2.5-VL, 33.3% retained tokens) shows OC-VTP at 98.7% average relative accuracy versus HiPrune's 98.8%, directly contradicting the headline claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces OC-VTP, a vision-token pruning method for VLMs. A Slot-Attention-based 'OC-pruner' is pre-trained on 40k COCO images and then inserted between a VLM's vision encoder and projector without fine-tuning the VLM. Given a token budget, the pruner maps vision tokens to slots, selects one token per slot via argmax attention, and prunes the rest. The pruner is trained with a reconstruction loss, including a proposed area-weighted MSE (AW-MSE), that reconstructs the original unpruned tokens from the selected tokens. The authors claim this provides a 'guaranteed' selection of the most representative vision tokens and that the method 'consistently' preserves the highest inference accuracy of mainstream VLMs at any pruning ratio. Experiments are reported on LLaVA-1.5, LLaVA-NeXT, and Qwen2.5-VL over multiple benchmarks and budgets.","tokens_in":16290,"tokens_out":7281,"duration_ms":76391,"significance":"The idea of using object-centric slot attention for token pruning is appealing, and the empirical comparison is broad, with code released. If the method worked as described, the plug-and-play, train-once property and the object-level interpretability would be useful contributions. However, the central 'guaranteed' claim is not established by the presented training formulation, and the uniformly-best accuracy claim is contradicted by one of the paper's own tables. The contribution is therefore conditional on substantial revision of both the method description and the claims.","major_comments":[{"comment":"The training objective cannot train the selector as described. Eq. (7) defines I = argmax_n(A), and Eq. (8) gathers Vp = V[I,:]. Since argmax is piecewise constant and the gathered rows are copies of the input tokens, the reconstruction loss in Eq. (10) has zero gradient with respect to the Slot Attention parameters. The text says the pruner is trained 'by reconstructing ... from the slots,' but Eq. (9) uses Vp, not S. No straight-through estimator, Gumbel relaxation, or surrogate gradient is described. Thus the paper does not establish that the selection mechanism learns anything; the Comment in §3.3 asserting that Eq. (10) 'ensures completeness' is unsupported. If training intentionally reconstructs from the slots S rather than the selected tokens Vp, the paper must say so and reconcile Eq. (9); if it truly reconstructs from Vp, the training scheme needs to be revised.","section":"§3.3, Eqs. (7)–(10)"},{"comment":"The 'guaranteed' claim is not proved. Minimizing a reconstruction loss on a training set does not imply that the greedy per-slot argmax yields the subset of size s with minimal reconstruction error on arbitrary inputs. Slot Attention's 'exclusiveness and completeness' properties are properties of the slot-attention mechanism, not formal optimality guarantees for the resulting token subset. Moreover, the target quantity in Eq. (2) is VLM accuracy, while Eq. (10) optimizes a proxy; no bound or transfer argument connects the two. The paper should either provide a precise formal statement with proof or remove 'guaranteed' and 'first guaranteed method' from the abstract and introduction.","section":"§1 and §3.3 Comment"},{"comment":"The claim that OC-VTP 'consistently helps mainstream VLMs to preserve the highest inference accuracy' across any pruning ratio is contradicted by Table 3: at 33.3% retained tokens on Qwen2.5-VL, OC-VTP's average relative accuracy is 98.7% versus HiPrune's 98.8%. Similarly, in Table 1 (LLaVA-1.5, 33.3% retained), reading the last column as the average relative accuracy, HiPrune is 109% while OC-VTP is 108%. The headline claim should be corrected to reflect the actual rankings, e.g., 'competitive' or 'best in most settings.'","section":"Abstract and Table 3"}],"minor_comments":[{"comment":"Typo: 'gauranteed' should be 'guaranteed.'","section":"Abstract"},{"comment":"'Beside, the vision tokens...' should be 'Besides, the vision tokens...'","section":"§3.4"},{"comment":"Training details are incomplete: report Slot Attention iterations, learning rate, batch size, optimizer, and the architecture/hyperparameters of the random autoregressive decoder. Currently only '40,000 images' is stated.","section":"§3.3 / §4"},{"comment":"The rightmost 'average proportion' column contains values over 100% and some missing benchmarks; clarify how averages are computed when a benchmark is unavailable for a method.","section":"Tables 1–3"},{"comment":"The notation for the ideal pruning objective is malformed; define the expectations and the indicator notation clearly.","section":"Eq. (2)"}],"recommendation":"major_revision","confidential_remarks":"The central 'guaranteed' framing is the main barrier. If the authors can clarify the training path (e.g., confirm a differentiable relaxation in the released code) and soften the optimality claims, the empirical study could be salvageable. As written, the technical description does not support the paper's headline contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper applies Slot Attention to vision token pruning, which is genuinely new, and the experimental effort is substantial. But the central “guaranteed” claim is not supported by the equations: the argmax selection in Eq. (7) is non-differentiable, and no relaxation is described, so the reconstruction loss in Eq. (10) cannot train the Slot Attention module. That is a load-bearing flaw. The paper either needs to describe a working gradient path or soften the guarantee to a heuristic.\n\nWhat is new: framing VTP as selecting tokens that minimize reconstruction error, using object-centric slots to identify representative tokens. The method is plug-and-play, requiring no VLM fine-tuning, and the FLOPs/latency numbers are useful. The ablation studies (slot count, insertion layer, AW-MSE) are thoughtful. Code is available, which increases trust.\n\nSoft spots: First, the gradient issue. If the code implements a straight-through estimator or Gumbel, the paper should say so explicitly. As written, only the autoregressive decoder appears trainable. Second, the guarantee is true by construction: you train with reconstruction error and then claim minimizing reconstruction error is the guarantee. That's circular. Third, Table 3 on Qwen2.5-VL at 33.3% retention shows OC-VTP at 98.7% vs HiPrune's 98.8%, contradicting the “consistently highest” claim in the abstract. Overclaiming like this invites scrutiny.\n\nNone of this kills the empirical contribution: the method does seem to preserve accuracy better than baselines in heavy pruning regimes. But the paper cannot be accepted with the current claims. A major revision that fixes the training description, replaces “guarantee” with “empirical”, and corrects the abstract would make it a solid practical paper.\n\nThe paper deserves peer review — the idea is interesting and the experiments are extensive. But I'd recommend reject in current form, or at least a strong request for revision.","headline":"New object-centric pruning idea, solid experiments, but the guarantee claim doesn't hold up and the training path is unclear.","tokens_in":16732,"tokens_out":2970,"would_cite":false,"duration_ms":33007,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Vision token pruning becomes a reconstruction problem, with a guarantee of keeping the most representative tokens.","keywords":["vision token pruning","vision language models","object-centric learning","slot attention","reconstruction-based selection","inference acceleration","token efficiency","interpretability"],"falsifier":"Measure the reconstruction loss (with the paper's own decoder and AW-MSE) on the token subsets selected by the trained OC-pruner versus equally sized random subsets on held-out images; if the selected subsets do not achieve notably lower reconstruction error than random, the training signal is not reaching the selection mechanism. Alternatively, fine-tune the same architecture with a differentiable relaxation of selection (e.g., Gumbel-top-k) and show it significantly outperforms the published argmax version, which would demonstrate that the hard selection is not being optimized.","tokens_in":15822,"feed_emoji":"🎯","tokens_out":4936,"duration_ms":48621,"temperature":0.7,"pith_summary":"This paper claims that vision token pruning for vision-language models can be turned from a heuristic art into a problem with an optimality guarantee. It proposes OC-VTP, which uses object-centric learning (Slot Attention) to group tokens into object-level slots, keeps the single most-attended token per slot, and trains a lightweight decoder to reconstruct the full token set from that kept subset. The authors argue that minimizing this reconstruction error is a direct way to keep the most representative tokens, and that the resulting pruner, pretrained once on generic images, plugs into existing VLMs without any fine-tuning. Across three VLM families and pruning ratios from 33% down to about 5% of tokens, it reports the highest retained accuracy compared with all prior attention- and similarity-based methods, with substantial FLOP and latency savings.","feed_headline":"Object-centric pruner keeps most representative vision tokens, provably","feed_subtitle":"A lightweight pretrained module drops into existing VLMs, cutting compute up to 17× without fine-tuning.","key_machinery":"The OC-pruner: a Slot Attention module (a learned clustering that groups input tokens into a fixed number of object-level slots through competitive attention) producing an attention map over vision tokens; an argmax over that map selecting one token per slot as the kept token; and a random autoregressive Transformer decoder reconstructing the original sequence from those kept tokens. Training uses an Area-Weighted Mean-Squared Error, re-weighting each slot's reconstruction error by the inverse of its mask area so small informative regions are preserved. The claimed guarantee combines Slot Attention's exclusiveness (competition makes slots cover distinct objects) with the completeness enforce","core_discovery":"The paper's central claim: the token subset that best reconstructs the original unpruned tokens is the most representative subset, and it can be found by a Slot Attention-based selector trained through reconstruction. The OC-pruner aggregates tokens into object slots equal to the budget, picks the argmax-attention token per slot, and reconstructs the full sequence with a random autoregressive Transformer; training minimizes an area-weighted MSE that up-weights small informative regions. The authors state this is the first guaranteed VTP method, that it beats FastV, SparseVLM, VisionZip, PyramidDrop, and HiPrune at every pruning ratio on LLaVA-1.5, LLaVA-NeXT, and Qwen2.5-VL, and that retaine","pith_inferences":["The strength of the 'guarantee' depends on how faithfully the random autoregressive decoder approximates the true information content of the original tokens; with a weak decoder, low reconstruction error may not imply high task-relevant information, so the guarantee is likely looser than stated.","The published training description does not show how the non-differentiable argmax selection is optimized; if gradients only flow through the decoder and slot attention's soft assignment, the hard selection indices may not be the ones learned, and the reported gains could stem largely from the slot-grouping itself rather than the final selection.","A natural test: replace the hard argmax with a Gumbel-softmax or straight-through estimator and compare reconstruction loss and downstream accuracy; a large gap would indicate that selection is not actually trained in the current formulation.","The weaker results on Qwen2.5-VL (which uses dynamic-resolution token counts) suggest the 'any budget, train once' claim may be specific to fixed-token-count encoders; variable token counts break the learned mapping from budget to number of slots."],"forward_implications":["If correct, this is the first provably reconstruction-optimal vision token pruner, replacing handcrafted attention/similarity heuristics with an optimized selection criterion.","A single pretrained OC-pruner transfers across VLM architectures without fine-tuning, so token efficiency can be added to existing models at near-zero deployment cost.","The 5–17x FLOP reduction at high pruning ratios, with comparable latency gains, could make high-resolution or multi-image VLM inference practical on resource-limited hardware.","The object-level interpretability of the kept tokens opens the door to explainable or controllable VLM inference, where users can see which objects the model is attending to."],"fun_headline_variants":["Guaranteed vision token pruning via reconstruction error minimization","OC-VTP: Prune vision tokens with a guaranteed reconstruction bound","Object-centric pruner keeps the most informative vision tokens","Reconstruction-based vision token pruning for efficient VLMs","Guaranteed representative token selection for VLM pruning"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The entire guarantee rests on the assumption that the hard argmax token-selection step is effectively trained by the reconstruction loss, even though argmax is non-differentiable and the paper does not describe any surrogate gradient or straight-through estimator—if that assumption fails, the selection is never actually optimized, and the 'most representative' claim collapses.","fun_headline_variants_meta":{"raw":{"variants":["Guaranteed vision token pruning via reconstruction error minimization","OC-VTP: Prune vision tokens with a guaranteed reconstruction bound","Object-centric pruner keeps the most informative vision tokens","Reconstruction-based vision token pruning for efficient VLMs","Guaranteed representative token selection for VLM pruning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00076,"raw_usage":{"total_tokens":3204,"prompt_tokens":729,"completion_tokens":2475,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":473,"completion_tokens_details":{"reasoning_tokens":2406}},"tokens_in":473,"tokens_out":2475,"duration_ms":17063,"temperature":1.0,"reasoning_tokens":2406,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T20:14:53.243066+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the reconstruction loss (with the paper's own decoder and AW-MSE) on the token subsets selected by the trained OC-pruner versus equally sized random subsets on held-out images; if the selected subsets do not achieve notably lower reconstruction error than random, the training signal is not reaching the selection mechanism. Alternatively, fine-tune the same architecture with a differentiable relaxation of selection (e.g., Gumbel-top-k) and show it significantly outperforms the published argmax version, which would demonstrate that the hard selection is not being optimized.","supporting_citations":[],"review_version":1}