{"id":"02fda842-9182-4361-a2a0-6d758d97640d","arxiv_id":"2506.13166","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A training-free token pruning method that combines cosine-similarity saliency with greedy redundancy removal to preserve accuracy at high compression ratios.","lead":"GreedyPrune is a plug-in method that speeds up vision-language models by cutting most visual tokens while keeping the important ones. It ranks tokens by how similar they are to the text prompt, then greedily removes similar ones, and reports high accuracy even when 89% of tokens are dropped.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline 94.98% is not recoverable from Table 1's row values and τ is changed across tables for identical settings, so the SOTA margin is currently unverifiable.","rationale":"I read the paper as claiming a fixed, training-free, plug-and-play method whose accuracy gains are robust across models and pruning ratios. For that claim to hold, the reported gains must not be artifacts of per-benchmark threshold selection or arithmetic errors. The manuscript's own tables currently fail this check: the same 64-token LLaVA-1.5-7B configuration appears with τ=0.94 in Table 1 and τ=0.78 in Table 4 while both report Avg 94.22%, and the LLaVA-1.5-13B row's listed per-dataset values imply a mean retention ratio near 95.9%, not 94.98%. I am not alleging intent; undefined averaging, table-copy errors, or test-set threshold tuning would produce the same symptoms. The concrete reproducibility check I propose would settle whether the headline numbers are trustworthy. This concern reinforces the reader's conditional verdict rather than overturning it, so I recommend no change to the verdict.","tokens_in":16627,"tokens_out":15652,"duration_ms":153880,"concrete_test":"Recompute every Avg column from the raw per-dataset evaluation logs using the paper's implied normalization (each metric divided by the corresponding Original row value, then averaged over the 9 datasets). Verify whether Table 1's Ours(τ=0.94) and Table 4's Ours(τ=0.78) at 64 retained tokens on LLaVA-1.5-7B both yield 94.22%, and whether the LLaVA-1.5-13B row yields 94.98% or approximately 95.9%. If the rows do not reproduce, request the exact τ-selection protocol and raw logs; the SOTA margin is not currently supported by the published tables.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim rests on Table 1's averaged retention ratios, but the paper never defines the averaging formula, and the tables are internally inconsistent. For LLaVA-1.5-7B at 64 retained tokens, Table 1 reports Ours(τ=0.94) with Avg 94.22%, while Appendix Table 4 reports Ours(τ=0.78) for the same 64-token setting with the same claimed Avg 94.22%, despite different per-dataset scores. For LLaVA-1.5-13B at 64 tokens, the row Ours(τ=0.78) lists per-dataset values whose mean retention ratio, computed by the normalization implied by the Original row's 100%, is about 95.9% rather than the reported 94.98%. Thus the claimed 4.08% margin over DivPrune cannot be checked from the published tables. Figure 5 shows strong sensitivity to τ, so the fact that different τ values are used for identical settings across tables raises the possibility that results were selected per benchmark rather than produced by a single fixed method. No code or raw evaluation logs are provided to resolve this.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GreedyPrune, a training-free visual token pruning method for large vision-language models (LVLMs). It formulates critical visual token selection as a combinatorial optimization problem: maximize total semantic saliency (measured by cosine similarity between the last text token and each visual token) subject to pairwise cosine-similarity constraints among selected tokens, and solves it with a greedy pivot-token selection. Experiments on LLaVA-1.5-7B/13B, LLaVA-1.6-7B, and Qwen2-VL across nine benchmarks claim state-of-the-art accuracy at high pruning ratios (e.g., 94.98% average at 88.9% pruning on LLaVA-1.5-13B, 4.08% above the second-best method) with reduced end-to-end latency.","tokens_in":16878,"tokens_out":7126,"duration_ms":58695,"significance":"If the empirical claims hold, GreedyPrune would be a practically useful training-free, plug-and-play acceleration method for LVLMs, with a simple and interpretable selection criterion that is compatible with Flash Attention. The paper's strengths include a clear problem formulation, extensive evaluation across multiple model families and nine benchmarks, ablations isolating the saliency metric and the redundancy-removal component, and qualitative visualizations. However, the central quantitative claims are currently undermined by internal inconsistencies in the reported tables and by an unspecified protocol for the threshold τ, so the significance cannot be fully assessed until these issues are resolved.","major_comments":[{"comment":"The main result for LLaVA-1.5-7B with 64 retained tokens is internally inconsistent. Table 1 reports Ours(τ=0.94) with Avg 94.22% and per-dataset scores (MME 1442, POPE 84.4, MMB 61.3, Ocrbench 286, TextVQA 42, OK-VQA 49.2, Nocaps 98.4, Flickr30K 69.5, GQA 60.4). Table 4 reports Ours(τ=0.78) for the same retained-token count with the same Avg 94.22% but different per-dataset scores (MME 1427, POPE 84.5, MMB 61.0, Ocrbench 284, TextVQA 41.9, OK-VQA 49.6, Nocaps 98.4, Flickr30K 69.7, GQA 60.2). Table 2's GreedyPrune row matches the Table 4 numbers, not the Table 1 numbers. This means the reported Avg cannot be reproduced from the displayed values, and the reader cannot tell which τ is the actual configuration for the headline result.","section":"Tables 1, 2, and 4"},{"comment":"The claimed Avg of 94.98% is not recoverable from the per-dataset values. Using the same normalization implied by the Original row (each per-dataset score divided by the Original score, then averaged), the Ours(τ=0.78) row yields about 95.9%, and the ratio-of-averages method yields 98.1%; neither matches 94.98%. The stated 4.08% margin over DivPrune therefore cannot be verified from the published table.","section":"Table 1, LLaVA-1.5-13B row"},{"comment":"The threshold τ is a free parameter with strong effect on accuracy (the ablation shows POPE peaking at 0.78 and MME still rising at 0.93), yet the paper does not specify how τ is selected for each setting. This is compounded by the fact that identical settings use different τ values in different tables (τ=0.94 in Table 1 vs τ=0.78 in Table 4 for LLaVA-1.5-7B at 64 tokens). Without a defined selection protocol (e.g., a validation split) or a sensitivity analysis, the state-of-the-art comparisons may reflect per-benchmark tuning rather than a single plug-and-play method.","section":"Section 3.3 and Figure 5"},{"comment":"The paper claims that greedy algorithms 'can efficiently achieve near-optimal solutions' to the formulated NP-hard UBQP problem, but no approximation bound, analysis, or empirical optimality-gap study is provided. The text only states that heuristics provide approximate solutions. This is an overstatement of the theoretical support and should be revised or substantiated with a concrete bound for the greedy policy.","section":"Section 3.2 and contributions"},{"comment":"The diversity constraint is written as cos(v_i, v_j) ≤ τ for all v_i, v_j ∈ V, but it should apply to the selected subset S; as written, the constraint is independent of the choice of S. The correct constraint appears only implicitly in Eq. (2) via the z_i z_j product. This is a formal error in the problem statement that should be corrected.","section":"Equation (1)"}],"minor_comments":[{"comment":"The title contains a typo: 'Retenting' should be 'Retaining'; similarly, 'tranning-free' in Section 1 and 'Reasearch' in the affiliations should be corrected.","section":"Title, Section 1"},{"comment":"The text refers to 'Figure 3' for the τ ablation, but the ablation is shown in Figure 5; the algorithm overview is Figure 3. Please fix the cross-references.","section":"Section 4.3"},{"comment":"The notation for the critical token set alternates between V_critic and Vcritic; please unify the notation throughout the paper.","section":"Section 3.3"},{"comment":"The paper would benefit from reporting variance or multiple runs for the key tables; no error bars or seeds are provided for the benchmark results.","section":"Tables 1–4"},{"comment":"The terminology for pruning ratios is inconsistent: 88.9% pruning corresponds to 11.1% retained, while Table 3 lists 'Retained token ratio' values of 33.3%, 22.2%, and 11.1%. Please ensure consistent language throughout.","section":"Section 4.2 and Table 3"}],"recommendation":"major_revision","confidential_remarks":"The inconsistencies in Tables 1, 2, and 4 are serious enough that I would want to see the authors' raw evaluation logs or a released implementation before accepting the quantitative claims. Please also ask whether τ was chosen per benchmark or fixed; if per-benchmark, the comparison should be presented as an upper envelope, not as a single method."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead GreedyPrune. The core idea is a sensible combination: use cosine similarity to the last text token as a saliency score, then greedily pick the highest-saliency token and drop all remaining tokens whose cosine similarity to it exceeds tau. The saliency metric is a legitimate alternative to cross-attention; their removal-impact diagnostic in Figure 2a is a good way to defend it. The greedy algorithm is clearly described and the experiments span several models and datasets, including Qwen2-VL. The paper is also honest about the limitations of existing attention- and diversity-based methods.\n\nThat said, the empirical headline is not verifiable from the published tables. For LLaVA-1.5-13B at 64 retained tokens, the per-dataset retention ratios in Table 1 average to about 95.9%, not the reported 94.98%. For LLaVA-1.5-7B at 64 tokens, Table 1 shows Ours(tau=0.94) with Avg 94.22%, while the appendix Table 4 lists Ours(tau=0.78) for the same 64-token setting with the same Avg 94.22% but different per-dataset scores. The same Avg from different tau and different numbers is not a rounding issue; it's an inconsistency. And Figure 5 shows that accuracy is quite sensitive to tau, so the fact that tau changes across tables and settings for identical pruning ratios raises a real concern that thresholds were selected post hoc per benchmark. No code or raw logs are supplied to resolve this.\n\nThe theoretical section is decorative: the UBQP formulation and Lagrange multipliers are not actually used by the algorithm, and the 'near-optimal' phrasing is not backed by a proof. That part could be cut without damage. Also, no error bars or variance across runs are reported anywhere, which is standard for this kind of latency-accuracy comparison.\n\nOn the positive side, the method is simple enough to reproduce, and the ablation separating the saliency metric from the diversity mechanism is well designed. The presentation has many typos ('tranning-free', 'Retenting', 'saliancy'), but nothing that obscures the algorithm.\n\nBottom line: this deserves a serious referee, but only if the authors can reconcile the tables and provide a fixed evaluation protocol for tau. Right now the SOTA claim should not be taken at face value. If the inconsistencies are resolved, this is a solid incremental contribution to efficient LVLMs. As it stands, I'd treat the quantitative margins with suspicion.\n\nBest.","headline":"The greedy saliency-plus-diversity idea is reasonable, but the reported SOTA margins don't survive arithmetic on the tables and the tau tuning looks per-benchmark.","tokens_in":17316,"tokens_out":3245,"would_cite":false,"duration_ms":29457,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GreedyPrune claims that visual token pruning should be treated as selecting a critical token set that maximizes semantic saliency under a pairwise diversity constraint, and that a greedy algorithm can do this well enough to preserve…","keywords":["visual token pruning","large vision-language models","greedy algorithm","semantic saliency","visual diversity","cosine similarity","training-free inference acceleration","token redundancy"],"falsifier":"Run GreedyPrune on a dataset where the prompt contains no image-relevant words, such as a generic 'Describe this image' instruction; if removing 88.9% of tokens on that prompt drops accuracy to or below random token selection, then the cosine-similarity saliency ranking, rather than the greedy diversity step, is not doing the claimed work.","tokens_in":16472,"feed_emoji":"⚡","tokens_out":8439,"duration_ms":79288,"temperature":0.7,"pith_summary":"GreedyPrune is a training-free visual token pruning method for large vision-language models (LVLMs). The paper argues that token pruning should be reformulated as selecting a 'critical visual token set' that keeps both semantic importance and visual diversity, and that a greedy algorithm can approximate this combinatorial problem well enough to preserve accuracy. The paper claims that its cosine-similarity saliency metric beats attention-score-based selection and that greedy redundancy removal keeps models accurate even at an 88.9% token reduction. On nine image benchmarks, GreedyPrune reports 94.22% average accuracy for LLaVA-1.5-7B and 94.98% for LLaVA-1.5-13B at that compression, ahead of the compared baselines. This matters because visual token processing dominates LVLM inference cost, especially on resource-constrained devices.","feed_headline":"Greedy token pruning holds LVLM accuracy at 89% compression","feed_subtitle":"GreedyPrune scores image patches against the prompt and drops clones, staying accurate under heavy pruning.","key_machinery":"The central machinery is a two-step greedy selection procedure. First, each visual token is scored by its cosine similarity to the last text token of the prompt, a semantic saliency metric the paper argues is less biased than cross-attention scores. Second, tokens are processed in descending saliency order: the top remaining token becomes a pivot and is added to the critical set, and all remaining tokens whose pairwise cosine similarity with the pivot exceeds a threshold $\\tau$ are discarded as redundant. The paper formulates the underlying objective as an unconstrained binary quadratic program, notes that exact solution is NP-hard, and uses the greedy rule as a tractable approximation that satisfies the constraint step by step. The threshold $\\tau$ controls the saliency-diversity trade-off, with lower $\\tau$ demanding stronger diversity and higher $\\tau$ allowing more high-saliency picks.","core_discovery":"The paper's central claim is that the key to training-free visual token pruning is to solve a critical-set selection problem: pick a fixed-size subset of visual tokens that maximizes total semantic saliency while keeping pairwise cosine similarity below a threshold, so the retained tokens are not redundant. The authors propose a specific saliency metric, the cosine similarity between the last text token and each visual token, arguing it captures semantics without the positional-attention bias of cross-attention scores and is compatible with FlashAttention. They then select tokens greedily: sort visual tokens by this saliency, take the highest-scoring remaining token as a pivot, remove all remaining tokens whose cosine similarity to the pivot exceeds the threshold tau, and repeat until the target subset size is reached. The paper demonstrates through ablation that both components matter, and reports that high pruning ratios, such as retaining only 64 of 576 tokens, keep near-full accuracy: 94.22% average on LLaVA-1.5-7B and 94.98% on LLaVA-1.5-13B across nine datasets, outperforming the second-best compared method by 3.82 and 4.08 points respectively.","pith_inferences":["The paper leaves open whether the cosine-similarity saliency metric remains reliable when the prompt is very short or does not name the image content; a natural extension would be to blend it with other signals, such as CLIP score or attention, only when the text is uninformative.","The greedy procedure requires pairwise comparisons between the pivot and all remaining tokens, an overhead that might matter at low pruning ratios; a batched or approximative similarity computation could extend the method to real-time settings.","The same critical-set formulation could be applied to other redundant sequences, such as KV caches or long context windows, by swapping the saliency metric while keeping the greedy diversity constraint.","Because $\\tau$ needs tuning per dataset, an adaptive threshold chosen per input or per image, rather than a global constant, is a testable improvement implied by the paper's own ablation."],"forward_implications":["If GreedyPrune works as claimed, LVLMs can run with about one ninth of their visual tokens while retaining most benchmark accuracy, making large models more practical on memory- and latency-limited hardware.","Attention-based pruning can be replaced by a cosine-similarity metric that avoids the bottom-of-image attention bias and remains compatible with FlashAttention.","The same plug-and-play procedure transfers across model families, including LLaVA-1.5, LLaVA-NeXT, and Qwen2-VL, without fine-tuning or calibration.","At equal pruning ratios and equal TFLOPS budgets, the reported results place GreedyPrune ahead of FastV, DART, and DivPrune on the nine evaluated benchmarks.","The optimal threshold $\\tau$ varies by dataset, so tuning it per deployment or per task is a direct way to trade diversity against semantic saliency."],"supporting_citations":[{"why":"FastV: supplies the shallow-layer, last-text-token cross-attention pruning baseline that GreedyPrune builds on and compares against.","marker":"Chen et al. 2024b"},{"why":"FasterVLM: documents attention bias in cross-attention selection criteria, motivating the alternative cosine-similarity saliency metric.","marker":"Zhang et al. 2024b"},{"why":"Shows that cross-attention-guided visual token selection can underperform random selection, the bias claim GreedyPrune is designed to avoid.","marker":"Endo, Wang, and Yeung-Levy 2024"},{"why":"Provides evidence that global image information propagates into the text token in shallow LLM layers, the basis for scoring visual tokens by similarity to the last text token.","marker":"Zhang et al. 2024c"},{"why":"DART: diversity-based baseline that removes redundancy by cosine similarity and is a main comparison point.","marker":"Wen et al. 2025b"},{"why":"DivPrune: diversity-based pruning baseline using Min-Max-Diversity, another main comparison point.","marker":"Alvar et al. 2025"},{"why":"Survey of unconstrained binary quadratic programming used to classify the critical-set selection problem as NP-hard.","marker":"Kochenberger et al. 2014"},{"why":"RoFormer: identifies the rotary position embedding property cited as the cause of attention bias in earlier pruning methods.","marker":"Su et al. 2023"}],"fun_headline_variants":["GreedyPrune selects critical visual tokens to keep accuracy at 89% compression","Greedy token pruning balances saliency and diversity for LVLM efficiency","GreedyPrune holds LVLM accuracy under heavy visual token pruning","Greedy critical-set selection retains accuracy with fewer visual tokens"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the cosine similarity between the prompt's last text token and a visual token is a trustworthy measure of that token's semantic importance, and that deleting tokens similar to already-selected tokens never removes information the model still needs.","fun_headline_variants_meta":{"raw":{"variants":["GreedyPrune selects critical visual tokens to keep accuracy at 89% compression","Greedy token pruning balances saliency and diversity for LVLM efficiency","GreedyPrune holds LVLM accuracy under heavy visual token pruning","Greedy critical-set selection retains accuracy with fewer visual tokens"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000181,"raw_usage":{"total_tokens":1319,"prompt_tokens":967,"completion_tokens":352,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":583,"completion_tokens_details":{"reasoning_tokens":272}},"tokens_in":583,"tokens_out":352,"duration_ms":4257,"temperature":1.0,"reasoning_tokens":272,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:36:12.410721+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run GreedyPrune on a dataset where the prompt contains no image-relevant words, such as a generic 'Describe this image' instruction; if removing 88.9% of tokens on that prompt drops accuracy to or below random token selection, then the cosine-similarity saliency ranking, rather than the greedy diversity step, is not doing the claimed work.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Survey of unconstrained binary quadratic programming used to classify the critical-set selection problem as NP-hard."}],"review_version":1}