{"id":"c0781af3-597a-41dd-8620-6833a1ff2006","arxiv_id":"2412.03248","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"AIM reduces multi-modal LLM inference cost by merging similar visual tokens and progressively pruning visual tokens based on attention importance, cutting FLOPs about 7x with near-parity accuracy and improving long-video understanding when more frames are used.","lead":"Researchers present a training-free way to speed up image and video language models by merging similar image tokens before the model and progressively dropping less important tokens inside it. The method cuts compute by roughly 7x with little accuracy loss, and at the same compute lets the model use many more video frames.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 1 is not a valid importance ranking: with row-stochastic softmax attention, s = (1/(N+M)) A s admits only uniform or zero solutions, so the described pruning mechanism cannot differentiate tokens and the actual implementation must differ from the paper.","rationale":"The central claim rests on the ability of Eq. 1 to rank which visual tokens are expendable. The reader's weakest assumption questioned whether PageRank scores are reliable as an importance oracle; my concern is more fundamental: the published equation, given row-stochastic softmax attention, is mathematically incapable of producing any non-uniform ranking. If correct, this means the paper's described mechanism cannot be what produces the reported results, and the actual algorithm must differ in an unspecified way. This is a concrete, checkable defect in the core method description, not merely a lack of theoretical guarantee. The paper provides open code and extensive empirical evaluations, so the concern is addressable by verifying the implementation and correcting the equation. Since the reader already issued a CONDITIONAL verdict, my finding does not change the verdict; it sharpens the condition: the authors must clarify the exact importance formula used and confirm it matches the code. The concern does not allege any misconduct; it identifies a likely typographical or notation error that is nonetheless load-bearing for reproducibility.","tokens_in":19876,"tokens_out":13692,"duration_ms":125115,"concrete_test":"Inspect the released code at https://github.com/LaVi-Lab/AIM and locate the token-pruning importance computation. Verify whether the update rule matches Eq. 1 or a standard PageRank (e.g., s <- d * A^T s + (1-d)/K * 1, with d=0.85). Independently, implement Eq. 1 as written with uniform initialization on a real attention matrix from LLaVA-OneVision and run power iterations; if the scores remain constant or decay, the described mechanism cannot rank tokens, confirming the paper's Eq. 1 is incorrect. Report which formula reproduces the ablations in Table 5.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 3.3, the paper defines the importance score via Eq. 1: s_i = (1/(N^l+M^l)) sum_j A^l_{i,j} s^l_j. Since A^l is softmax-normalized, each row sums to 1 (row-stochastic). The equation is therefore s = (1/K) A s, which implies A s = K s. Because the spectral radius of a stochastic matrix is 1 and K = N^l + M^l is at least 2 in any realistic setting, the only solution is s = 0. Even a single power iteration from a uniform start yields s_i = 1/K for every token, and repeated iterations decay to zero; no non-uniform ranking can emerge. This is not a question of whether PageRank is a reliable importance oracle: the equation as written cannot rank tokens at all. The paper must be using a different formula, likely a transpose or a damped PageRank variant. If the released code implements Eq. 1 literally, the pruning is effectively random and the reported accuracy-efficiency curves (Tables 1, 5, 8) would not be reproducible. If the code uses a corrected formula, then Eq. 1 is a material mis-specification of the core mechanism, preventing readers from reconstructing the method from the paper.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces AIM, a training-free adaptive inference method for multimodal LLMs. Visual tokens are first merged by cosine similarity before entering the LLM, then progressively pruned inside each LLM layer according to a PageRank-style importance score derived from attention weights, with a piecewise-linear retention scheduler. The method is evaluated on LLaVA-OneVision-7B for video and LLaVA-1.5-7B for image benchmarks, reporting up to 6.8x/8.0x FLOPs/prefill-time reductions with minimal accuracy loss, and a +4.6 MLVU improvement when the saved compute is re-spent on more sampled frames.","tokens_in":20201,"tokens_out":6338,"duration_ms":57812,"significance":"If the implementation matches a corrected version of the scoring formula, the work is practically relevant: it is training-free, applies to both video and image LLMs, and includes a broad comparison against FastV, VTW, PDrop, and LLaVA-Prumerge. Strengths include public code, detailed ablations on merging ratios and pruning schedulers (Tables 4, 5, A, B), and the clear demonstration that re-spending FLOPs on additional frames improves long-video understanding (Table 2). The main limitation is that the core scoring equation, as printed, is mathematically degenerate, so the method cannot be reconstructed from the paper until that equation is corrected and the selection of default hyperparameters is addressed.","major_comments":[{"comment":"Equation (1) cannot produce a token ranking as written. Since A^l is softmax-normalized, each row sums to 1, and the equation is s = (1/(N^l+M^l)) A s. A nonzero solution would be an eigenvector of the row-stochastic matrix A^l with eigenvalue N^l+M^l >= 2, contradicting the Perron-Frobenius bound that the spectral radius is 1. Starting from the uniform initialization stated in the text, one power iteration leaves every score at 1/(N^l+M^l), and further iterations decay to zero. Thus Eq. (1) either is not the rule used in the experiments (in which case the paper mis-specifies its core mechanism) or the pruning is effectively random (in which case the reported accuracy-efficiency curves are not reproducible). Please rewrite Eq. (1) with the actual PageRank variant, e.g., a damped iteration with a teleport term or a transpose with a damping factor, and state the initialization and iteration count explicitly.","section":"§3.3, Eq. (1)"},{"comment":"The default configuration (merging retention 25%, l1=14, l2=22 for video) was selected using the VideoMME ablations in Table 5, and the same benchmark is then used for the headline \"no performance drop\" result in Table 1. This makes the VideoMME row of Table 1 an in-sample selection result. Please either hold out a validation split, report hyperparameter selection on a development set separate from the reported benchmark, or explicitly frame VideoMME as the tuning benchmark and base the main claims on the other benchmarks (MVBench, MLVU, EgoSchema, NextQA, PerceptionTest), which are out-of-sample and do show small losses.","section":"§4, Implementation Details and Table 5"},{"comment":"Prefill time is reported as computed with the LLM-Viewer library rather than measured on hardware. Since the \"8.0x prefill time\" reduction is a headline claim, please provide wall-clock prefill measurements for at least the base model and the default configuration on a reference GPU, or clearly label the reported values as roofline estimates and validate them on a subset. The same applies to the FLOPs numbers, which can be estimated accurately but are not actual runtime measurements.","section":"§4, Implementation Details and Tables 1–3"}],"minor_comments":[{"comment":"The sentence \"the number of visual tokens input to the next layer is N^1 × rl\" should presumably read N^l × rl; please correct the superscript.","section":"§3.3, text after Eq. (2)"},{"comment":"The heading \"Addtional Ablation Study\" should be \"Additional Ablation Study\".","section":"Appendix A"},{"comment":"The caption states that \"with less computation cost, our method outperforms baselines on most benchmarks,\" but there are two operating points for Ours with different FLOPs; please specify which operating point is being compared.","section":"Table 3, caption"},{"comment":"Please clarify that Exp. 1 (l1=28, l2=29) is equivalent to no pruning under the scheduler in Eq. (2), since r_l = 1 for all layers l <= 28; naming this condition explicitly would help readers interpret the ablation.","section":"Table 5"},{"comment":"The statement that the method is \"not compatible\" with FlashAttention is clearer if expanded: token pruning is incompatible with FlashAttention because attention probabilities are not materialized, while token merging alone is compatible, as Table D shows.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The mathematical issue in Eq. (1) is serious because it concerns the paper's core pruning mechanism. I recommend major revision rather than rejection because the empirical design is otherwise sound, the ablation results are internally consistent, and the code is public, so the authors can correct the equation and, if necessary, verify from the released implementation that the ranking is not uniform. The editor may wish to ask the authors to confirm that the released code does not implement Eq. (1) literally."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is practical and well-executed: merge similar visual tokens before the LLM and prune them progressively inside the LLM, with a piecewise schedule that preserves early cross-modal fusion and cuts late-layer redundancy. The long-video experiment is the most interesting result — spending the saved compute on more frames beats the 32-frame base by +4.6 on MLVU at matched FLOPs. That is a real, non-obvious payoff, and the ablation study is thorough enough that the main qualitative claims (early layers matter, text tokens should not be pruned) are credible. Credit is also due for measuring overhead, comparing against four baselines, and releasing code.\n\nNow the soft spots, in proportion.\n\nFirst, the stress-test is right. Eq. 1 as printed is s = (1/K) A s with A row-stochastic; the only fixed point is zero, and power iteration from uniform decays to a constant vector. It cannot rank tokens. The paper almost certainly means the transpose (with or without a damping factor), as in the Zero-TPRune work it cites. This is a material mis-specification, not a cosmetic typo: a reader cannot reconstruct the pruning mechanism from the paper, and the reported accuracy-efficiency curves would not be reproducible from the equations alone. The code needs to be checked, and the formula fixed.\n\nSecond, the headline VideoMME trade-off is partly in-sample: the merging ratio and (l1, l2) were selected on VideoMME ablations and then reported on VideoMME. The method still shows gains on other video benchmarks, so this is a caveat, not a knockout, but the cleanest numbers are the most selection-affected.\n\nThird, FLOPs and prefill time come from LLM-Viewer with assumed text-token counts, not wall-clock measurements. That is fine for relative comparison across methods, but the \"7-fold\" reduction is a simulator estimate.\n\nFourth, the abstract says \"preserving performance\" for image LLMs, but Table 3 shows a clear drop on TextVQA and several other benchmarks at the 12.5% configuration. The word \"minimum\" would be fair; \"preserving\" overstates it.\n\nNone of this sinks the method. The empirical evidence for the central claim is strong enough that the paper deserves a careful referee, and the flaws are addressable. The one requirement for acceptance is correcting Eq. 1 and confirming that the released code matches the corrected formula. I would send this to review; I would also bring it to a reading group, because the long-video result and the layer-behavior analysis generate good discussion.","headline":"A genuinely useful efficiency method with strong empirical results, but the PageRank equation as printed is degenerate — likely a typo — and the headline numbers carry selection and simulator caveats.","tokens_in":20705,"tokens_out":2528,"would_cite":true,"duration_ms":26252,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training-free adapter that merges similar visual tokens before the LLM and progressively prunes attention-unimportant ones inside LLM layers can cut computation by roughly 7x with near-unchanged accuracy, and re-spending the savings on…","keywords":["adaptive inference","token merging","token pruning","multi-modal LLM","visual token redundancy","PageRank importance","long video understanding","training-free acceleration"],"falsifier":"Run the same merging and pruning schedule on the same base model but choose the visual tokens to drop uniformly at random within each layer; if the random version matches the reported VideoMME score, the attention-based importance ranking is not doing the work, and the paper's mechanism claim would be falsified.","tokens_in":19684,"feed_emoji":"⚡","tokens_out":6215,"duration_ms":52426,"temperature":0.7,"pith_summary":"The paper claims that a training-free adapter can make pre-trained multimodal LLMs much cheaper to run by removing redundant visual tokens in two steps: merging visually similar tokens before they enter the LLM, then progressively dropping the visual tokens that attention-based PageRank scores mark as least important at each LLM layer. On LLaVA-OneVision-7B this reduces FLOPs by a factor of 6.8 and prefill time by 8.0 while keeping video-benchmark accuracy nearly unchanged, and on LLaVA-1.5-7B it reduces FLOPs 3.7x with only a manageable image-benchmark loss. The same budget can then be re-spent: at matched compute, sampling 192 frames instead of 32 raises long-video accuracy by +4.6 on MLVU. If the paper is right, efficiency gains come from exploiting redundancy already present in the frozen model, no retraining is needed, and the saved budget can buy denser temporal coverage.","feed_headline":"Token pruning cuts multimodal LLM costs 7x without retraining","feed_subtitle":"A merging-plus-PageRank adapter keeps accuracy and lets long-video models see 6x more frames","key_machinery":"The load-bearing object is the PageRank importance score of Eq. (1): each token's score is a sum of its incoming attention weights weighted by the scores of the tokens that attend to it, making the attention matrix an adjacency graph over all LLM tokens. The other two components are iterative token merging by cosine similarity before the LLM, and the piecewise scheduler of Eq. (2) with parameters $\\ell^1$ and $\\ell^2$ that decides which LLM layers start and stop visual-token pruning. Together they turn a frozen multimodal LLM into a configurable accuracy-efficiency device whose operating point is set by a retention ratio and two layer indices.","core_discovery":"The central claim is that most visual tokens in a multimodal LLM are redundant at inference time, so a carefully scheduled combination of pre-LLM merging and in-LLM pruning can cut computation by an order of magnitude without retraining. The method merges adjacent visual tokens by cosine similarity before the LLM, then at each layer computes a PageRank-style importance score from the softmax attention matrix and prunes only visual tokens, leaving text tokens untouched. A piecewise scheduler keeps all visual tokens in early layers, prunes them gradually in middle layers, and removes them in later layers, reflecting the paper's finding that early layers perform cross-modal fusion while later layers focus on text reasoning. The authors support this with video and image benchmarks showing near-base accuracy at large compute reductions, and with an adaptive table showing a 40-fold FLOPs range at under 13% VideoMME drop.","pith_inferences":["A testable extension is to replace the PageRank-based token choice with random visual-token pruning under the same merging and layer schedule; if random pruning matches accuracy, the attention ranking is not the source of the gains.","Because the scheduler and merging are modality-agnostic, the same adapter may transfer to audio or other non-text token streams in multimodal LLMs, though the paper only evaluates vision.","The incompatibility with FlashAttention noted by the paper suggests a natural next step: computing approximate token importance without materializing the full attention matrix, which would combine token reduction with efficient attention kernels.","The frame-budget result implies that for long-video tasks the optimal operating point may involve many more frames than current 32-frame training and evaluation protocols use; searching the frame-count versus token-retention frontier per video length is a direct extension."],"forward_implications":["Frozen multimodal LLMs can be accelerated at deployment time by plugging in merging and pruning, with no fine-tuning or weight changes.","One checkpoint can serve many hardware budgets: adjusting the merging ratio and l1, l2 spans roughly a 40x FLOPs range at under 13% VideoMME accuracy loss.","The FLOPs saved by token reduction can be reinvested in denser video sampling; at equal compute, 192-frame inputs beat the 32-frame base model by +4.6 on MLVU.","Visual token redundancy is large: keeping only 25% of video tokens preserves close-to-base accuracy, and later LLM layers tolerate aggressive visual pruning, suggesting architectural guidance for future efficient multimodal LLMs.","Pruning text tokens during inference is costly (VideoMME drops from 58.2 to 45.7), so efficient designs should keep text tokens intact."],"supporting_citations":[{"why":"Provides the base video LLM (LLaVA-OneVision-7B) that the method accelerates and compares against.","marker":"[33]"},{"why":"Provides the base image LLM (LLaVA-1.5-7B) and its evaluation protocol.","marker":"[41]"},{"why":"Supplies the token-merging technique that the paper adapts to merge visual tokens before the LLM.","marker":"[3]"},{"why":"Supplies the PageRank-on-attention idea that the paper uses to score visual token importance in each LLM layer.","marker":"[68]"},{"why":"Baseline that prunes visual tokens at one selected LLM layer; the paper's progressive scheduler is compared against it.","marker":"[5]"},{"why":"Baseline that prunes and merges using vision-encoder key-query pairs; the strongest comparison point at matched FLOPs.","marker":"[62]"},{"why":"VideoMME benchmark, the paper's main accuracy measure for ablations and adaptive trade-off tables.","marker":"[17]"},{"why":"MLVU benchmark used for the long-video claim that more frames at matched compute improve accuracy by +4.6.","marker":"[100]"},{"why":"Provides the FLOPs and prefill-time measurement methodology used for all efficiency numbers.","marker":"[90]"}],"fun_headline_variants":["Merging and pruning tokens makes multimodal LLMs 7x cheaper","Training-free token pruning cuts multimodal LLM compute 7x","AIM: Cut 7x FLOPs in multimodal LLMs by pruning visual tokens","Adaptive token merging and pruning slashes 7x compute for vision LLMs","No retraining: merge and prune tokens to cut multimodal LLM cost 7x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the PageRank scores computed from each layer's softmax attention reliably rank which visual tokens can be removed at that layer, so that dropping most visual tokens in later layers costs almost no accuracy.","fun_headline_variants_meta":{"raw":{"variants":["Merging and pruning tokens makes multimodal LLMs 7x cheaper","Training-free token pruning cuts multimodal LLM compute 7x","AIM: Cut 7x FLOPs in multimodal LLMs by pruning visual tokens","Adaptive token merging and pruning slashes 7x compute for vision LLMs","No retraining: merge and prune tokens to cut multimodal LLM cost 7x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000261,"raw_usage":{"total_tokens":1609,"prompt_tokens":976,"completion_tokens":633,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":530}},"tokens_in":592,"tokens_out":633,"duration_ms":4949,"temperature":1.0,"reasoning_tokens":530,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:36:57.073728+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same merging and pruning schedule on the same base model but choose the visual tokens to drop uniformly at random within each layer; if the random version matches the reported VideoMME score, the attention-based importance ranking is not doing the work, and the paper's mechanism claim would be falsified.","supporting_citations":[{"cited_title":"Improved baselines with visual instruction tuning, 2023","cited_arxiv_id":null,"evidence_quote":"Provides the base image LLM (LLaVA-1.5-7B) and its evaluation protocol."},{"cited_title":"Zero- tprune: Zero-shot token pruning through leveraging of the attention graph in pre-trained transformers","cited_arxiv_id":null,"evidence_quote":"Supplies the PageRank-on-attention idea that the paper uses to score visual token importance in each LLM layer."},{"cited_title":"Llava-prumerge: Adaptive token reduction for efficient large multimodal models","cited_arxiv_id":null,"evidence_quote":"Baseline that prunes and merges using vision-encoder key-query pairs; the strongest comparison point at matched FLOPs."},{"cited_title":"Mlvu: A comprehensive benchmark for multi- task long video understanding, 2024","cited_arxiv_id":null,"evidence_quote":"MLVU benchmark used for the long-video claim that more frames at matched compute improve accuracy by +4.6."},{"cited_title":"Llm inference unveiled: Survey and roofline model in- sights, 2024","cited_arxiv_id":null,"evidence_quote":"Provides the FLOPs and prefill-time measurement methodology used for all efficiency numbers."}],"review_version":1}