{"id":"bb96e697-1e68-4417-bca4-36cb30418b4e","arxiv_id":"2505.12782","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"AdaToken-3D uses attention-derived contribution scores to prune spatial tokens layer by layer in 3D LMMs, achieving about 60 percent FLOPs reduction with roughly unchanged benchmark scores.","lead":"This paper presents AdaToken-3D, a method that prunes redundant 3D scene tokens inside large multimodal AI models to speed up inference. It reports 21 percent faster inference and 63 percent fewer FLOPs on the LLaVA-3D model while keeping accuracy, and argues that most spatial tokens add little to predictions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 60%/<5% redundancy claim is not causally validated: it is a threshold applied to the unvalidated INF metric in Eq. 4, so the paper's theoretical conclusion is unsupported.","rationale":"The reader's CONDITIONAL verdict is on target. I read the paper as an empirical engineering contribution with an overreaching theoretical interpretation. The efficiency gains are plausible but under-reported, and the strongest single point of failure is the derivation of the '60% contribute less than 5%' claim from an unvalidated attention-derived metric. If that metric is arbitrary, the paper's second contribution ('theoretical foundations') collapses, and the pruning schedule becomes a heuristic with no demonstrated mechanism. The concrete test of causal masking would settle whether the metric tracks true contribution. Because this is an addressable validation gap rather than a proven error, CONDITIONAL remains the right verdict.","tokens_in":10645,"tokens_out":3724,"duration_ms":40349,"concrete_test":"Select 100 questions from SQA3D and ScanQA; for each, compute the answer-token log-likelihood with the full LLaVA-3D model and then with each spatial token individually masked, focusing on tokens ranked lowest by INF. Measure the average change in answer log-likelihood when those lowest-ranked tokens are removed, and compare it with the 5% threshold asserted in the abstract. Also re-run the INF computation with epsilon in {1e-3, 1e-2, 1e-1} and with sigma and gamma varied over plausible ranges; if the set of tokens labelled 'minimal contribution' changes by more than 20% across settings, or if masking the lowest-INF tokens shifts the answer probability by more than 5% on average, the 60%/<5% claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"AdaToken-3D's central scientific claim, that over 60% of spatial tokens contribute minimally (<5%) to the final predictions, is not measured causally. It is a threshold applied to the Information Contribution INF(i) in Eq. 4, which combines an exponential of cross-modal attention divided by an unreported epsilon, a Spatial Flow Value with unspecified sigma and gamma, and a log-term. The hyperparameters alpha_i, a1, a2, epsilon, sigma, gamma, and M are not specified in the paper, and no sensitivity analysis is given. Moreover, the layer-wise retention ratios O_pre(i) in Eq. 5 are fitted to this same INF signal via Eq. 7, so the efficiency numbers in Tables II and III do not independently validate INF; they only show that some pruning schedule preserves accuracy. The ranking of individual tokens uses a separate attention score V_j = Q_t (K_s)^T, which is also not compared with causal importance measures. Thus the 60%/<5% redundancy statement, and the 'theoretical foundations' derived from it, rest on an unvalidated proxy. In addition, the reported efficiency gains are internally inconsistent (abstract: 21% faster and 63% FLOPs reduction; Section IV-C: 72% latency reduction and 60% FLOPs reduction; Table III total-time implies roughly 25% faster on one scene), so the precise magnitude of the central efficiency claim is not reproducible from the paper as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"AdaToken-3D proposes an adaptive spatial-token pruning framework for 3D large multimodal models. The method defines an attention-based 'Information Contribution' INF(i) that combines intra-modal and inter-modal attention statistics with a spatial flow recurrence, fits an exponential per-layer retention schedule O_pre(i) to INF(i) via a derivative-constrained optimization, and ranks individual spatial tokens by an attention similarity V_j = Q_t (K_s)^T. On LLaVA-3D-7B, the paper reports accuracy preservation on SQA3D, ScanQA, ScanRefer, and Scan2Cap while claiming large FLOPs and latency reductions, and it interprets the results as evidence that over 60% of spatial tokens contribute minimally (<5%) to final predictions, presenting this as a theoretical foundation for efficient 3D multimodal learning.","tokens_in":11102,"tokens_out":6085,"duration_ms":61450,"significance":"If the reported efficiency-accuracy trade-off is reproducible, AdaToken-3D would be a practically valuable plug-and-play sparsification method for 3D LMMs, extending 2D pruning insights to the 3D setting and comparing against PyramidDrop and FastV on a range of benchmarks, including 2D transfer. The paper's analytical contribution—quantifying spatial-token contribution via attention patterns—is potentially useful but is not yet causally validated; as written, the redundancy conclusion is an interpretation of the method's own proxy rather than an independent empirical finding. The efficiency claim is significant but requires corrected, consistent numbers to be credible. The experimental breadth, especially the multiple benchmarks and baselines, is a strength, though the lack of error bars and the unspecified hyperparameters limit the strength of the parity claims.","major_comments":[{"comment":"Several constants in the central definitions are never given values: epsilon in Eq. (4), sigma and gamma in Eq. (2), a1 and a2 in Eq. (3), alpha_i in Eq. (4), and M in Eq. (5). The implementation details mention only alpha=0.8 and the text constrains alpha to [0.5,1.2], but no values are reported for the other parameters. Because Eq. (7) fits O_pre(i) to INF(i), different choices of these constants change the retention schedule and therefore the efficiency results. Please report all parameter values and a sensitivity analysis, or the method is not reproducible from the paper as written.","section":"§III-B, Eqs. (2)–(5)"},{"comment":"The claim that over 60% of spatial tokens contribute minimally (<5%) to final predictions is a threshold applied to the attention-derived scores INF(i) and V_j, not a measured property of the model's predictions. The paper never validates these proxies against causal token importance, such as the change in answer likelihood or task metrics when a token is removed. Since the retention schedule is itself fit to INF(i) via Eq. (7), the benchmark results show only that one attention-derived pruning schedule preserves accuracy; they do not establish that the pruned tokens are causally minimal contributors. I recommend either removing the theoretical redundancy claim or adding a rank-correlation comparison against a causal oracle on a sample of SQA3D or ScanRefer examples.","section":"§III-B, §V"},{"comment":"The headline efficiency numbers are mutually inconsistent. The abstract reports 21% faster inference and 63% FLOPs reduction; Section IV-C reports 72% latency reduction and 60% FLOPs reduction; Table III for Scene0011-00 implies (5.957-4.439)/5.957 = 25.5% time reduction and 4.57/11.46 = 60.1% FLOPs reduction; Section III-C states a 23% increase in token utilization and a 63% FLOPs reduction. The training-time reductions also differ (37% in the introduction versus about 45% in Section IV-B). Please specify exactly which setting each number refers to and ensure the abstract matches the principal experimental measurement.","section":"Abstract, §IV-C, Table III"},{"comment":"Table III is internally difficult to interpret. The header row \"Total-time(s) FLOPs(T) Total-time(s) Latency\" does not indicate which columns are absolute measurements and which are percentage reductions. The LLaVA-3D row lists 72% in the \"Latency\" column even though it is the baseline row, and the PyramidDrop row lists 16.6%, while the AdaToken-3D row has a dash; presumably these are reductions relative to LLaVA-3D, but the table should state this explicitly and give the corresponding AdaToken-3D reduction. Please clarify the table structure, units, and what the percentages are relative to.","section":"Table III"},{"comment":"The paper states that AdaToken-3D \"can prune up to 90% of the spatial tokens without degrading the performance of LLaVA-3D,\" but Table II reports only 10%, 20%, and 40% drop ratios. No experimental evidence for the 90% claim is presented. Please add the corresponding experiment or remove/qualify this claim.","section":"§IV-B"}],"minor_comments":[{"comment":"The symbol n is used both as the number of layers and as the summation index over tokens; rename one of these to avoid ambiguity.","section":"Eqs. (1) and (3)"},{"comment":"The notation alpha_i in Eq. (4) conflicts with alpha in Eq. (5), and the later statement \"default setting of alpha=0.8\" does not clarify which alpha is intended.","section":"Eqs. (4) and (5)"},{"comment":"The sentence \"the the token utilization rate increased by 23%\" contains a typo and should read \"the token utilization rate increased by 23%.\"","section":"§III-C"},{"comment":"References [6] and [31] appear to be the same paper (\"An Embodied Generalist Agent in 3D World\"); please merge them.","section":"References"},{"comment":"The figure caption includes \"3x 3.5x\" and training-time values without axes or units, leaving unclear what these multipliers compare; please clarify.","section":"Fig. 1"},{"comment":"No error bars or multiple-seed statistics are reported; since parity claims rest on differences of one to two points, variance estimates would strengthen the conclusions.","section":"Tables II and IV"},{"comment":"The sentence \"less spatial tokens also mean less useless tokens\" is grammatically awkward and should be rewritten.","section":"§IV-B"},{"comment":"The statement that SLSQP \"iteratively approximates the Hessian matrix\" is imprecise; SLSQP is a quasi-Newton method, but the sentence should be reworded for accuracy.","section":"§III-C"}],"recommendation":"major_revision","confidential_remarks":"The reader's assessment is fair: the efficiency results are plausible but not reproducible as written due to inconsistent numbers and unspecified parameters, and the theoretical redundancy claim is overreaching without causal validation. I would not reject the paper; with full parameter reporting, reconciled efficiency numbers, and either a causal validation study or a softened theoretical claim, it could become acceptable. The paper fits the journal's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Let me give you the short version: this is an incremental but useful application of visual-token pruning to 3D LMMs. The general pruning idea is old (FastV, PyramidDrop, DynamicViT), but the adaptation to LLaVA-3D with a layer-wise retention schedule and the finding that you can drop a large fraction of spatial tokens without much accuracy loss is a real empirical contribution. If the numbers hold, AdaToken-3D is a practical plug-in for 3D reasoning systems.\n\nThat said, the paper has three soft spots.\n\nFirst, the central scientific claim — that over 60% of spatial tokens contribute minimally (<5%) to the final predictions — is not actually measured. It comes from thresholding the INF metric in Eq. 4, which mixes cross-modal attention, a 'Spatial Flow Value' with free constants sigma and gamma, and a log term. epsilon, alpha_i, M, and the rest are never specified, and there is no sensitivity analysis. So the contribution claim is an interpretation of an unvalidated proxy, not a causal finding. Calling this 'theoretical foundations' is an overclaim.\n\nSecond, the efficiency numbers are internally inconsistent. The abstract says 21% faster and 63% FLOPs reduction; Section IV-C says 72% latency reduction and 60% FLOPs; Table III's total-time implies about 25% faster on one scene, and the SQA3D latency column shows 72% in the baseline row, which does not make sense as printed. This is the kind of thing that erodes trust fast.\n\nThird, the method section is under-specified. The ranking uses V_j = Q_t K_s^T, but the pruning ratios come from a fitted curve O_pre(i) that is itself fit to the INF signal. So the accuracy results show that some pruning schedule works, but they do not independently confirm that INF is measuring what it claims. There are also no error bars, no code release, and only one model (LLaVA-3D-7B) is evaluated.\n\nThe accuracy preservation on ScanQA, SQA3D, ScanRefer, and Scan2Cap is broadly consistent and genuinely impressive at high drop ratios. That part looks solid. The cited prior work is appropriate. So the core engineering contribution is probably real, but the paper needs to disclose constants, add error bars and sensitivity analysis, fix the speedup inconsistencies, and temper the theory language. A serious referee would catch all of this; this is a paper that could become a decent conference contribution after major revision. I would send it to review rather than desk reject.","headline":"Incremental but real token-pruning gains for 3D LMMs; the headline redundancy claim is unvalidated and the speedup numbers are inconsistent.","tokens_in":11500,"tokens_out":2804,"would_cite":false,"duration_ms":26970,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 3D multimodal model can discard most of its spatial tokens per layer and still answer 3D scene questions, with 21% faster inference and 63% fewer FLOPs.","keywords":["3D large multimodal models","spatial token pruning","attention contribution analysis","inference acceleration","LLaVA-3D","token redundancy","3D scene understanding","FLOPs reduction"],"falsifier":"Run a causal test on a sample of SQA3D questions: prune the tokens ranked lowest by $INF(i)$ and compare answer accuracy against pruning a random subset and against pruning the highest-ranked tokens. If removing the lowest-ranked tokens degrades accuracy as much as removing the highest-ranked tokens, the contribution measure is not causal; if the lowest-ranked tokens can be removed with little effect, the redundancy claim is confirmed. A complementary check is attention patching: replace each spatial token's key and value states with zero vectors and measure the resulting change in the generated answer.","tokens_in":10425,"feed_emoji":"⚡","tokens_out":8475,"duration_ms":77373,"temperature":0.7,"pith_summary":"AdaToken-3D claims that a 3D large multimodal model can discard most of its spatial tokens during inference without losing task accuracy, because the model's own attention patterns reveal which tokens matter. The method scores each spatial token in every layer by an information-contribution value that combines intra-modal attention, cross-modal attention, and a smoothed historical flow, then schedules layer-wise token retention by fitting an exponential decay curve to that signal. On LLaVA-3D-7B the authors report 21% faster inference and 63% fewer FLOPs while keeping accuracy on 3D question answering, visual grounding, and dense captioning benchmarks at the level of the unpruned model. They also report that more than 60% of spatial tokens contribute less than 5% to final predictions, and that the same pruning schedule can shorten training time.","feed_headline":"Spatial-token pruning cuts 3D multimodal FLOPs by 63%","feed_subtitle":"Attention-derived scores keep accuracy on 3D QA and grounding while making inference 21 percent faster.","key_machinery":"The mechanism is the Information Contribution score $INF(i)$, built from a weighted sum of inter-modal attention scores between spatial tokens and prompt/system tokens, plus the Spatial Flow Value $F^i_{3D} = \\sigma\\,S^{\\mathrm{self},i}_{3D} + \\gamma\\,F^{i-1}_{3D}$, a damped running total of intra-modal spatial attention. Each layer's retention ratio comes from fitting an exponential decay curve $O_{\\mathrm{pre}}(i)=\\alpha\\exp(-\\beta(i-L))+M$ under a derivative-smoothing loss, optimized with SLSQP under a global token-reduction constraint. Individual spatial tokens are ranked by $V_j=Q_t(K^s)^T$, the similarity between the last instruction token's query and each spatial token's key state. The pruning schedule is therefore adaptive to the architecture rather than a fixed ratio.","core_discovery":"The central claim is that spatial-token redundancy in 3D large multimodal models is strong and layer-dependent: shallow layers carry the spatial information the reasoning depends on, while deeper layers can shed a large share of tokens. AdaToken-3D measures each spatial token's information contribution from attention patterns, separates intra-modal attention (spatial tokens attending to spatial tokens) from inter-modal attention (spatial tokens interacting with prompt and system tokens), and uses a fitted exponential layer-wise retention schedule to prune tokens at the end of each stage. The paper reports that this preserves accuracy on ScanQA, SQA3D, ScanRefer, and Scan2Cap while cutting 63% of FLOPs and about 21% of inference time, and that over 60% of spatial tokens contribute under 5% to the final predictions. The contribution is therefore an efficiency method plus an empirical claim about where redundancy lives in 3D multimodal reasoning.","pith_inferences":["If the attention-derived contribution measure truly tracks causal importance, the same scoring could guide where to add tokens, not only where to remove them; one testable extension is increasing spatial resolution only in shallow layers, where the paper locates most contribution.","The claim that over 60% of spatial tokens contribute under 5% predicts that a 3D LMM trained from scratch with only a fraction of the spatial tokens should reach near-baseline accuracy; the paper does not run this training-time test.","Because the retention curve is fitted to the same attention-derived signal that defines INF, confirming the redundancy claim would require an independent causal intervention, such as attention patching, which the paper does not report."],"forward_implications":["3D large multimodal models can be served with substantially less GPU compute during inference, which makes real-time 3D question answering and embodied-agent workloads more practical.","The pruning schedule can be reused during training, and the paper reports that it reduces GPU hours by roughly 37%.","Applied to 2D tasks through the same backbone, the method matches dedicated 2D pruning methods on VQA and multimodal benchmarks, so the redundancy analysis transfers across modalities.","The layer-wise contribution curves give a quantitative target for architectural changes, such as concentrating spatial computation in shallow layers instead of spreading it uniformly."],"supporting_citations":[{"why":"Supplies the information-flow analysis idea and the definition of information flow as influence of token types on answer tokens, which the paper extends to 3D.","marker":"[1]"},{"why":"Provides the attention-map redundancy analysis and the attention-score pruning baseline that AdaToken-3D compares against and improves.","marker":"[2]"},{"why":"PyramidDrop is the hierarchical token-elimination baseline whose pruning strategy and efficiency numbers anchor the main comparisons.","marker":"[12]"},{"why":"LLaVA-3D is the base 7B model for nearly all experiments, so the efficiency and accuracy claims are measured on it.","marker":"[16]"},{"why":"SQA3D is the main 3D question-answering benchmark used to study layer-wise pruning effects and to report retained accuracy.","marker":"[18]"},{"why":"ScanRefer is the 3D grounding benchmark that shows naive 2D token dropping disrupts grounding while AdaToken-3D preserves it.","marker":"[22]"},{"why":"Scan2Cap is the dense-captioning benchmark used in the same grounding and captioning comparison.","marker":"[23]"},{"why":"LLaVA-NeXT serves as the comparative 2D large multimodal model in zero-shot 2D benchmark evaluations.","marker":"[13]"}],"fun_headline_variants":["Adaptive token pruning cuts 3D multimodal FLOPs by 63%","63% fewer FLOPs in 3D LMMs with adaptive spatial gating","Pruning redundant 3D tokens speeds inference 21% without accuracy loss","Layer-wise token gating trims 63% FLOPs in 3D multimodal reasoning","Spatial token redundancy exploited for efficient 3D LMM inference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the attention-derived Information Contribution $INF(i)$ measures a spatial token's true causal contribution to the model's answer; if that proxy is wrong, both the token ranking and the claim that over 60% of tokens contribute less than 5% collapse.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive token pruning cuts 3D multimodal FLOPs by 63%","63% fewer FLOPs in 3D LMMs with adaptive spatial gating","Pruning redundant 3D tokens speeds inference 21% without accuracy loss","Layer-wise token gating trims 63% FLOPs in 3D multimodal reasoning","Spatial token redundancy exploited for efficient 3D LMM inference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000679,"raw_usage":{"total_tokens":3097,"prompt_tokens":965,"completion_tokens":2132,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":581,"completion_tokens_details":{"reasoning_tokens":2024}},"tokens_in":581,"tokens_out":2132,"duration_ms":13760,"temperature":1.0,"reasoning_tokens":2024,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:26:04.811321+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a causal test on a sample of SQA3D questions: prune the tokens ranked lowest by $INF(i)$ and compare answer accuracy against pruning a random subset and against pruning the highest-ranked tokens. If removing the lowest-ranked tokens degrades accuracy as much as removing the highest-ranked tokens, the contribution measure is not causal; if the lowest-ranked tokens can be removed with little effect, the redundancy claim is confirmed. A complementary check is attention patching: replace each spatial token's key and value states with zero vectors and measure the resulting change in the generated answer.","supporting_citations":[{"cited_title":"From Redundancy to Relevance: Information Flow in LVLMs Across Reasoning Tasks","cited_arxiv_id":null,"evidence_quote":"Supplies the information-flow analysis idea and the definition of information flow as influence of token types on answer tokens, which the paper extends to 3D."},{"cited_title":"An Image is Worth 1/2 Tokens After Layer 2: Plug- and-Play Inference Acceleration for Large Vision-Language Models","cited_arxiv_id":null,"evidence_quote":"Provides the attention-map redundancy analysis and the attention-score pruning baseline that AdaToken-3D compares against and improves."},{"cited_title":"Scan2Cap: Context-aware Dense Captioning in RGB-D Scans","cited_arxiv_id":null,"evidence_quote":"Scan2Cap is the dense-captioning benchmark used in the same grounding and captioning comparison."}],"review_version":1}