{"id":"6b44d5a1-8ef8-4a80-abcd-b5c510077648","arxiv_id":"2412.16602","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"VMeanba speeds up VMamba's selective scan by averaging its internal channel dimension down to 1, achieving up to 1.12x end-to-end speedup with under 3% accuracy loss on ImageNet and ADE20k.","lead":"VMeanba is a training-free trick that replaces the many internal channels of VMamba's state-space scan with a single averaged channel, cutting computation at a modest cost in accuracy. It matters because state-space models are a promising alternative to transformers, and this is a simple way to make their inference faster on standard hardware.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The <3% accuracy-loss claim is fitted to the same validation set used for layer/K selection (Eq. 3, Algorithm 1), so it is not an independent estimate; a held-out evaluation is required.","rationale":"The reader's formal weakest_assumption targets the channel-constant property, but the paper's own method section exposes a more direct threat to the central claim: layer selection and accuracy reporting share the same validation images. This is a textbook selection-bias problem. The reader did mention this in the rationale, but did not elevate it to the weakest_assumption. I consider it the single most load-bearing concern because it does not depend on how the mean approximation behaves internally; it attacks the validity of the reported numbers themselves. A favorable result on a held-out split would substantially rehabilitate the claim, which is why the verdict should remain conditional rather than outright reject. If the held-out evaluation shows more than 3% loss, the abstract's headline would need to be revised. The pruning claim should additionally be clarified by reporting the drop from the unpruned original model, not from the pruned baseline.","tokens_in":8031,"tokens_out":8235,"duration_ms":72774,"concrete_test":"Split the ImageNet-1k validation set into two disjoint halves (25k each; similarly split ADE20k validation into 1k/1k). Use the first half exactly as in Algorithm 1 to select the K layers and to choose K for each backbone. Then apply the selected VMeanba configuration to the second half and compute the accuracy drop relative to the unmodified model on that second half. If the drop exceeds 3 percentage points for any of the K values advertised in Table 4 (e.g., Tiny K=4, Small K=4, Base K=8), the central claim fails to generalize beyond the selection set.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.2 defines the layer impact score S_layer = Acc(OriginalModel) − Acc(VMeanba on layer) and selects the K layers with the smallest scores on the validation set (Algorithm 1). Appendix D states that all experiments use only the validation set. Table 4 then reports accuracy for these selected configurations on that same validation set. Consequently, the reported drops (e.g., Tiny K=4: 82.5→80.7, Base K=4: 83.7→83.3) are not independent predictions: the method has already been tuned to minimize loss on the exact images used to evaluate it. The abstract's headline 'less than 3% accuracy loss' is therefore a fitted, in-sample statistic. No test-set numbers, error bars, or code are provided, so there is no way to know the out-of-sample loss. This issue is independent of the channel-constant mechanism: even if low channel variance is real, the empirical claim as reported is not supported. The pruning sub-claim is also ambiguous: Table 1 compares VMeanba+pruning against a pruned baseline (80.1% for Conv2D), not against the original 83.7% model, so the 'under 3%' drop is relative to an already degraded model.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VMeanba, a training-free compression method for the selective scan operation in VMamba. The method replaces the inner channel dimension of the scan inputs (A, Bu, C) with their mean along the channel axis, runs the scan with dimension 1, and broadcasts the output back to the original dimension. Layer selection is performed by scoring each layer with Eq. (3) on the validation set and choosing the K layers with the smallest score (Algorithm 1). Experiments on ImageNet classification and ADE20k segmentation report up to 1.12x end-to-end speedup with less than 3% accuracy loss, and a combination with 40% unstructured pruning is claimed to keep the drop under 3%.","tokens_in":8252,"tokens_out":4996,"duration_ms":46463,"significance":"If the empirical claims survive a properly held-out evaluation, the method would be practically useful: it is simple, training-free, addresses a real bottleneck (the selective scan kernel), and appears orthogonal to pruning. The complexity and memory reductions in Appendix B and Tables 2-3 are substantial in principle. The paper also ships a concrete selection algorithm rather than leaving layer choice entirely unspecified. However, the current evaluation protocol prevents assessing the central claim: all accuracy numbers are computed on the same validation set used for selection, and the kernel speedup comparison uses kernels with different scopes. These issues are fixable with additional experiments, so the work is not beyond repair.","major_comments":[{"comment":"The headline accuracy claim is in-sample by construction. Eq. (3) defines the layer impact score using Acc(VMeanba on layer) on the validation set, Algorithm 1 selects the K layers from those scores, and Figure 4/Table 4 report accuracy for the selected K on the same validation set. Appendix D explicitly states that only the validation set is used. Thus the reported '<3% accuracy loss' is the loss of a configuration already optimized on the exact images used for evaluation, and K is chosen post hoc from the same curve. This does not provide an independent estimate of generalization. Please report accuracy on the held-out test set (or a cross-validated estimate), with the layer-selection procedure fixed before evaluating on the test set, and include variance or confidence intervals.","section":"§2.2, Eq. (3), Algorithm 1, Appendix D, Table 4, Figure 4"},{"comment":"The abstract's claim that combining VMeanba with 40% pruning keeps the accuracy drop under 3% is not supported by Table 1 for the Conv2D case. In Table 1, the Conv2D-pruned baseline is 80.1% and VMeanba+pruning is 77.5%, a drop of 2.6 points relative to the pruned baseline, but relative to the original Base VMamba accuracy of 83.7% (Table 4, K=0) the drop is 6.2 points. The Linear Layers case stays under 3% against either baseline, but the current table does not distinguish these baselines. Please state the comparison baseline explicitly and correct the abstract or add the missing comparison against the unpruned model.","section":"§3.2, Table 1, abstract"},{"comment":"The kernel speedup numbers compare kernels with different scopes. The original CUDA kernel includes both discretization and the scan operation, while the new kernel handles only the scan operation, with discretization executed outside the kernel. Reporting a 273x speedup for the scan kernel alone against a kernel that also performs discretization is not an apples-to-apples comparison and does not by itself justify the end-to-end speedups in Table 4. Please report kernel-level timings for identical work scopes, or a full-block comparison that includes the discretization cost.","section":"Appendix D, Tables 2 and 3"},{"comment":"The core modeling assumption that y_layer[:, d, :] ≈ y_layer[:, d', :] is supported only by visual inspection of Figure 3. There is no quantitative variance threshold, no error metric between the full scan and the reduced scan, and no evidence that the property transfers across model sizes, resolutions, or after pruning. Because the entire method rests on this near-constant-channel property, please provide quantitative measurements, for example the ratio of channel variance to total activation norm per layer, the relative output error of the reduced scan versus the full scan, and the per-layer accuracy sensitivity. Without such evidence, the claim that 'only 1 hidden dimension' is sufficient is not established.","section":"§2.1, Eq. (1), Figure 3"}],"minor_comments":[{"comment":"There is a typo: 'indroduce' should be 'introduce'. Also Eq. (2) writes y_layer = T^{-1}(Mamba(T(A, Bu, C))) but does not specify how T acts on the product Bu; since u is computed inside the block, the notation should be clarified.","section":"§2.2"},{"comment":"The column header 'Accuracy (Acc@1 / aAcc)' is confusing for an image classification table; aAcc is only used for segmentation. Please separate the metrics for the two tasks.","section":"Table 4"},{"comment":"The FLOP notation in Eq. (10) is typeset inconsistently (e.g., 'F LOPM amba'), and the variable d in '9BLd' is not defined at that point. Please define all symbols and give a step-by-step derivation of the reduced FLOP count, including the mean and broadcast costs.","section":"Appendix B, Eq. (10)"},{"comment":"The sentence 'We only use the validation set of them for the experiments' is a significant protocol statement; it should be either removed after switching to test-set evaluation or accompanied by an explicit explanation of why no test set is used. The paper also does not provide code or checkpoints, which would help reproducibility.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":"The main obstacle is the validation-set-based selection and evaluation loop. If the authors can report held-out test accuracy with a pre-registered selection rule and provide a fair kernel comparison, the paper could become acceptable. I did not find evidence of deliberate misconduct, but the current reporting makes the central claim unverifiable as stated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new thing here is not the mean transform; it's the empirical observation that VMamba's inner scan dimension has near-constant activations across channels, and the resulting training-free D-to-1 reduction. That is a real, checkable observation, and the method is refreshingly simple: collapse the scan inputs, run one scan, broadcast back. The paper is honest that this is an optimization, not a new architecture, and the idea is orthogonal to pruning.\n\nThe big soft spot is the evaluation. The paper uses the validation set both to choose which K layers to compress (Eq. 3, Algorithm 1) and to report the accuracy of those choices (Table 4). The “less than 3% loss” is therefore fitted, not predicted. K is also picked post hoc from the same curve. You cannot tell from these numbers what the out-of-sample loss is. The appendix confirms all experiments use only the validation sets. This is a real problem, and the stress-test note is on target.\n\nThe kernel-level speedups (273x etc.) are scan-only microbenchmarks: the original kernel includes discretization while the new kernel runs only the scan. The paper discloses this in Appendix D, but the headline speedup should not be the kernel number. The end-to-end number is at most 1.12x—useful, not transformative. The pruning subclaim is also weakened: Table 1 compares against a pruned baseline (80.1%), not the original 83.7%, so “under 3%” is relative to an already degraded model.\n\nThere are no error bars, no code, and no test-set split. For a workshop paper, the idea is worth airing, but the empirical claim as stated is not supported. A referee should ask for a held-out split, a fixed K chosen before evaluation, and a fair kernel comparison. That is fixable. The low-variance observation itself can be validated quickly and would make the paper solid if it survives.\n\nI would send this to review, because the idea is concrete and the flaw is correctable, but I would not accept the current accuracy claims.","headline":"The paper's real contribution is the observation that VMamba's inner scan channels are near-constant and the resulting training-free mean-reduction, but the headline accuracy claim is fitted to the same validation set used to select layers, so the empirical case is not yet made.","tokens_in":8848,"tokens_out":1993,"would_cite":false,"duration_ms":18299,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a mean transform over channels can replace the hidden dimension in visual state space models, giving up to a 1.12x speedup with under 3% accuracy loss.","keywords":["State Space Models","VMamba","selective scan","model compression","training-free compression","inference acceleration","channel redundancy","semantic segmentation"],"falsifier":"Measure, for every Mamba block of a pretrained VMamba model on ImageNet validation, the channel-wise variance of the block's output activations, and compare the model's accuracy before and after mean-collapsing only those blocks whose variance is near zero; if replacing a near-zero-variance block still costs more than 3 percent accuracy, the presumed link between channel constancy and scan fidelity is broken.","tokens_in":7783,"feed_emoji":"⚡","tokens_out":8512,"duration_ms":69525,"temperature":0.7,"pith_summary":"The paper claims that visual state space models such as VMamba waste computation in their selective-scan blocks: each block's output is nearly constant across the inner channel dimension, so the scan inputs $\\bar{A}$, $\\bar{B}u$, and $C$ can be replaced by their channel-wise mean without retraining. The proposed VMeanba transform collapses the inner dimension from $D$ to $1$, runs the scan in the reduced space, and broadcasts the result back. On ImageNet-1k classification and ADE20k semantic segmentation, the paper reports up to a $1.12\\times$ end-to-end speedup with less than 3% accuracy loss, and compatibility with 40% unstructured pruning. If the paper is right, a large fraction of the channel dimension in these scan blocks is redundant at inference time, making post-training acceleration cheaper than quantization or pruning.","feed_headline":"Visual state space models only need 1 hidden dimension","feed_subtitle":"Channel-averaging the scan inputs gives up to 1.12x faster VMamba inference with less than 3% accuracy loss.","key_machinery":"The load-bearing mechanism is a transform pair $(T, T^{-1})$ around the selective-scan recurrence, the inner state-space scan over sequence length $L$ that dominates Mamba-block FLOPs. $T$ is the mean operator along the inner channel dimension $D$: it maps the three scan inputs $\\bar{A}$, $\\bar{B}u$, $C$ to single-channel representatives, the scan runs with $D=1$, and $T^{-1}$ broadcasts the output back to the full width. In FLOP terms the scan and state update go from $9BDL$ to about $B(D+10)L$, an 89% reduction for large $D$, which is why collapsing the channel axis rather than pruning weights or quantizing values is the paper's chosen lever.","core_discovery":"The central claim is that the hidden dimension of the Mamba scan inside VMamba is essentially redundant. For each layer, the paper asserts $y_{\\text{layer}}[:, d, :] \\approx y_{\\text{layer}}[:, d', :]$ for all inner channels $d, d'$, and it attributes this to the scan inputs $\\bar{A}$, $\\bar{B}u$, $C$ being jointly low-rank along the channel axis. The mechanism is a transform $T$ that averages those inputs over the channel dimension to dimension $1$, followed by the original Mamba scan and a broadcast $T^{-1}$, so that $y_{\\text{layer}} = T^{-1}(\\mathrm{Mamba}(T(\\bar{A}, \\bar{B}u, C)))$. A validation-set pipeline replaces only the $K$ layers whose substitution costs the least accuracy. With this setup the paper reports under 3% top-1 accuracy loss on ImageNet classification and under 3% all-pixel accuracy loss on ADE20k, up to $1.12\\times$ end-to-end batch-inference speedup, and a similar accuracy bound when combined with 40% unstructured pruning of linear or convolutional weights.","pith_inferences":["An untested extension of the paper's mechanism is that other selective-scan vision backbones with similar channel-invariant activations would enjoy the same mean-collapse speedup; a quick test on a different SSM backbone would show whether the redundancy is architectural or specific to VMamba.","The paper's layer-selection pipeline needs a validation forward pass per candidate layer; that calibration cost is not included in the reported speedup, so a variance-based heuristic would be needed for truly one-shot deployment.","The paper does not examine whether channel variance stays low after pruning or at higher input resolutions, so the 3% accuracy bound should be expected to degrade in regimes where channel invariance breaks down.","The gap between the appendix's kernel-level speedups (up to 293x) and the end-to-end 1.12x suggests that fusing the reduced scan with discretization, and removing the mean/broadcast overhead, is the concrete next step to turn this into a larger wall-clock win."],"forward_implications":["VMamba inference can be accelerated without retraining simply by mean-collapsing the scan inputs of the selected layers, making the method a drop-in post-training step.","The accuracy-versus-$K$ curve lets a user trade speed for accuracy per model, selecting the $K$ lowest-impact layers from a validation set.","Since VMeanba operates on the scan inputs while pruning operates on weights, the two optimizations are orthogonal in the paper's experiments, so they can be stacked for further speedup.","Kernel memory traffic drops by orders of magnitude (e.g., from 3.3G/823.5M bytes to 6.4M/1.3M bytes at one configuration), enabling longer scan sequences or larger batches on the same GPU."],"supporting_citations":[{"why":"Provides the pretrained VMamba backbones (tiny, small, base) and the baseline accuracy and latency that VMeanba compresses.","marker":"[16]"},{"why":"Defines the selective scan operation and its GPU kernel, the computational target VMeanba reduces.","marker":"[7]"},{"why":"Supplies ImageNet-1k validation data used for the classification accuracy evaluation and layer selection.","marker":"[2]"},{"why":"Supplies ADE20k validation data for the semantic segmentation evaluation.","marker":"[28]"},{"why":"Provides the UperNet head used on top of VMamba for the segmentation experiments.","marker":"[26]"},{"why":"Gives the state-space model equations that the scan and discretization are built on.","marker":"[10]"}],"fun_headline_variants":["Channel averaging makes visual SSMs need only 1 hidden dimension","VMeanba: 1.12x faster vision SSMs with under 3% accuracy loss","Visual state space models slimmed to 1 hidden dimension","Averaging channels speeds up VMamba by 1.12x with minimal loss","Training-free compression: SSMs drop to 1 hidden dimension"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument rests on the claim that each Mamba block's output is nearly identical across its inner channel dimension, so averaging the scan inputs along that dimension and broadcasting the output back loses almost nothing.","fun_headline_variants_meta":{"raw":{"variants":["Channel averaging makes visual SSMs need only 1 hidden dimension","VMeanba: 1.12x faster vision SSMs with under 3% accuracy loss","Visual state space models slimmed to 1 hidden dimension","Averaging channels speeds up VMamba by 1.12x with minimal loss","Training-free compression: SSMs drop to 1 hidden dimension"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000445,"raw_usage":{"total_tokens":2278,"prompt_tokens":1001,"completion_tokens":1277,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":617,"completion_tokens_details":{"reasoning_tokens":1179}},"tokens_in":617,"tokens_out":1277,"duration_ms":8221,"temperature":1.0,"reasoning_tokens":1179,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:25:32.315236+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure, for every Mamba block of a pretrained VMamba model on ImageNet validation, the channel-wise variance of the block's output activations, and compare the model's accuracy before and after mean-collapsing only those blocks whose variance is near zero; if replacing a near-zero-variance block still costs more than 3 percent accuracy, the presumed link between channel constancy and scan fidelity is broken.","supporting_citations":[{"cited_title":"Gu and T","cited_arxiv_id":null,"evidence_quote":"Defines the selective scan operation and its GPU kernel, the computational target VMeanba reduces."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies ADE20k validation data for the semantic segmentation evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the UperNet head used on top of VMamba for the segmentation experiments."}],"review_version":1}