{"id":"4c697ae5-d642-47ce-bc50-af5755ff63a4","arxiv_id":"2505.12685","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A plug-in module with learnable memory selection and multi-scale depthwise convolutions is claimed to improve vision Mamba models, but the paper's tables contain numerous internal inconsistencies.","lead":"This paper proposes two small add-on modules for Mamba, an efficient AI sequence model, to help it process images. The modules add memory retention and spatial awareness, and the authors test them as a full backbone, a performance booster, and a fine-tuning adapter on image classification and detection tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 cannot retrieve per-token hidden states with the identity trick it describes, so Adaptor-T's central mechanism is not implementable as written and the paper's headline results are unverifiable.","rationale":"I read the paper as claiming a plug-in module that requires modifying how the SSM solver is used. The strongest claim is experimental: adding Adaptor-T and Adaptor-S improves vision Mamba on ImageNet and COCO. For those experiments to be meaningful, Adaptor-T must be able to read and modify hidden states, but the paper's only implementation description is Algorithm 1, and the identity-matrix trick is mathematically wrong. The standard selective scan kernel outputs y_t, not h_t; setting C to a vector of ones produces a summed scalar per channel, not the hidden vector, and setting C to a matrix is not supported by the kernel described. This is more fundamental than the differentiable-discrete-selection issue the reader identified: even if selection were differentiable, the hidden states needed for selection are never obtained in the first place. This concern is independent of the table inconsistencies and the booster control problem, and it attacks the core module's existence. I agree with the reader that the gradient flow through discrete memory indices is also unspecified, but the hidden-state extraction problem is prior and more decisive. Without a working hidden-state-extraction path, the experimental tables cannot be connected to the proposed mechanism. I recommend UNVERDICTED rather than REJECT because a custom kernel or corrected implementation could in principle make the approach work, but the current submission provides no way to verify it.","tokens_in":15663,"tokens_out":6966,"duration_ms":79212,"concrete_test":"Obtain or reconstruct the Mamba layer and implement Algorithm 1 with the mamba_ssm selective_scan_cuda call using the b1 configuration shapes (e.g., D=48, N=16), then check the shape and values of hidden_state against the recurrent h_t computed from Eq. 3. If the tensor is not (B, L, N) and equal to h_t at every position, or if `C * hidden_state` raises a shape error, Adaptor-T as described is not runnable. In that case, ask the authors for their modified kernel or released code before any experimental claims are accepted.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim depends on inserting Adaptor-T into the SSM recurrence. This requires per-token hidden states h_t from Eq. 3, but the cited Mamba solver computes outputs y_t = C_t h_t + D_t u_t and returns y, not h_t. Algorithm 1 tries to recover h_t by calling SelectiveScanCuda with C_identity = torch.ones_like(C) and D_zeros. If C has the standard input-dependent shape (B, L, N), this yields a scalar sum over N state components, not the N-vector h_t; if C is instead a matrix, ones_like(C) is not an identity matrix. The resulting tensor has the scan output shape (B, L, D), so the subsequent Adaptor_T(hidden_state) and y = C * hidden_state are shape-inconsistent unless D = N, which does not hold for the reported b1/b2 backbones. Recovering h_t at every token requires a custom selective-scan kernel that materializes the recurrent state, but no such kernel, code, or checkpoints are provided. Since Adaptor-T is the memory-retention module claimed to mitigate long-range forgetting, this is not a cosmetic typo: the method as described cannot be implemented with the stated solver, and the reported gains cannot be checked.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Mamba-Adaptor, a plug-and-play extension for vision Mamba backbones. It consists of Adaptor-T, a learnable memory-retention module that selects and aggregates a small set of previous hidden states to mitigate long-range forgetting, and Adaptor-S, a multi-scale dilated depthwise-convolution module applied to the 2D-reshaped output to inject spatial inductive bias. The paper reports three usage modes: a from-scratch backbone for ImageNet-1K classification and COCO detection/segmentation with Mask R-CNN, a booster that continues training a pretrained VMamba for 10 additional epochs, and a parameter-efficient transfer-learning adaptor on CIFAR-100, SVHN, and Food-101. The stated contributions are efficiency and performance gains over Mamba baselines.","tokens_in":15965,"tokens_out":7450,"duration_ms":78403,"significance":"The conceptual direction is timely and potentially useful: lightweight adaptors targeted at known SSM weaknesses (temporal forgetting and missing 2D locality) are a natural complement to scanning-strategy work, and the three usage settings give the evaluation practical breadth. The ablations in Tables 5 and 6 provide some evidence that each module contributes. However, the central Adaptor-T mechanism is not implementable from the paper as written: the described identity-matrix trick cannot return per-token hidden states from the stock Mamba selective-scan kernel, and the discrete memory-coordinate selection is left underspecified. The booster experiment also lacks a continuation-training control. These issues bear directly on the headline claims, so the paper cannot be accepted without major revision and, ideally, a release of code and checkpoints.","major_comments":[{"comment":"The claimed identity-matrix decoupling cannot retrieve per-token hidden states from the standard Mamba selective-scan solver. With the usual input-dependent C of shape (B, L, N), setting C_identity = torch.ones_like(C) makes the kernel compute a scalar sum over the N state components rather than the N-vector h_t, and the returned tensor has the scan-output shape, not the recurrent-state shape. If C were instead a matrix, ones_like(C) would not be an identity matrix. Equations (5)-(9) require the full vector h_i for every token, so Adaptor-T is not implementable as written with the stated solver unless D = N, which does not hold for the reported b1/b2 backbones. No custom kernel, code, or checkpoints are provided to close this gap. Because Adaptor-T is the paper's core memory-retention mechanism, the headline ImageNet/COCO results cannot be verified from the manuscript.","section":"§3.5, Algorithm 1"},{"comment":"The learnable memory-selection scheme predicts continuous coordinates p_k and coefficients c_k, but the paper never states how p_k is converted to discrete hidden-state indices h(p_k) or how gradients flow through this discrete selection. Algorithm 1 omits this step entirely. Without this specification, the claimed dynamic memory retention is underspecified and cannot be reproduced by a reader.","section":"§3.3, Eq. (8)"},{"comment":"The booster experiment trains the adapted VMamba models for an additional 10 epochs, but it reports no control in which VMamba-T/S/B is trained for the same 10 additional epochs without the adaptor. Since the reported gains are only 0.1-0.2% top-1 accuracy, they are within the range one would expect from additional training alone, so the booster claim is not supported without this control.","section":"§4.2, Table 3"}],"minor_comments":[{"comment":"The value -41.2 for APm of Mamba-Adaptor-b1 under the 3× schedule is impossible; please correct the entry and recheck all numbers in this table.","section":"Table 2"},{"comment":"The rows for CoAtNet-T, UniRepLKNet-T, and ConvNeXt-T report identical parameters, FLOPs, and accuracy (29/4.5/82.1); these are likely copy errors and should be replaced with the actual reported numbers.","section":"Table 1"},{"comment":"The text says Mamba-Adaptor-b2 reaches 82.9 top-1 and exceeds VMamba-T by 0.2% and Swin-T by 2.6%, but Table 1 shows 83.0, with differences of 0.4 and 1.7, respectively; please align the text and table.","section":"§4.1"},{"comment":"The text repeatedly uses 'casual' where 'causal' is intended, and there are several other typos ('Adapator', 'Apdator', 'Mamble', 'paramters', 'matric'); a careful proofread is needed.","section":"Abstract and Section 1"},{"comment":"The claim of state-of-the-art performance is stronger than Table 1 supports (for example, InternImage-T reaches 83.5 and Mamba-Adaptor-b2 reports 83.0); please qualify the claim to the compared Mamba baselines or provide a more complete comparison.","section":"Abstract"},{"comment":"The paper states that scaling is open and that application to other vision Mamba variants is future work; given the scope of the three claimed usages, stating these limitations clearly is helpful, but Table 7 should be referenced in the main text where the generalization experiments are described.","section":"§5"}],"recommendation":"major_revision","confidential_remarks":"Given the implementation gap in Algorithm 1 and the absence of code and checkpoints, I strongly suggest asking the authors for a public implementation or at least a detailed shape-and-gradient trace of Adaptor-T. Please also request the missing continuation-training control for the booster experiments. If the hidden-state retrieval cannot be implemented as described, the paper should be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper has a genuinely interesting idea—learned memory selection on Mamba hidden states plus multi-scale depthwise conv on the 2D output—but the central implementation trick in Algorithm 1 is wrong, and the tables have enough errors that the reported results can't be taken at face value.\n\nWhat's new and good: The combination of Adaptor-T and Adaptor-S is a sensible response to real limitations of vision Mamba, namely long-range forgetting and weak 2D spatial structure. The three usages (backbone, booster, fine-tuning adaptor) make for a practical package, and the ablation shows at least monotonic gains. That is a reasonable starting point.\n\nThe problems, in order of severity:\n\n1. Algorithm 1 cannot do what it claims. You can't recover the per-token hidden state h_t by calling SelectiveScanCuda with C_identity = ones_like(C). With the standard Mamba API, that yields a per-channel sum over the N state components, not the state vector itself, and the subsequent y = C * hidden_state is shape-inconsistent unless D = N, which isn't true for the reported b1/b2 backbones. Without a custom kernel, Adaptor-T's memory selection in Eq. 8-9 is not implementable as described. No code or checkpoints are provided, so the headline numbers are unverifiable.\n\n2. The tables have obvious copy-paste errors: three distinct models in Table 1 share identical params, FLOPs, and accuracy, and Table 2 contains a negative APm. A reader can't tell which numbers are real.\n\n3. The booster experiment (Table 3) has no control: you train baseline plus adaptor for 10 extra epochs, but no baseline-only 10-epoch control. The 0.1-0.2% gains could just be from longer training.\n\n4. The transfer-learning comparison is thin; it only compares against linear probing and VPT, not other adaptors, and the abstract's SOTA claim is overreach given the comparison set.\n\nBottom line: the raw idea has merit, but the paper as written doesn't support its claims. It needs a rewrite with correct implementation details, released code, and honest tables before it deserves referee time. I'd desk-reject this version and invite a resubmission.","headline":"The core idea—learned memory selection on Mamba hidden states plus multi-scale depthwise conv on the output—is worth a look, but Algorithm 1 can't work as written and the tables have enough errors that the reported gains are unverifiable.","tokens_in":16473,"tokens_out":4150,"would_cite":false,"duration_ms":43067,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68T45"],"pacs":[],"model":"deepseek-v4-flash","headline":"Memory and convolution add-ons help vision Mamba top ImageNet and COCO","keywords":["state space models","Mamba","visual recognition","memory retention","spatial modeling","dilated convolution","transfer learning","image classification"],"falsifier":"Train Mamba-Adaptor-b1 with Adaptor-T's selected coordinates replaced by fixed, randomly chosen coordinates while keeping everything else identical; if top-1 accuracy on ImageNet stays above roughly 78%, the learned selection is not load-bearing, and the reported gain over the static-selection ablation would collapse.","tokens_in":15493,"feed_emoji":"🧠","tokens_out":5051,"duration_ms":43911,"temperature":0.7,"pith_summary":"This paper argues that three known weaknesses of vision Mamba—causal context, long-range forgetting, and weak 2D spatial modeling—can be repaired by a single plug-in module with two parts. Adaptor-T lets the model select, per hidden state, a small set of earlier hidden states to re-inject, countering memory decay. Adaptor-S applies multi-scale dilated depthwise convolutions to the 2D-reshaped output, restoring spatial inductive bias. The authors show that Mamba-Adaptor serves as a backbone, a booster for pretrained Mamba, and a parameter-efficient adapter for transfer learning, reporting top-1 accuracy of 78.4 and 83.0 on ImageNet for two model sizes and gains on COCO detection.","feed_headline":"Two tiny modules help vision Mamba top ImageNet and COCO","feed_subtitle":"Plug-in memory retention and spatial convolution boost Mamba past Swin-T at similar cost.","key_machinery":"The two modules: Adaptor-T replaces the Mamba solver's default hidden-state update with a learned memory-selection step, where a linear layer $\\phi_p$ predicts $K$ coordinates $p_k$ per hidden state and a softmax-weighted sum aggregates those earlier states; Adaptor-S applies multi-scale dilated depthwise convolutions to the reshaped 2D output. The implementation decouples the optimized Mamba solver by setting $C$ to an identity matrix and $D$ to zero, so the solver outputs hidden states, letting Adaptor-T operate on them and Adaptor-S operate on the matrix-multiplication output.","core_discovery":"The central claim is that vision Mamba can match or beat transformer backbones if the SSM solver is augmented at two points: during hidden-state recurrence, a lightweight linear predictor (Adaptor-T) identifies and aggregates the most-forgotten earlier states; at the output, multi-scale dilated depthwise convolutions (Adaptor-S) inject 2D locality. The paper reports that Mamba-Adaptor-b2 reaches 83.0% top-1 accuracy on ImageNet-1K, exceeding VMamba-T by 0.4 points while using fewer FLOPs, and that as a transfer adapter it recovers most of full fine-tuning's accuracy on CIFAR-100 and SVHN with under 10% of the parameters.","pith_inferences":["Adaptor-T's learned index selection is effectively a sparse attention over past states; the same mechanism could transfer to other recurrent layers, not just SSMs.","If the index prediction is made differentiable explicitly (e.g., via Gumbel-softmax or straight-through estimation), the memory-retention gain might exceed the 0.1-0.4 point ImageNet improvement reported here.","The zero-initialization and weight-sharing choices suggest the adaptor's benefit concentrates early in fine-tuning; testing with fewer than 10 epochs could reveal how quickly the gains accumulate."],"forward_implications":["Mamba backbones with the adaptor match or exceed Swin-T and VMamba-T on ImageNet at comparable FLOPs (83.0 vs 82.6 top-1).","On COCO Mask R-CNN, Mamba-Adaptor-b2 improves over VMamba-T by about 0.8 points in APb and 1.3 points in APm under the 1x schedule.","The adaptor can be inserted into a pretrained VMamba and retrained for only 10 additional epochs, raising top-1 accuracy by 0.1-0.2 points with small parameter growth.","As a transfer-learning adapter, it outperforms linear probing and visual prompt tuning, coming within 0.7-1.5 points of full fine-tuning on CIFAR-100 while using 5.6-9.3% of the parameters."],"supporting_citations":[{"why":"Supplies the selective state space solver (S6) that the adaptor augments.","marker":"[13]"},{"why":"Provides the VMamba baseline and backbone architecture that Mamba-Adaptor builds on and compares against.","marker":"[32]"},{"why":"LocalMamba, a windowed-scanning baseline that Mamba-Adaptor aims to surpass on ImageNet and COCO.","marker":"[24]"},{"why":"Visual Prompt Tuning, the primary parameter-efficient fine-tuning baseline for transfer-learning comparisons.","marker":"[25]"},{"why":"Mask R-CNN, the detection framework used to evaluate backbone quality on COCO.","marker":"[18]"},{"why":"Swin Transformer, the transformer backbone whose ImageNet and COCO results are headline comparison points.","marker":"[33]"}],"fun_headline_variants":["Mamba-Adaptor: two modules fix vision Mamba's context blindness","Memory augments and local convs lift Mamba to top ImageNet score","Plug-in predict-and-conv modules make Mamba beat Swin on ImageNet","Lightweight adaptor helps vision Mamba surpass VMamba on ImageNet"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole gain rests on Adaptor-T's linear layer being able to learn which earlier hidden states to blend, yet the paper never specifies how the predicted coordinates $p_k$ are made discrete or how gradients pass through that selection; if that step is not differentiable, the memory-retention mechanism cannot be trained as described.","fun_headline_variants_meta":{"raw":{"variants":["Mamba-Adaptor: two modules fix vision Mamba's context blindness","Memory augments and local convs lift Mamba to top ImageNet score","Plug-in predict-and-conv modules make Mamba beat Swin on ImageNet","Lightweight adaptor helps vision Mamba surpass VMamba on ImageNet"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000258,"raw_usage":{"total_tokens":1590,"prompt_tokens":964,"completion_tokens":626,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":580,"completion_tokens_details":{"reasoning_tokens":543}},"tokens_in":580,"tokens_out":626,"duration_ms":5685,"temperature":1.0,"reasoning_tokens":543,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:29:04.660536+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train Mamba-Adaptor-b1 with Adaptor-T's selected coordinates replaced by fixed, randomly chosen coordinates while keeping everything else identical; if top-1 accuracy on ImageNet stays above roughly 78%, the learned selection is not load-bearing, and the reported gain over the static-selection ablation would collapse.","supporting_citations":[{"cited_title":"Mamba: Linear-time sequence mod- eling with selective state spaces","cited_arxiv_id":null,"evidence_quote":"Supplies the selective state space solver (S6) that the adaptor augments."},{"cited_title":"Vmamba: Visual state space model","cited_arxiv_id":null,"evidence_quote":"Provides the VMamba baseline and backbone architecture that Mamba-Adaptor builds on and compares against."},{"cited_title":"Localmamba: Visual state space model with windowed selective scan","cited_arxiv_id":null,"evidence_quote":"LocalMamba, a windowed-scanning baseline that Mamba-Adaptor aims to surpass on ImageNet and COCO."},{"cited_title":"Vi- sual prompt tuning","cited_arxiv_id":null,"evidence_quote":"Visual Prompt Tuning, the primary parameter-efficient fine-tuning baseline for transfer-learning comparisons."},{"cited_title":"Mask r-cnn","cited_arxiv_id":null,"evidence_quote":"Mask R-CNN, the detection framework used to evaluate backbone quality on COCO."},{"cited_title":"Swin transformer: Hierarchical vision transformer using shifted windows","cited_arxiv_id":null,"evidence_quote":"Swin Transformer, the transformer backbone whose ImageNet and COCO results are headline comparison points."}],"review_version":1}