{"id":"54f07298-1ffa-48a9-871b-0fbd0dabc997","arxiv_id":"2602.17363","paper_version":3,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Introduces 2Mamba by simplifying Mamba-2 and enhancing its A-mask and hidden-state order, yielding linear-complexity attention competitive with softmax accuracy.","lead":"This paper simplifies Mamba-2 to its core components and then modifies the A-mask and raises the hidden state order to produce 2Mamba, a linear attention model that reaches near-softmax accuracy while using far less memory on long sequences. Engineers building long-context AI systems may read it to see a concrete path toward efficient alternatives to quadratic attention.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Accuracy gains attributed to A-mask and hidden-state-order changes may stem from unisolated implementation or tuning differences during simplification.","rationale":"The reader's weakest assumption directly identifies the same attribution risk. Full text availability allows checking for ablations, but the concern remains technical and internal to the empirical argument. This moves the provisional UNVERDICTED verdict to CONDITIONAL pending isolation of the claimed changes.","tokens_in":1692,"tokens_out":311,"duration_ms":19140,"concrete_test":"Re-run the Mamba-2S baseline and 2Mamba variant on the same long-context benchmarks while changing only the A-mask definition and hidden-state order (keeping all other hyperparameters, code paths, and training details identical to the reported Mamba-2S); if the accuracy delta matches the paper's reported gap to softmax attention, the modifications are causal.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires that accuracy improvements from Mamba-2S to 2Mamba are caused specifically by the A-mask modification and increased hidden-state order. The paper describes first simplifying Mamba-2 by evaluating component importance, then applying those two changes. If other unstated adjustments (e.g., normalization, initialization, optimizer settings, or dataset-specific tuning) occurred during the simplification or improvement phases, the attribution to the named modifications is not secured. This is especially load-bearing because the headline result is an empirical accuracy comparison to softmax attention rather than a parameter-free derivation.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript describes the development of 2Mamba, obtained by first simplifying Mamba-2 to a core variant called Mamba-2S through evaluation of component importance, followed by an A-mask modification and an increase in hidden-state order. The central empirical claim is that 2Mamba achieves accuracy nearly matching softmax attention while remaining linear in complexity and more memory-efficient for long contexts; code for all experiments is released.","tokens_in":1816,"tokens_out":332,"duration_ms":25057,"significance":"If the accuracy comparisons hold under controlled conditions, the work could advance practical linear-attention alternatives for long-sequence modeling. The explicit component-wise simplification process and code release are positive elements that support reproducibility and allow the community to verify the reported gains.","major_comments":[{"comment":"Experiments section: the transition from Mamba-2S to 2Mamba attributes accuracy gains specifically to the A-mask and increased hidden-state order, yet the reported ablations do not isolate these two changes from other implementation or hyperparameter adjustments that may have occurred during the simplification phase. Because the headline result is an empirical accuracy comparison rather than a parameter-free derivation, this attribution is load-bearing and requires clearer incremental tables or controlled re-runs.","section":"Experiments"}],"minor_comments":[{"comment":"Abstract: the phrase 'nearly as accurate as softmax attention' would benefit from a parenthetical reference to the specific datasets and metric values that support the claim.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback on the experiments section. We agree that clearer isolation of the A-mask and hidden-state order contributions is important given the empirical nature of the headline claims, and we will strengthen this in the revision.","responses":[{"response":"We acknowledge that the current presentation of ablations could be strengthened to more explicitly isolate the A-mask modification and hidden-state order increase. The simplification to Mamba-2S was performed first via component-wise evaluation with fixed hyperparameters, after which only the A-mask and order changes were introduced on the resulting Mamba-2S configuration. To directly address the concern, we will add new incremental ablation tables to the revised manuscript. These tables will apply the A-mask change alone, the order increase alone, and both changes together, all starting from the identical Mamba-2S baseline with no other implementation or hyperparameter adjustments. This will make the attribution of gains fully transparent and controlled.","revision_made":"yes","referee_comment":"[Experiments] Experiments section: the transition from Mamba-2S to 2Mamba attributes accuracy gains specifically to the A-mask and increased hidden-state order, yet the reported ablations do not isolate these two changes from other implementation or hyperparameter adjustments that may have occurred during the simplification phase. Because the headline result is an empirical accuracy comparison rather than a parameter-free derivation, this attribution is load-bearing and requires clearer incremental tables or controlled re-runs."}],"tokens_in":1210,"tokens_out":320,"duration_ms":43517,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing here is that the authors take Mamba-2, strip it to a simpler core version they call Mamba-2S by testing component importance, then change the A-mask and increase the hidden-state order to reach 2Mamba. They report this gets close to softmax accuracy while staying memory-efficient for long contexts, and they also look at which original pieces help beat softmax in places. Code release is noted, which helps with checking the details later.","headline":"They simplify Mamba-2, tweak the A-mask and raise hidden-state order to claim near-softmax accuracy in linear time, but the numbers and isolation of those changes are not shown in the abstract.","tokens_in":2329,"tokens_out":181,"would_cite":false,"duration_ms":67646,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[],"headline":"Empirical ML architecture work on linear attention with no RS machinery","alignment":"orthogonal","rationale":"The paper performs ablations on Mamba-2 components (A-mask variants, convolutions, discretization) and introduces a squared QK inner product for second-order hidden states to approach softmax accuracy at linear cost. None of this invokes J-cost, cosh(ρ ln φ), φ-ladders, 8-tick periodicity, or parameter-free constant derivations. Domain is sequence modeling; RS framework has no opinion.","tokens_in":54344,"confidence":"high","tokens_out":128,"duration_ms":8637,"cache_read_input_tokens":32896,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A simplified Mamba-2 variant with A-mask and hidden-state tweaks nearly matches softmax attention accuracy while staying linear in complexity.","keywords":["linear attention","Mamba","softmax attention","transformers","long context","memory efficiency","sequence modeling"],"falsifier":"Re-run the 2Mamba experiments using the original A-mask and the lower hidden-state order from Mamba-2S and check whether accuracy falls back to the level of the simplified model without those two changes.","tokens_in":2576,"feed_emoji":"⚡","tokens_out":613,"duration_ms":24831,"temperature":0.7,"pith_summary":"The authors start from Mamba-2, a strong linear-attention model, and strip it down to its essential parts to form Mamba-2S. They then modify the A-mask and raise the order of the hidden state to create 2Mamba. This produces a model whose accuracy approaches that of softmax attention on standard benchmarks yet scales linearly and uses far less memory when sequences grow long. The work also identifies additional Mamba-2 elements that can push performance above the softmax baseline. The result matters because quadratic softmax attention becomes impractical for very long contexts while linear alternatives have historically paid an accuracy price.","feed_headline":"Simplified Mamba variant nearly matches softmax accuracy at linear cost","feed_subtitle":"A-mask and hidden-state-order changes close the accuracy gap while keeping memory use linear for long sequences.","key_machinery":"The A-mask and the order of the hidden state, whose specific modifications turn the simplified Mamba-2S into the higher-accuracy 2Mamba.","core_discovery":"By simplifying Mamba-2 to its core components and then improving the A-mask together with an increase in hidden-state order, the resulting 2Mamba model reaches accuracy nearly equal to softmax attention while remaining linear in complexity and therefore far more memory-efficient at long context lengths.","pith_inferences":["The same mask and state-order adjustments might transfer to other linear-attention families beyond Mamba.","A hybrid system could switch between 2Mamba for long prefixes and softmax for short, high-precision segments.","Direct measurement of memory usage versus sequence length on hardware would quantify the practical efficiency gain."],"forward_implications":["2Mamba can handle much longer sequences than softmax attention at comparable memory cost.","Accuracy remains competitive on tasks where standard transformers are typically evaluated.","Certain untouched components of the original Mamba-2 can push accuracy above the softmax baseline.","The simplification step isolates which design choices most affect final performance."],"fun_headline_variants":["2Mamba nearly matches softmax accuracy at linear complexity","2Mamba nears softmax accuracy at linear complexity","2Mamba reaches near softmax accuracy with linear complexity","2Mamba variant nears softmax accuracy linearly"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The observed accuracy gains are produced by the A-mask and hidden-state-order changes themselves rather than by other unstated implementation choices or hyperparameter adjustments made during the simplification process.","fun_headline_variants_meta":{"raw":{"variants":["2Mamba nearly matches softmax accuracy at linear complexity","2Mamba nears softmax accuracy at linear complexity","2Mamba reaches near softmax accuracy with linear complexity","2Mamba variant nears softmax accuracy linearly"]},"model":"grok-4.3","cost_usd":0.014404,"raw_usage":{"total_tokens":6074,"prompt_tokens":571,"num_sources_used":0,"completion_tokens":53,"cost_in_usd_ticks":144040500,"prompt_tokens_details":{"text_tokens":571,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":5450,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":571,"tokens_out":53,"duration_ms":68307,"temperature":1.0,"reasoning_tokens":5450,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-21T11:45:24.926234+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Re-run the 2Mamba experiments using the original A-mask and the lower hidden-state order from Mamba-2S and check whether accuracy falls back to the level of the simplified model without those two changes.","supporting_citations":[],"review_version":1}