{"id":"a7efd39c-e41e-411e-9797-670eff08f5a1","arxiv_id":"2412.16745","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"A Vision Mamba stereo network runs at about 50 FPS, but with higher average error than top models, and a self-defined score is used to declare the trade-off solved.","lead":"ViM-Disparity swaps transformer blocks for Vision Mamba blocks in a stereo depth network to run faster, and adds a new score combining speed, error, and memory. Tests show it runs about five times faster than the next baseline, but with worse accuracy on most benchmarks, so the win depends on how speed is weighted.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOMER advantage rests on FPS measurements whose comparability is not established; unless baselines are rerun under identical hardware, resolution, and timing protocol, the trade-off claim is unsupported.","rationale":"The reader's weakest assumption is the comparability of FPS/memory measurements, and I agree that this is the load-bearing point. The paper gives no evidence that the baseline speed or memory figures were measured under the same hardware and protocol as the proposed model, even though SOMER is constructed to make FPS the dominant factor. A controlled rerun of all baselines on the same machine would settle whether the speed advantage is real. I add a narrower, internal-consistency concern: the reported D1 values are hard to reconcile with the reported EPE values under the standard 3-pixel threshold, which makes the accuracy portion of Table I unreliable. This does not change the reader's REJECT verdict, but it narrows the reason: the manuscript's quantitative evidence is not yet trustworthy enough to support the central claim, and the missing experimental protocol is the concrete fix.","tokens_in":8510,"tokens_out":7063,"duration_ms":63643,"concrete_test":"Download the official checkpoints for Unimatch, I-GEV, AnyNet, RAFT-Stereo, and ViM-Disparity. Run all methods on the same Nvidia A6000, at the native resolution of each dataset, with batch size 1, the same precision (e.g., fp32), and time only the disparity-function call exactly as described in Section IV. Recompute FPS, memory footprint, EPE, D1, and SOMER for Table I. If any baseline FPS more than doubles relative to the reported values, or if the proposed method no longer has the highest SOMER on KITTI, SceneFlow, and VKITTI2, then the central trade-off claim collapses. Also recompute D1 with the standard 3-pixel/5% threshold from the saved disparity maps and check whether the reported EPE and D1 pairs are mutually consistent.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central conclusion in Section V (“dissolve the trade-off”) is driven entirely by SOMER, which is linearly proportional to FPS and inversely proportional to EPE and log memory. Table I reports the proposed method at 51.53 FPS on KITTI versus 10.02 for Unimatch and 3.30 for RAFT, and the proposed SOMER is highest on KITTI, SceneFlow, and VKITTI2. However, Section IV states only that “all experiments are performed” on an Nvidia A6000; it never states that Unimatch, I-GEV, AnyNet, and RAFT were also executed on that same machine, at the same input resolution, with the same batch size, precision, and official checkpoints, and with the same “disparity-function-only” timing protocol. If the baseline FPS numbers come from other papers or other GPUs, the 3–5× speed gap is not attributable to the proposed architecture. Because SOMER scales linearly with FPS, a modest timing or hardware difference is enough to erase or reverse every SOMER win. A further red flag is that the D1 and EPE columns are not internally consistent: for SceneFlow, EPE=4.4 with D1=0.18 would require an average error among the flagged pixels of hundreds of pixels under a 3-pixel threshold, which is implausible at that resolution. Thus the accuracy table itself is not reliable enough to benchmark a trade-off claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Visual Mamba (ViM) based stereo disparity estimation architecture and a new composite metric, SOMER = FPS / (EPE × log(M)), intended to jointly evaluate speed, accuracy, and memory. The model is compared against Unimatch, I-GEV, AnyNet, and RAFT on KITTI, SceneFlow, Sintel, and VKITTI2. The authors report the highest SOMER on KITTI, SceneFlow, and VKITTI2, and claim that this 'dissolves' the trade-off between real-time inference, accuracy, and memory footprint for disparity map generation. The paper also includes an ablation study and releases code at a GitHub repository.","tokens_in":8802,"tokens_out":4515,"duration_ms":40209,"significance":"If the central claims were substantiated, the work would be a useful step for real-time stereo matching in robotics, and the open-source release is a practical contribution. The paper also honestly reports cases where it does not achieve the best EPE/D1 and includes an ablation study. However, the significance of the claimed trade-off dissolution is currently undermined by three load-bearing issues: the FPS measurements are not shown to be comparable across methods, the SOMER metric is introduced without external validation or principled justification, and the accuracy table contains an internal inconsistency (SceneFlow EPE/D1). Because the headline conclusion depends on these, the current evidence does not support the paper's main claim.","major_comments":[{"comment":"The FPS comparison is not controlled. Section IV states only that all experiments were performed on an Nvidia A6000 and that FPS is computed from the time difference before and after the disparity function call. It does not state that Unimatch, I-GEV, AnyNet, and RAFT were run on the same GPU, at the same input resolution, with the same batch size and numerical precision, using their official checkpoints, and with the same timing protocol. Since SOMER is linearly proportional to FPS (Eq. 5), a modest difference in timing conditions could alter the ranking. The authors must report the full measurement protocol and either rerun all baselines under identical conditions or clearly state the source of each baseline FPS number.","section":"IV, Table I"},{"comment":"The SOMER metric is introduced as an ad-hoc composition of FPS, EPE, and log(M) without external validation or a derivation from deployment requirements. The choice of the reciprocal of EPE and the logarithm of memory is not justified, and no evidence is provided that SOMER correlates with practical suitability for real-time robotics. Moreover, the paper's own Table I shows that RAFT outperforms the proposed model on Sintel in SOMER (1.692 vs 0.785), so the claimed dissolution of the trade-off is not universal. The conclusion should be scoped to datasets where the proposed model wins, or the metric should be validated against established efficiency-accuracy trade-off analyses (e.g., Pareto fronts).","section":"II-C, Eq. (5)"},{"comment":"The reported SceneFlow values EPE=4.4 and D1=0.18 are internally inconsistent. D1 is the fraction of pixels with absolute disparity error greater than a threshold (typically 3 pixels). In a 1960×540 image, 0.18% of pixels is a tiny fraction; for the mean EPE to be 4.4, those outlier pixels would need errors averaging thousands of pixels, which is implausible for the disparity ranges in SceneFlow. This suggests an error in computing or reporting one or both metrics. Because the accuracy columns underpin the entire trade-off analysis, this inconsistency must be corrected and all EPE/D1 values verified before the paper's claims can be assessed.","section":"Table I, SceneFlow row"},{"comment":"The model is trained only on VKITTI2 and SceneFlow, yet Table I reports results on Sintel and KITTI. The paper does not discuss how the model was adapted to these datasets, whether any fine-tuning was performed, or how domain shift affects the evaluation. The large EPE on Sintel (11.53 vs 0.45 for RAFT) suggests poor cross-domain generalization, which directly weakens the claim that the model is 'accurate' across the benchmarked datasets. The authors should clearly state the evaluation protocol for each dataset (zero-shot vs fine-tuned) and temper accuracy claims accordingly.","section":"IV, Implementation details"}],"minor_comments":[{"comment":"The formula as typeset (SOMER = FPS / EPE × log(M)) is ambiguous: it could be read as (FPS/EPE)·log(M). Add parentheses to make the intended denominator EPE·log(M) explicit.","section":"II-C, Eq. (5)"},{"comment":"The abbreviations 'DMG' and 'DGM' are used inconsistently (e.g., Section II-C uses 'DGM' and the conclusion uses 'DMG'). Please unify to a single abbreviation, preferably 'DMG' as defined in the title and abstract.","section":"Throughout"},{"comment":"The column headers in the ablation study ('w 1-pass', 'w SA', 'w 2-pass w/o SA') are cryptic and not explained in the text. Spell out what each configuration denotes (e.g., one pass through the ViM block, with versus without self-attention).","section":"Table III"},{"comment":"The baseline called 'RAFT' in the tables appears to be RAFT-Stereo, but the cited reference [13] is the optical-flow RAFT paper, and RAFT-Stereo is cited separately as [34]. The table should identify the exact method (RAFT-Stereo) and cite the corresponding paper in context.","section":"References, [13] and [34]"},{"comment":"The conclusion states that the proposed model 'was able to dissolve the gap in speed, accuracy and memory better than any other state-of-the-art techniques as per our proposed measure,' but the body text earlier acknowledges that on Sintel the proposed model does not achieve the best SOMER. Please qualify the conclusion to match the actual results.","section":"Section V, Conclusion"}],"recommendation":"reject","confidential_remarks":"The paper is likely an extended abstract or workshop-style submission, and the experimental section lacks the detail necessary to support the central claim. The internal inconsistency in Table I (SceneFlow EPE/D1) is a red flag that the reported numbers have not been carefully verified. The FPS comparability issue and the unvalidated SOMER metric are fundamental, not cosmetic, and would require re-running all baselines and rethinking the evaluation framework. The code release and ablation are positive aspects, but the current manuscript does not meet the bar for publication in a serious venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this one. First, it swaps the transformer blocks in a stereo matching pipeline for Vision Mamba blocks and adds a new composite metric, SOMER = FPS / (EPE × log M). That's a legitimate extension, and the code and models are released. Second, the central claim—that the model dissolves the speed/accuracy/memory trade-off—is not backed by the tables. On SceneFlow the proposed model's EPE is 4.4, roughly 6–10× worse than Unimatch and I-GEV, and on Sintel it's 11.53 versus RAFT's 0.45. On Sintel, even their own SOMER ranks RAFT higher. So the paper really describes a speed-for-accuracy trade, not a dissolution.\n\nWhat's genuinely useful: the ViM swapping is straightforward and the ablation gives some insight into the design choices. The SOMER metric, while simple, is at least an attempt to summarize the three-axis problem, and practitioners might find it handy if it's validated.\n\nThe soft spots are significant. The SceneFlow row reports EPE=4.4 with D1=0.18. With a 3-pixel threshold, that D1 means 99.82% of pixels are within 3px; the average error can't be 4.4 unless the remaining 0.18% have absurd errors (on the order of hundreds of pixels). That's implausible, and it makes the accuracy numbers unreliable. The FPS protocol is also underspecified: the paper says experiments were on an A6000, but doesn't say whether baselines were rerun on that machine, at what resolution, batch size, or precision, or with a consistent timing method. Since SOMER scales linearly with FPS, any discrepancy there changes every ranking. Finally, the baseline list omits most real-time stereo methods—no HITNET, StereoNet, or other efficient models—so the trade-off claim isn't tested against the relevant population.\n\nMy honest take: the paper is an incremental backbone application with a promising but unvalidated metric and an overreaching conclusion. It deserves a serious referee because the central speed claim is testable from the released code, and the idea of a Mamba-based stereo model is timely. But it needs major revision: fix or explain the EPE/D1 inconsistency, document the timing protocol completely, rerun baselines under identical conditions, and temper the conclusion. I'd send it out, but I'd also tell the authors that the trade-off claim has to be rebuilt from the ground up.","headline":"ViM-stereo is a credible engineering experiment, but the 'dissolved trade-off' claim is undercut by the paper's own accuracy numbers and an underspecified FPS benchmark.","tokens_in":9309,"tokens_out":3913,"would_cite":false,"duration_ms":32952,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A stereo disparity network built on Vision Mamba state space blocks claims to dissolve the speed-accuracy-memory trade-off, reporting the best combined SOMER score on three of four benchmarks.","keywords":["stereo disparity estimation","Vision Mamba","state space models","real-time inference","SOMER","memory footprint","stereo matching","edge deployment"],"falsifier":"Take the released checkpoints of Unimatch, I-GEV, RAFT-Stereo, and AnyNet, run them at the proposed model's native resolution and batch size on the same A6000 hardware, and compare FPS; if the proposed model's speed advantage over accurate baselines falls below its accuracy disadvantage, the claim that the trade-off is dissolved fails.","tokens_in":8316,"feed_emoji":"👁️","tokens_out":5165,"duration_ms":43459,"temperature":0.7,"pith_summary":"This paper tries to dissolve the long-standing trade-off in stereo disparity map generation among inference speed, accuracy, and memory use by replacing the transformer blocks of a stereo matching network with Vision Mamba state space units. The authors argue that this keeps the accuracy of attention-based matching while cutting compute and memory, and they introduce SOMER, a single score equal to FPS divided by endpoint error times the log of memory footprint, to rank models jointly. On KITTI, SceneFlow, and VKITTI2 their model reports the highest SOMER among the compared methods, though on Sintel the RAFT baseline scores higher. If the claim holds, real-time disparity estimation for robots and edge devices can run at about 50 FPS while staying accurate.","feed_headline":"Mamba stereo model outruns rivals on speed-accuracy-memory blend","feed_subtitle":"Vision Mamba and a new SOMER metric post the highest combined score on KITTI, SceneFlow, and VKITTI2.","key_machinery":"The load-bearing object is the Vision Mamba (ViM) block, a bidirectional selective state space model derived from Mamba, used as a drop-in replacement for transformer attention in a stereo matching encoder. Features from both images receive sine and cosine positional encodings, are flattened into tokens, and are concatenated in both orders, left-right and right-left, so that six ViM layers process them jointly; after splitting, the network performs depth-candidate matching and convex upsampling. The argument also rests on the proposed measure $\\text{SOMER} = \\frac{\\text{FPS}}{\\text{EPE} \\times \\log(M)}$, which turns speed, error, and memory into one ranking number.","core_discovery":"The central claim is that a disparity estimation network built on bidirectional Visual Mamba blocks, fed symmetrically concatenated left-right features, can match the accuracy of transformer-based stereo matchers while running much faster and using less memory. In the authors' measurements the model achieves 51.53 FPS on KITTI, 47.41 on SceneFlow, 52.53 on Sintel, and 50.62 on VKITTI2, with SOMER values of 6.409, 1.83, 0.785, and 7.644 respectively, the best among compared models except on Sintel. The authors further claim that existing metrics EPE and D1 ignore speed and memory, so SOMER gives a fairer overall comparison of disparity map generation methods.","pith_inferences":["If the FPS measurements were repeated with all baselines at identical resolution, batch size, and inference precision, the ranking could change; the paper does not report those settings for competing methods.","Because SOMER uses the logarithm of memory, memory differences beyond a few hundred megabytes matter little, and a metric using raw memory would rank the low-memory RAFT and AnyNet baselines higher.","The same symmetric concatenation plus ViM design could be tested on optical flow and monocular depth estimation, where a similar speed-accuracy trade-off exists.","At higher input resolutions the linear state space should widen its speed gap over quadratic attention, a testable prediction that goes beyond the paper's reported benchmarks."],"forward_implications":["Real-time robot vision could run disparity estimation at roughly 50 FPS with a memory footprint near 345 MB, making on-device deployment more plausible.","State space models can substitute for attention in stereo matching, pointing to linear rather than quadratic scaling at higher image resolutions.","Adopting SOMER would make inference speed and memory footprint explicit parts of disparity benchmark reporting rather than afterthoughts.","The network still trails the most accurate baselines on some EPE and D1 values, so the contribution is a balance shift across the three axes rather than a strict accuracy win."],"supporting_citations":[{"why":"Supplies the Vision Mamba block, the bidirectional state space backbone the paper substitutes for transformer attention.","marker":"[35]"},{"why":"Supplies the Mamba selective state space model whose linear-time scaling motivates the speed and memory claims.","marker":"[36]"},{"why":"The strongest accuracy baseline; its lower EPE on KITTI and Sintel defines the accuracy side of the trade-off the paper must match.","marker":"[15]"},{"why":"The RAFT baseline, recurrent and accurate but slow, used as a comparison in FPS and SOMER rankings.","marker":"[13]"},{"why":"The I-GEV baseline, the most accurate on SceneFlow and Sintel EPE in the comparison.","marker":"[14]"},{"why":"The AnyNet baseline, a real-time stereo network whose FPS is the speed target to exceed.","marker":"[26]"},{"why":"Provides the KITTI benchmark data used for evaluation.","marker":"[38]"},{"why":"Provides the SceneFlow dataset used for training and evaluation.","marker":"[5]"},{"why":"Provides the Sintel dataset where the proposed method does not reach the best SOMER.","marker":"[39]"},{"why":"Provides the Virtual KITTI 2 dataset used for training and evaluation.","marker":"[40]"}],"fun_headline_variants":["Visual Mamba matches transformer accuracy at higher speed","New SOMER metric ranks stereo models fairly","ViM-Disparity: fast, accurate, memory-light stereo","Mamba-based stereo nets beat speed-accuracy tradeoff","Bidirectional Mamba blocks speed up disparity maps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central trade-off claim assumes the published FPS numbers for all compared models were measured on the same hardware, at the same input resolution, with the same batch size and inference precision, but the paper only says FPS came from wall-clock time around the disparity function call, not that these settings were matched.","fun_headline_variants_meta":{"raw":{"variants":["Visual Mamba matches transformer accuracy at higher speed","New SOMER metric ranks stereo models fairly","ViM-Disparity: fast, accurate, memory-light stereo","Mamba-based stereo nets beat speed-accuracy tradeoff","Bidirectional Mamba blocks speed up disparity maps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000159,"raw_usage":{"total_tokens":1149,"prompt_tokens":784,"completion_tokens":365,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":400,"completion_tokens_details":{"reasoning_tokens":288}},"tokens_in":400,"tokens_out":365,"duration_ms":3758,"temperature":1.0,"reasoning_tokens":288,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:14:50.680850+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the released checkpoints of Unimatch, I-GEV, RAFT-Stereo, and AnyNet, run them at the proposed model's native resolution and batch size on the same A6000 hardware, and compare FPS; if the proposed model's speed advantage over accurate baselines falls below its accuracy disadvantage, the claim that the trade-off is dissolved fails.","supporting_citations":[{"cited_title":"Unifying flow, stereo and depth estimation,","cited_arxiv_id":null,"evidence_quote":"The strongest accuracy baseline; its lower EPE on KITTI and Sintel defines the accuracy side of the trade-off the paper must match."},{"cited_title":"Raft: Recurrent all-pairs field transforms for opti- cal flow,","cited_arxiv_id":null,"evidence_quote":"The RAFT baseline, recurrent and accurate but slow, used as a comparison in FPS and SOMER rankings."},{"cited_title":"Iterative geometry encoding volume for stereo matching,","cited_arxiv_id":null,"evidence_quote":"The I-GEV baseline, the most accurate on SceneFlow and Sintel EPE in the comparison."},{"cited_title":"Improvement of anynet- based end-to-end phased binocular stereo matching network,","cited_arxiv_id":null,"evidence_quote":"The AnyNet baseline, a real-time stereo network whose FPS is the speed target to exceed."},{"cited_title":"A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation,","cited_arxiv_id":null,"evidence_quote":"Provides the SceneFlow dataset used for training and evaluation."},{"cited_title":"A naturalistic open source movie for optical flow evaluation,","cited_arxiv_id":null,"evidence_quote":"Provides the Sintel dataset where the proposed method does not reach the best SOMER."}],"review_version":1}