REVIEW 4 major objections 5 minor 40 references
ViM-Disparity: Bridging the Gap of Speed, Accuracy and Memory for Disparity Map Generation
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read A stereo disparity network built on Vision Mamba state space blocks claims to dissolve the speed-accuracy-memory trade-off, reporting the best combined SOMER score on three of four benchmarks.
desk verdict ViM-stereo is a credible engineering experiment, but the 'dissolved trade-off' claim is undercut by the paper's own accuracy numbers and an underspecified FPS benchmark. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Vision Mamba (ViM) block, a bidirectional selective state space model derived from Mamba, used as a drop-in replacement for transformer attention in a stereo matching encoder. Features from both images receive sine and cosine positional encodings, are flattened into tokens, and are concatenated in both orders, left-right and right-left, so that six ViM layers process them jointly; after splitting, the network performs depth-candidate matching and convex upsampling. The argument also rests on the proposed measure $\text{SOMER} = \frac{\text{FPS}}{\text{EPE} \times \log(M)}$, which turns speed, error, and memory into one ranking number.
What would settle it
Take the released checkpoints of Unimatch, I-GEV, RAFT-Stereo, and AnyNet, run them at the proposed model's native resolution and batch size on the same A6000 hardware, and compare FPS; if the proposed model's speed advantage over accurate baselines falls below its accuracy disadvantage, the claim that the trade-off is dissolved fails.
Extended reading notes
Core claim
The central claim is that a disparity estimation network built on bidirectional Visual Mamba blocks, fed symmetrically concatenated left-right features, can match the accuracy of transformer-based stereo matchers while running much faster and using less memory. In the authors' measurements the model achieves 51.53 FPS on KITTI, 47.41 on SceneFlow, 52.53 on Sintel, and 50.62 on VKITTI2, with SOMER values of 6.409, 1.83, 0.785, and 7.644 respectively, the best among compared models except on Sintel. The authors further claim that existing metrics EPE and D1 ignore speed and memory, so SOMER gives a fairer overall comparison of disparity map generation methods.
Load-bearing premise
The central trade-off claim assumes the published FPS numbers for all compared models were measured on the same hardware, at the same input resolution, with the same batch size and inference precision, but the paper only says FPS came from wall-clock time around the disparity function call, not that these settings were matched.
Editorial extensions
If this is right
- Real-time robot vision could run disparity estimation at roughly 50 FPS with a memory footprint near 345 MB, making on-device deployment more plausible.
- State space models can substitute for attention in stereo matching, pointing to linear rather than quadratic scaling at higher image resolutions.
- Adopting SOMER would make inference speed and memory footprint explicit parts of disparity benchmark reporting rather than afterthoughts.
- The network still trails the most accurate baselines on some EPE and D1 values, so the contribution is a balance shift across the three axes rather than a strict accuracy win.
Reading between the lines
- If the FPS measurements were repeated with all baselines at identical resolution, batch size, and inference precision, the ranking could change; the paper does not report those settings for competing methods.
- Because SOMER uses the logarithm of memory, memory differences beyond a few hundred megabytes matter little, and a metric using raw memory would rank the low-memory RAFT and AnyNet baselines higher.
- The same symmetric concatenation plus ViM design could be tested on optical flow and monocular depth estimation, where a similar speed-accuracy trade-off exists.
- At higher input resolutions the linear state space should widen its speed gap over quadratic attention, a testable prediction that goes beyond the paper's reported benchmarks.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a Visual Mamba (ViM) based stereo disparity estimation architecture and a new composite metric, SOMER = FPS / (EPE × log(M)), intended to jointly evaluate speed, accuracy, and memory. The model is compared against Unimatch, I-GEV, AnyNet, and RAFT on KITTI, SceneFlow, Sintel, and VKITTI2. The authors report the highest SOMER on KITTI, SceneFlow, and VKITTI2, and claim that this 'dissolves' the trade-off between real-time inference, accuracy, and memory footprint for disparity map generation. The paper also includes an ablation study and releases code at a GitHub repository.
Significance. If the central claims were substantiated, the work would be a useful step for real-time stereo matching in robotics, and the open-source release is a practical contribution. The paper also honestly reports cases where it does not achieve the best EPE/D1 and includes an ablation study. However, the significance of the claimed trade-off dissolution is currently undermined by three load-bearing issues: the FPS measurements are not shown to be comparable across methods, the SOMER metric is introduced without external validation or principled justification, and the accuracy table contains an internal inconsistency (SceneFlow EPE/D1). Because the headline conclusion depends on these, the current evidence does not support the paper's main claim.
major comments (4)
- [IV, Table I] The FPS comparison is not controlled. Section IV states only that all experiments were performed on an Nvidia A6000 and that FPS is computed from the time difference before and after the disparity function call. It does not state that Unimatch, I-GEV, AnyNet, and RAFT were run on the same GPU, at the same input resolution, with the same batch size and numerical precision, using their official checkpoints, and with the same timing protocol. Since SOMER is linearly proportional to FPS (Eq. 5), a modest difference in timing conditions could alter the ranking. The authors must report the full measurement protocol and either rerun all baselines under identical conditions or clearly state the source of each baseline FPS number.
- [II-C, Eq. (5)] The SOMER metric is introduced as an ad-hoc composition of FPS, EPE, and log(M) without external validation or a derivation from deployment requirements. The choice of the reciprocal of EPE and the logarithm of memory is not justified, and no evidence is provided that SOMER correlates with practical suitability for real-time robotics. Moreover, the paper's own Table I shows that RAFT outperforms the proposed model on Sintel in SOMER (1.692 vs 0.785), so the claimed dissolution of the trade-off is not universal. The conclusion should be scoped to datasets where the proposed model wins, or the metric should be validated against established efficiency-accuracy trade-off analyses (e.g., Pareto fronts).
- [Table I, SceneFlow row] The reported SceneFlow values EPE=4.4 and D1=0.18 are internally inconsistent. D1 is the fraction of pixels with absolute disparity error greater than a threshold (typically 3 pixels). In a 1960×540 image, 0.18% of pixels is a tiny fraction; for the mean EPE to be 4.4, those outlier pixels would need errors averaging thousands of pixels, which is implausible for the disparity ranges in SceneFlow. This suggests an error in computing or reporting one or both metrics. Because the accuracy columns underpin the entire trade-off analysis, this inconsistency must be corrected and all EPE/D1 values verified before the paper's claims can be assessed.
- [IV, Implementation details] The model is trained only on VKITTI2 and SceneFlow, yet Table I reports results on Sintel and KITTI. The paper does not discuss how the model was adapted to these datasets, whether any fine-tuning was performed, or how domain shift affects the evaluation. The large EPE on Sintel (11.53 vs 0.45 for RAFT) suggests poor cross-domain generalization, which directly weakens the claim that the model is 'accurate' across the benchmarked datasets. The authors should clearly state the evaluation protocol for each dataset (zero-shot vs fine-tuned) and temper accuracy claims accordingly.
minor comments (5)
- [II-C, Eq. (5)] The formula as typeset (SOMER = FPS / EPE × log(M)) is ambiguous: it could be read as (FPS/EPE)·log(M). Add parentheses to make the intended denominator EPE·log(M) explicit.
- [Throughout] The abbreviations 'DMG' and 'DGM' are used inconsistently (e.g., Section II-C uses 'DGM' and the conclusion uses 'DMG'). Please unify to a single abbreviation, preferably 'DMG' as defined in the title and abstract.
- [Table III] The column headers in the ablation study ('w 1-pass', 'w SA', 'w 2-pass w/o SA') are cryptic and not explained in the text. Spell out what each configuration denotes (e.g., one pass through the ViM block, with versus without self-attention).
- [References, [13] and [34]] The baseline called 'RAFT' in the tables appears to be RAFT-Stereo, but the cited reference [13] is the optical-flow RAFT paper, and RAFT-Stereo is cited separately as [34]. The table should identify the exact method (RAFT-Stereo) and cite the corresponding paper in context.
- [Section V, Conclusion] The conclusion states that the proposed model 'was able to dissolve the gap in speed, accuracy and memory better than any other state-of-the-art techniques as per our proposed measure,' but the body text earlier acknowledges that on Sintel the proposed model does not achieve the best SOMER. Please qualify the conclusion to match the actual results.
Circularity Check
No significant circularity; SOMER is an explicitly defined evaluation metric and the trade-off claim is explicitly qualified as being measured by it.
full rationale
The paper does not contain a derivation chain in which a prediction is equivalent to a fitted input. The proposed SOMER metric (Eq. 5) is a definition chosen by the authors to combine FPS, EPE, and memory, and the conclusion that the model 'dissolves the trade-off' is explicitly stated as being 'as per our proposed measure for DMG' (Section V). This is an evaluation choice, not a circular derivation: the metric is not fitted to the model's outputs, nor is it defined in terms of the model's performance. The accuracy and speed numbers in Table I are independent measurements, regardless of whether they are comparable across baselines; comparability is a correctness/validity concern, not circularity. The self-citations (refs. [19] and [20]) appear only in related-work context and bear no load on the central claim. The ViM architecture is constructed from externally published components (Mamba, Vision Mamba) via citation, which is standard engineering reuse, not an ansatz smuggled in to guarantee the conclusion. No uniqueness theorem or prior work by the same authors is invoked to forbid alternatives. Thus no circular step can be exhibited from the paper's own equations or citations.
Assumptions & free parameters
assumptions (4)
- domain assumption Vision Mamba blocks can substitute transformer blocks in a stereo matching pipeline without unacceptable accuracy loss.
- domain assumption The comparison baselines were run under conditions comparable to the proposed model, including hardware, input size, and inference framework.
- domain assumption Training on Virtual KITTI 2 and SceneFlow transfers to KITTI and Sintel without dataset-specific fine-tuning.
- ad hoc to paper SOMER = FPS / (EPE * log(M)) is a valid and meaningful joint measure of DMG model quality.
Cite this review
Pith. "Pith review of ViM-Disparity: Bridging the Gap of Speed, Accuracy and Memory for Disparity Map Generation." pith.science (2026). https://pith.science/paper/7F3LD4GT
@misc{pith2026241216745,
author = {Pith},
title = {Pith review of: ViM-Disparity: Bridging the Gap of Speed, Accuracy and Memory for Disparity Map Generation},
year = {2026},
howpublished = {\url{https://pith.science/paper/7F3LD4GT}},
note = {Machine review of arXiv:2412.16745}
}
read the original abstract
In this work we propose a Visual Mamba (ViM) based architecture, to dissolve the existing trade-off for real-time and accurate model with low computation overhead for disparity map generation (DMG). Moreover, we proposed a performance measure that can jointly evaluate the inference speed, computation overhead and the accurateness of a DMG model. The code implementation and corresponding models are available at: https://github.com/MBora/ViM-Disparity.
Figures
Reference graph
Works this paper leans on
-
[1]
Literature survey on stereo vision disparity map algorithms,
R. A. Hamzah, H. Ibrahim et al. , “Literature survey on stereo vision disparity map algorithms,” Journal of Sensors , vol. 2016, 2016
work page 2016
-
[2]
A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,
D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” International journal of computer vision , vol. 47, pp. 7–42, 2002
work page 2002
-
[3]
Learning for disparity estimation through feature constancy,
Z. Liang, Y . Feng, Y . Guo, H. Liu, W. Chen, L. Qiao, L. Zhou, and J. Zhang, “Learning for disparity estimation through feature constancy,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2811–2820
work page 2018
-
[4]
Stereo matching algorithm based on deep learning: A survey,
M. S. Hamid, N. Abd Manap, R. A. Hamzah, and A. F. Kadmin, “Stereo matching algorithm based on deep learning: A survey,” Journal of King Saud University-Computer and Information Sciences , vol. 34, no. 5, pp. 1663–1673, 2022
work page 2022
-
[5]
N. Mayer, E. Ilg, P. Hausser, P. Fischer, D. Cremers, A. Dosovitskiy, and T. Brox, “A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 4040– 4048
work page 2016
-
[6]
Pyramid stereo matching network,
J.-R. Chang and Y .-S. Chen, “Pyramid stereo matching network,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 5410–5418
2018
-
[7]
Stereonet: Guided hierarchical refinement for real-time edge- aware depth prediction,
S. Khamis, S. Fanello, C. Rhemann, A. Kowdle, J. Valentin, and S. Izadi, “Stereonet: Guided hierarchical refinement for real-time edge- aware depth prediction,” in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 573–590
work page 2018
-
[8]
Edgestereo: A context integrated residual pyramid network for stereo matching,
X. Song, X. Zhao, H. Hu, and L. Fang, “Edgestereo: A context integrated residual pyramid network for stereo matching,” in Computer Vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part V 14 . Springer, 2019, pp. 20–35
work page 2018
Show all 40 references
-
[9]
Anytime stereo image depth estimation on mobile devices,
Y . Wang, Z. Lai, G. Huang, B. H. Wang, L. Van Der Maaten, M. Camp- bell, and K. Q. Weinberger, “Anytime stereo image depth estimation on mobile devices,” in 2019 international conference on robotics and automation (ICRA) . IEEE, 2019, pp. 5893–5900
2019
-
[10]
Pcw-net: Pyramid combination and warping cost volume for stereo matching,
Z. Shen, Y . Dai, X. Song, Z. Rao, D. Zhou, and L. Zhang, “Pcw-net: Pyramid combination and warping cost volume for stereo matching,” in European conference on computer vision . Springer, 2022, pp. 280–297
2022
-
[11]
Digging into uncertainty in self-supervised multi-view stereo,
H. Xu, Z. Zhou, Y . Wang, W. Kang, B. Sun, H. Li, and Y . Qiao, “Digging into uncertainty in self-supervised multi-view stereo,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 6078–6087
2021
-
[12]
H-net: Un- supervised attention-based stereo depth estimation leveraging epipolar geometry,
B. Huang, J.-Q. Zheng, S. Giannarou, and D. S. Elson, “H-net: Un- supervised attention-based stereo depth estimation leveraging epipolar geometry,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 4460–4467
2022
-
[13]
Raft: Recurrent all-pairs field transforms for opti- cal flow,
Z. Teed and J. Deng, “Raft: Recurrent all-pairs field transforms for opti- cal flow,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16 . Springer, 2020, pp. 402–419
2020
-
[14]
Iterative geometry encoding volume for stereo matching,
G. Xu, X. Wang, X. Ding, and X. Yang, “Iterative geometry encoding volume for stereo matching,” in Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , 2023, pp. 21 919– 21 928
2023
-
[15]
Unifying flow, stereo and depth estimation,
H. Xu, J. Zhang, J. Cai, H. Rezatofighi, F. Yu, D. Tao, and A. Geiger, “Unifying flow, stereo and depth estimation,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023
2023
-
[16]
Deep learning for monocular depth estimation: A review,
Y . Ming, X. Meng, C. Fan, and H. Yu, “Deep learning for monocular depth estimation: A review,”Neurocomputing, vol. 438, pp. 14–33, 2021
2021
-
[17]
Monocular depth estimation using deep learning: A review,
A. Masoumian, H. A. Rashwan, J. Cristiano, M. S. Asif, and D. Puig, “Monocular depth estimation using deep learning: A review,” Sensors, vol. 22, no. 14, p. 5353, 2022
2022
-
[18]
Seeing in 3-d with just one eye: Stereopsis without binocular vision,
D. Vishwanath and P. B. Hibbard, “Seeing in 3-d with just one eye: Stereopsis without binocular vision,” Psychological science , vol. 24, no. 9, pp. 1673–1685, 2013
2013
-
[19]
Recent advancement in 3d biometrics using monocular camera,
A. Mukherjee and A. Das, “Recent advancement in 3d biometrics using monocular camera,” in 2023 IEEE International Joint Conference on Biometrics (IJCB) . IEEE, 2023, pp. 1–14
2023
-
[20]
Depth-guided robust face morphing attack detection,
H. Rachalwar, M. Fang, N. Damer, and A. Das, “Depth-guided robust face morphing attack detection,” in 2023 IEEE International Joint Conference on Biometrics (IJCB) . IEEE, 2023, pp. 1–9
2023
-
[21]
Learning monocular depth estimation infusing traditional stereo knowledge,
F. Tosi, F. Aleotti, M. Poggi, and S. Mattoccia, “Learning monocular depth estimation infusing traditional stereo knowledge,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2019, pp. 9799–9809
2019
-
[22]
Efficient disparity map generation for moving multi-camera system using tof depth sensor,
Y .-S. Kang and Y .-S. Ho, “Efficient disparity map generation for moving multi-camera system using tof depth sensor,” in 2013 20th International Conference on Systems, Signals and Image Processing (IWSSIP) , 2013, pp. 67–70
2013
-
[23]
Monocular vision with deep neural networks for autonomous mobile robots navigation,
W. K. Sleaman, A. A. Hameed, and A. Jamil, “Monocular vision with deep neural networks for autonomous mobile robots navigation,” Optik, vol. 272, p. 170162, 2023
2023
-
[24]
An image is worth 16x16 words: Transformers for image recognition at scale,
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al. , “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020
2010 arXiv
-
[25]
Attention concatenation volume for accurate and efficient stereo matching,
G. Xu, J. Cheng, P. Guo, and X. Yang, “Attention concatenation volume for accurate and efficient stereo matching,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 12 981–12 990
2022
-
[26]
Improvement of anynet- based end-to-end phased binocular stereo matching network,
S. Chen, D. Ergu, B. Ma, Y . Cai, and F. Liu, “Improvement of anynet- based end-to-end phased binocular stereo matching network,” Procedia Computer Science , vol. 199, pp. 1450–1457, 2022
2022
-
[27]
Real-time stereo matching network based on 3d channel and disparity attention for edge devices toward autonomous driving,
B. Liang, H. Yang, J. Huang, C. Liu, and R. Yang, “Real-time stereo matching network based on 3d channel and disparity attention for edge devices toward autonomous driving,” IEEE Access, vol. 11, pp. 76 781– 76 792, 2023
2023
-
[28]
Distill-then-prune: An efficient compression framework for real-time stereo matching network on edge devices,
B. Pan, J. Jiao, J. Pang, and J. Cheng, “Distill-then-prune: An efficient compression framework for real-time stereo matching network on edge devices,” 2024. [Online]. Available: https://arxiv.org/abs/2405.11809
2024 arXiv
-
[29]
Hierarchical deep stereo matching on high-resolution images,
G. Yang, J. Manela, M. Happold, and D. Ramanan, “Hierarchical deep stereo matching on high-resolution images,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 5515–5524
2019
-
[30]
Learning unsupervised multi-view stereopsis via robust photometric consistency,
T. Khot, S. Agrawal, S. Tulsiani, C. Mertz, S. Lucey, and M. Hebert, “Learning unsupervised multi-view stereopsis via robust photometric consistency,” arXiv preprint arXiv:1905.02706 , 2019
1905 arXiv
-
[31]
Mvs2: Deep unsupervised multi-view stereo with multi-view symmetry,
Y . Dai, Z. Zhu, Z. Rao, and B. Li, “Mvs2: Deep unsupervised multi-view stereo with multi-view symmetry,” in 2019 International Conference on 3D Vision (3DV) . Ieee, 2019, pp. 1–8
2019
-
[32]
Progressive fusion for unsupervised binocular depth estimation using cycled networks,
A. Pilzer, S. Lathuili `ere, D. Xu, M. M. Puscas, E. Ricci, and N. Sebe, “Progressive fusion for unsupervised binocular depth estimation using cycled networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 10, pp. 2380–2395, 2019
2019
-
[33]
Hitnet: Hierarchical iterative tile refinement network for real-time stereo matching,
V . Tankovich, C. Hane, Y . Zhang, A. Kowdle, S. Fanello, and S. Bouaziz, “Hitnet: Hierarchical iterative tile refinement network for real-time stereo matching,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 14 362–14 372
2021
-
[34]
Raft-stereo: Multilevel recurrent field transforms for stereo matching,
L. Lipson, Z. Teed, and J. Deng, “Raft-stereo: Multilevel recurrent field transforms for stereo matching,” in 2021 International Conference on 3D Vision (3DV) . IEEE, 2021, pp. 218–227
2021
-
[35]
Vision mamba: Efficient visual representation learning with bidirectional state space model,
L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang, “Vision mamba: Efficient visual representation learning with bidirectional state space model,” arXiv preprint arXiv:2401.09417 , 2024
2024 arXiv
-
[36]
Mamba: Linear-time sequence modeling with selective state spaces,
A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2023
2023 arXiv
-
[37]
Mambavision: A hybrid mamba- transformer vision backbone,
A. Hatamizadeh and J. Kautz, “Mambavision: A hybrid mamba- transformer vision backbone,” 2024. [Online]. Available: https: //arxiv.org/abs/2407.08083
2024 arXiv
-
[38]
Vision meets robotics: The kitti dataset,
A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, “Vision meets robotics: The kitti dataset,” The International Journal of Robotics Research , vol. 32, no. 11, pp. 1231–1237, 2013
2013
-
[39]
A naturalistic open source movie for optical flow evaluation,
D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black, “A naturalistic open source movie for optical flow evaluation,” in Computer Vision– ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part VI 12 . Springer, 2012, pp...
2012
-
[40]
Virtual kitti 2,
Y . Cabon, N. Murray, and M. Humenberger, “Virtual kitti 2,” arXiv preprint arXiv:2001.10773, 2020
2001 arXiv
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.