Pith. sign in

REVIEW 4 major objections 6 minor 28 references

DepthMamba with Adaptive Fusion

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that fusing single-view and multi-view depth estimates with an attention-based adaptive mechanism and a shared mamba backbone produces depth maps that stay accurate even when camera poses are noisy, reporting AbsRel 0.042…

desk verdict Announces a noisy-pose robustness benchmark that never appears in the experiments; the clean-pose numbers are competitive but unverifiable. read the letter →

arxiv 2412.19964 v1 pith:IKBHJI3U submitted 2024-12-28 cs.CV cs.AI

Zelin Meng , Zhichen Wang This is my paper · ORCID
classification cs.CVcs.AI
keywords multi-viewdepthestimationmonocularsingle-viewandfusionadaptiveattentionMambabackbonenoisycameraposesrobustnessbenchmarkautonomousdriving
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that multi-view depth estimation can be made robust to inaccurate camera poses by fusing a single-view depth branch with a multi-view depth branch instead of trusting either one alone. The proposed two-branch network shares a mamba-based feature extraction backbone across both branches, builds a group-wise correlation volume and a variance volume from the extracted features, and uses an attention-based adaptive fusion module to weight the two branches per region. On the KITTI Eigen split the method reports an AbsRel of 0.042 and RMSE of 1.695, and on DDAD an RMSE of 6.12, improving RMSE by 2.8% over AFNet on KITTI and by 15.3% on DDAD. The authors also introduce a robustness benchmark with noisy pose settings and report that existing single-view, multi-view, and fusion methods degrade sharply under it.

What carries the argument

The load-bearing mechanism is the adaptive fusion of two cost volumes: the multi-view branch produces a group-wise correlation volume from reference and source features, while the variance volume is constructed from the same features; a multi-scale attention module followed by a 3D hourglass network turns the correlation volume into attention weights, and the element-wise multiplication of variance volume with those weights yields the attention volume used for depth regression. The feature extraction backbone is a "depth-mamba" variant of the Mamba selective state-space sequence model adapted to image feature extraction, augmented with local feature blocks to compensate for mamba's weaker spatial localization. This mechanism is what lets the network choose the more reliable branch per location.

What would settle it

Train the exact described architecture from scratch on the KITTI Eigen split with the stated recipe (40 epochs, AdamW, one-cycle schedule, 128 depth hypotheses, three input frames) and check whether it reaches AbsRel 0.042 and RMSE 1.695 on KITTI and RMSE 6.12 on DDAD; if the published recipe cannot reproduce these numbers, or if swapping the attention-weighted fusion for concatenation does not produce the claimed 33.3% AbsRel gap, the central claim fails.

Watch

Extended reading notes

Core claim

The central claim is that the reliability of depth estimates can be improved by adaptively selecting between single-view and multi-view evidence rather than merging them blindly. The paper proposes a two-branch architecture in which a shared depth-mamba backbone extracts multi-scale features from the reference and source images; a multi-view branch builds a group-wise correlation volume, a single-view branch builds a variance volume, and a multi-scale attention module plus a 3D hourglass network produces attention weights that are multiplied onto the variance volume to form the final attention volume. Depth is then obtained by residual regression from that volume. On this design the paper reports AbsRel 0.042 on KITTI and RMSE 6.12 on DDAD, and it attributes the gain to the mamba backbone and to the attention-based fusion, with ablations showing the fusion module lowers AbsRel by 33.3% versus simple concatenation and by 23.6% versus cross-attention. The paper further claims these design choices keep the system accurate under noisy camera poses, a setting where current methods fail.

Load-bearing premise

The entire experimental result rests on the assumption that the network described only in prose—shared depth-mamba backbone, group-wise correlation volume, variance volume, 3D hourglass, and attention-weighted multiplication—was implemented exactly as described and trained with the stated recipe, because no equations, layer dimensions, or branch-specific losses are provided to pin down the architecture.

Editorial extensions

If this is right

  • Multi-view depth systems could be deployed in autonomous driving without relying on highly accurate pose estimates, since the adaptive fusion keeps the single-view branch as a fallback where geometry is unreliable.
  • The same two-branch fusion principle can be applied to other multi-modal estimation tasks where one modality is robust and another is precise but brittle.
  • The reported mamba backbone with local feature blocks offers a practical alternative to CNN and transformer backbones for stereo and multi-view depth networks.
  • The noisy-pose robustness benchmark introduced in the paper gives a standard way to compare depth estimators under realistic pose errors rather than only ideal poses.
  • If the reported numbers hold, the method sets a new accuracy bar on the KITTI Eigen split with AbsRel 0.042 and on DDAD with RMSE 6.12.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is to visualize the learned attention weights directly: if the fusion is truly adaptive, the weights should be low in occluded or textureless regions where multi-view matching is unreliable.
  • The robustness benchmark could be extended to perturb camera intrinsics as well as extrinsics, probing sensitivity to calibration errors beyond pose noise.
  • Because the architecture description lacks implementation specifics, independent replication would currently require reverse-engineering the described modules; releasing code and exact layer configurations would let the community verify and build on the claimed improvements.
  • The adaptive-fusion idea may transfer to fusing learned depth with LiDAR or radar depth, where the same tension exists between dense monocular cues and sparse but accurate geometric measurements.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a two-branch depth estimation network for fusing single-view and multi-view cues, built around a Depth-Mamba feature extraction backbone and an attention-based adaptive fusion module. The authors claim that existing multi-view and fusion methods fail under noisy camera poses, introduce a new robustness benchmark for this setting, and report competitive results on KITTI and DDAD, including an AbsRel of 0.042 on KITTI and an RMSE of 6.12 on DDAD. Ablation studies on the backbone and fusion module are also reported. The central noisy-pose robustness claim, however, is never experimentally supported, and the method itself is described only at a high level without mathematical or architectural specificity.

Significance. If the claims were substantiated, robustness to imperfect camera poses would be a practically important contribution for autonomous driving, and the use of a mamba-based backbone with adaptive single-view/multi-view fusion could be of interest to the multi-view depth estimation community. The paper also deserves credit for reporting evaluations on two public benchmarks and for including ablations of the backbone and fusion choices. As written, however, the announced robustness benchmark is not defined or executed, the architecture is described only in prose, and the main comparison table has unverifiable baseline citations. These issues currently prevent the results from being assessed or reproduced, so the paper's scientific contribution is not yet established.

major comments (4)
  1. [Abstract and Section 4.4] The central claim of the paper is robustness to noisy camera poses, but no noisy-pose experiment appears anywhere. The abstract announces a new robustness benchmark and states that current methods fail under noisy poses, and Section 4.4 repeats that current methods fail, yet there is no definition of the benchmark, no noise model, no perturbation magnitudes, no evaluation protocol, and no table or figure with noisy-pose results. Tables 1-3 all report clean-pose results on the standard KITTI and DDAD splits. Because the proposed adaptive fusion is motivated specifically as selecting the more robust branch under pose noise, the paper's main differentiating claim is untestable as submitted.
  2. [Section 3] The proposed method is described only in prose. Section 3.2 states that a group-wise correlation volume and a variance volume are constructed and that the variance volume is multiplied by attention weights, but no equations define these volumes, the attention module, the 3D hourglass network, or the depth regression. Section 3.1 does not give layer counts, channels, resolutions, or the structure of the Depth-Mamba and Pyramid Mamba modules, and Section 3.4 provides only the final MAE loss without specifying whether branch-level losses are applied to the single-view and multi-view outputs. Without these details, the link between the described architecture and the numbers in Table 1 cannot be verified, and the implementation is not reproducible.
  3. [Table 1 and references] The main comparison is compromised by inconsistent sourcing. The text calls AFNet the current state-of-the-art and cites it as [2], but reference [2] is AdaBins, and no reference for AFNet appears in the bibliography. The asterisks attached to many entries in Table 1 are never explained, making it unclear which results were produced by the authors and which were taken from other papers. In addition, Section 4.5.1 says the backbone ablation compares ConvNeXt-T, VMamba, and Depth-Mamba, but Table 2 lists only Base-Mamba, Base-VMamba, and Base-DepthMamba, with no ConvNeXt-T row. These issues prevent the reader from validating either the state-of-the-art comparison or the claimed ablations.
  4. [Section 4.4 and Table 1] The DDAD performance claim is stated selectively. The text reports an RMSE improvement of 15.3% over AFNet and MaGNet, but in Table 1 the proposed method has DDAD AbsRel 0.107, which is worse than AFNet's 0.088, and SqRel 1.49, which is also worse than AFNet's 1.41. The 15.3% figure applies only to RMSE relative to AFNet, not to MaGNet, where the relative RMSE improvement is about 33.7%. The overall claim of competitive performance on DDAD therefore depends on a single metric and omits the fact that the method is worse on two of the three reported error measures.
minor comments (6)
  1. [Section 4.1] Reference [10] is cited for the DDAD dataset, but [10] is the Monodepth2 paper by Godard et al.; the DDAD dataset should be cited to its original source (Guizilini et al., 2020).
  2. [Equation (1)] The notation in the loss function is inconsistent with the metrics: Eq. (1) uses y_i and y_i^hat, while Eqs. (2)-(4) use y_i and y_i^pred; please unify the notation.
  3. [Related Works] The text 'Building on this, [MVSNet] pioneered...' uses an unnumbered citation placeholder; MVSNet should be given a numbered reference consistent with the bibliography.
  4. [Section 4.3] The implementation details omit several settings needed for reproducibility, including the depth range, the source-view selection and baseline distances, the input resolution for training and testing, and the number of training samples used from each camera in DDAD.
  5. [Section 4.5] The phrase 'Ablation studies prove the effectiveness' overstates what a single-dataset ablation can establish; 'support' or 'demonstrate' would be more appropriate.
  6. [Conclusions] There is a typo in the Conclusions section: 'The ablation studies certificate the effectiveness' should be 'certify the effectiveness' or 'demonstrate the effectiveness.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: reported metrics are external test-set measurements with no self-citation or fitted-input predictions.

full rationale

The paper is an empirical training/evaluation study; the load-bearing reported results (AbsRel 0.042, RMSE 1.695 on KITTI; RMSE 6.12 on DDAD in Table 1, plus ablations in Tables 2 and 3) are test-set measurements, not quantities obtained by fitting a parameter to a subset of data and then predicting a closely related quantity. No equation in the paper defines one reported quantity in terms of another by construction: the loss in Eq. (1) and the evaluation metrics in Eqs. (2)-(5) are standard external definitions, and the stated improvement percentages (2.8%, 15.3%, 27.5%, 22.2%, 33.3%, 23.63%) are arithmetic comparisons to independent baseline table entries, not fitted inputs renamed as predictions. The architecture sections are described in prose, so there is no derivation chain that could reduce to its own inputs. The paper contains no self-citations: all references point to external prior work (Mamba, VMamba, KITTI, DDAD, AFNet, etc.), and no uniqueness theorem or prior result by the same authors is invoked to force a choice. A separate concern is that the announced 'new robustness benchmark' for noisy camera poses is never defined or run, and the claim that current methods fail under noisy poses is asserted without supporting experiments; this is a serious empirical-support gap and a correctness risk, but it is not circular reasoning. Accordingly, the central derivation is self-contained with respect to circularity, and the score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 4 invented entities

The central claim depends on a fully specified but absent implementation: a new mamba-based backbone and an attention-based fusion of single-view and multi-view branches. All listed entities and assumptions are taken from the paper's prose without independent or formal support.

free parameters (3)
  • number of depth hypothesis planes = 128
    Hand-chosen number of depth bins for the cost volume; directly controls the disparity search range and affects accuracy (Section 4.3).
  • number of input frames = 3
    Multi-view matching uses three consecutive frames; this choice changes the geometry available to the multi-view branch (Section 4.3).
  • maximum learning rate = 1e-4
    AdamW with one-cycle schedule; standard but hand-set and part of the training recipe that produces the reported numbers (Section 4.3).
assumptions (4)
  • domain assumption Multi-view depth can be estimated from a cost volume built by differentiable warping of features, assuming known camera intrinsics and poses.
    The multi-view branch inherits the standard MVS cost-volume paradigm (MVSNet etc.) without derivation or discussion of failure modes.
  • domain assumption Mamba/VMamba can serve as a visual feature extractor for dense prediction tasks such as depth estimation.
    The paper adopts VMamba as a backbone and assumes it transfers from image classification to depth regression; no analysis is given.
  • domain assumption The baseline numbers in Table 1 are accurate and were obtained under comparable training and evaluation protocols.
    Many entries carry an unexplained asterisk and there is no description of how baselines were trained on DDAD; the comparison rests on this assumption.
  • domain assumption KITTI Eigen split and DDAD LiDAR depth provide reliable ground truth for supervised training and evaluation.
    The datasets are introduced briefly and their depth quality and preprocessing are taken for granted.
invented entities (4)
  • Depth-Mamba backbone
    purpose: Feature extraction backbone for both branches, combining Mamba with local feature blocks to improve spatial detail.
    No equations, tensor shapes, or ablations beyond final accuracy are given; the only evidence is the paper's own tables.
  • Pyramid Mamba module
    purpose: Extracts multi-scale features at different input resolutions for cost volume construction.
    Mentioned in the Introduction and Figure 1 but never defined or evaluated in isolation.
  • Adaptive Fusion module
    purpose: Computes attention weights to fuse the variance volume from the two branches.
    Its operation is described only as multiplying variance volume by attention weights (Section 3.2); no equation or standalone validation exists.
  • Attention volume and variance volume
    purpose: Intermediate tensors that are multiplied together before depth regression.
    Neither tensor is defined mathematically; their construction is the crux of the claimed contribution but is left as prose.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DepthMamba with Adaptive Fusion." pith.science (2026). https://pith.science/paper/IKBHJI3U

@misc{pith2026241219964,
  author       = {Pith},
  title        = {Pith review of: DepthMamba with Adaptive Fusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IKBHJI3U}},
  note         = {Machine review of arXiv:2412.19964}
}
read the original abstract

Multi-view depth estimation has achieved impressive performance over various benchmarks. However, almost all current multi-view systems rely on given ideal camera poses, which are unavailable in many real-world scenarios, such as autonomous driving. In this work, we propose a new robustness benchmark to evaluate the depth estimation system under various noisy pose settings. Surprisingly, we find current multi-view depth estimation methods or single-view and multi-view fusion methods will fail when given noisy pose settings. To tackle this challenge, we propose a two-branch network architecture which fuses the depth estimation results of single-view and multi-view branch. In specific, we introduced mamba to serve as feature extraction backbone and propose an attention-based fusion methods which adaptively select the most robust estimation results between the two branches. Thus, the proposed method can perform well on some challenging scenes including dynamic objects, texture-less regions, etc. Ablation studies prove the effectiveness of the backbone and fusion method, while evaluation experiments on challenging benchmarks (KITTI and DDAD) show that the proposed method achieves a competitive performance compared to the state-of-the-art methods.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 23 canonical work pages

  1. [2]

    Adabins: Depth estimation using adaptive bins

    Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. In CVPR, 2021. 2, 6, 8

    work page 2021

  2. [1]

    Multiview depth estimation by fusing single-view depth probability with multi-view geometry

    Gwangbin Bae, Ignas Budvytis, and Roberto Cipolla. Multiview depth estimation by fusing single-view depth probability with multi-view geometry. In CVPR, 2022. 2, 3, 5, 6, 7, 8

    work page 2022

  3. [3]

    Proposed Methods The primary works of the proposed method is to perform depth estimation from images using a multi-view stereo approach within a neural network framework. The forward pass extracts features from both a reference image and multiple source images, constructs a 3D volume representation based on these features, and calculates depth predictions...

    work page

  4. [4]

    Two prominent datasets in this field are the KITTI and DDAD datasets, each providing valuable resources for developing and benchmarking stereo vision systems

    Experiments 4.1 Dataset In stereo depth estimation task, high -quality datasets are significantly essential for training and evaluating depth estimation models. Two prominent datasets in this field are the KITTI and DDAD datasets, each providing valuable resources for developing and benchmarking stereo vision systems. The KITTI dataset, introduced by the ...

    work page 1936

  5. [5]

    Point -based multi-view stereo network

    Rui Chen, Songfang Han, Jing Xu, and Hao Su. Point -based multi-view stereo network. In ICCV , 2019. 2

    work page 2019

  6. [6]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Yang L, Kang B, Huang Z, et al. Depth anything: Unleashing the power of large-scale unlabeled data. In CVPR, 2024.2

    work page 2024

  7. [7]

    Repurposing diffusion-based image generators for monocular depth estimation

    Ke B, Obukhov A, Huang S, et al. Repurposing diffusion-based image generators for monocular depth estimation. In CVPR, 2024.2

    work page 2024

  8. [8]

    Depth map prediction from a single image using a multi-scale deep network

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep network. NeurIPS, 2014. 4, 5

    work page 2014

Show all 28 references
  1. [9]

    Single-view and multi-view depth fusion

    Jose M Facil, Alejo Concha, Luis Montesano, and Javier Civera. Single-view and multi-view depth fusion. RA-L, 2017. 2, 3

    2017

  2. [10]

    Deep ordinal regression network for monocular depth estimation

    Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, and Dacheng Tao. Deep ordinal regression network for monocular depth estimation. In CVPR, 2018. 2, 6

    2018

  3. [11]

    Vision meets robotics: The kitti dataset

    Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. IJRR, 2013. 2, 4, 5, 6, 7, 8

    2013

  4. [12]

    Digging into self-supervised monocular depth estimation

    Clement Godard, Oisin Mac Aodha, Michael Firman, and Gabriel J Brostow. Digging into self-supervised monocular depth estimation. In ICCV , 2019. 1, 2, 4, 5, 6, 7, 8

    2019

  5. [13]

    Cascade cost volume for high-resolution multi-view stereo and stereo matching

    Xiaodong Gu, Zhiwen Fan, Siyu Zhu, Zuozhuo Dai, Feitong Tan, and Ping Tan. Cascade cost volume for high-resolution multi-view stereo and stereo matching. In CVPR,

  6. [14]

    From big to small: Multi-scale local planar guidance for monocular depth estimation

    Jin Han Lee, Myung-Kyu Han, Dong Wook Ko, and Il Hong Suh. From big to small: Multi-scale local planar guidance for monocular depth estimation. arXiv:1907.10326,

    1907 arXiv

  7. [15]

    Multi- view depth estimation using epipolar spatio-temporal networks

    Xiaoxiao Long, Lingjie Liu, Wei Li, Christian Theobalt, and Wenping Wang. Multi- view depth estimation using epipolar spatio-temporal networks. In CVPR, 2021. 2

    2021

  8. [16]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 5

    2017 arXiv

  9. [17]

    Multi -level context ultra-aggregation for stereo matching

    Guang-Yu Nie, Ming-Ming Cheng, Yun Liu, Zhengfa Liang, Deng-Ping Fan, Yue Liu, and Yongtian Wang. Multi -level context ultra-aggregation for stereo matching. In CVPR, 2019. 2

    2019

  10. [18]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019. 5

    2019

  11. [19]

    Feature-metric loss for self- supervised learning of depth and egomotion

    Chang Shu, Kun Yu, Zhixiang Duan, and Kuiyuan Yang. Feature-metric loss for self- supervised learning of depth and egomotion. In ECCV , 2020. 6

    2020

  12. [20]

    Deepv2d: Video to depth with differentiable structure from motion

    Zachary Teed and Jia Deng. Deepv2d: Video to depth with differentiable structure from motion. arXiv preprint arXiv:1812.04605, 2018. 6

    2018 arXiv

  13. [21]

    Patchmatchnet: Learned multi-view patchmatch stereo

    Fangjinhua Wang, Silvano Galliani, Christoph V ogel, Pablo Speciale, and Marc Pollefeys. Patchmatchnet: Learned multi-view patchmatch stereo. In CVPR, 2021. 6, 7, 8

    2021

  14. [22]

    Itermvs: Iterative probability estimation for efficient multi-view stereo

    Fangjinhua Wang, Silvano Galliani, Christoph V ogel, and Marc Pollefeys. Itermvs: Iterative probability estimation for efficient multi-view stereo. In CVPR, 2022. 5, 6, 7, 8, 2

    2022

  15. [23]

    Mvs2d: Efficient multi-view stereo via attention-driven 2d convolutions

    Zhenpei Yang, Zhile Ren, Qi Shan, and Qixing Huang. Mvs2d: Efficient multi-view stereo via attention-driven 2d convolutions. In CVPR, 2022. 3, 5, 6, 7, 8, 1, 2

    2022

  16. [24]

    Metric3d: Towards zero-shot metric 3d prediction from a single image

    Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from a single image. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9043–9053, 2023. 6

    2023

  17. [25]

    Fast -mvsnet: Sparse-todense multi-view stereo with learned propagation and gaussnewton refinement

    Zehao Yu and Shenghua Gao. Fast -mvsnet: Sparse-todense multi-view stereo with learned propagation and gaussnewton refinement. In CVPR, 2020. 2

    2020

  18. [26]

    Computing the stereo matching cost with a convolutional neural network

    Jure Zbontar and Yann LeCun. Computing the stereo matching cost with a convolutional neural network. In CVPR, 2015. 2

    2015

  19. [27]

    Mamba: Linear -time sequence modeling with selective state spaces

    Gu A, Dao T. Mamba: Linear -time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023. 12

    2023 arXiv

  20. [28]

    VMamba: Visual State Space Model

    Yue Liu, Yunjie Tian . VMamba: Visual State Space Model . arXiv preprint arXiv:2401.10166, 2024. 1

    2024 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.