Pith. sign in

REVIEW 3 major objections 7 minor 64 references

DepthSync: Diffusion Guidance-Based Depth Synchronization for Scale- and Geometry-Consistent Video Depth Estimation

T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DepthSync adds two guidance signals to the denoising loop, cross-window scale synchronization and intra-window geometric alignment, improving depth accuracy and consistency on long videos without retraining.

desk verdict A genuinely useful training-free guidance for long-video depth scale consistency, but the geometry-guidance PnP self-loop needs one more ablation before I'd fully trust the pose gains. read the letter →

arxiv 2507.01603 v2 pith:UVERKSNG submitted 2025-07-02 cs.CV

Yue-Jiang Dong , Wang Zhao , Jiale Xu , Ying Shan , Song-Hai Zhang This is my paper
classification cs.CV
keywords videodepthestimationdiffusionguidancelongconsistencyslidingwindowscalesynchronizationgeometryalignmenttraining-freeinference3Dreconstructionmulti-frame
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DepthSync aims to show that the two persistent failure modes of diffusion-based video depth estimation on long videos, depth scale drift between overlapping windows and 3D geometric misalignment within a window, can be corrected without retraining by steering the denoising process itself. The method adds two guidance signals to the last steps of denoising: scale guidance, which fits a least-squares scale and shift between overlapping windows and pushes the current window's depth toward the aligned result, and geometry guidance, which derives camera poses from the predicted depth via PnP and optimizes the depth against reprojection, tracking, surface-normal, and smoothness losses. The two terms work together, and on ScanNet, GMU Kitchen, KITTI, and Bonn they consistently improve relative depth error, threshold accuracy, and multi-frame consistency over the base model and other baselines, with the gap widening as videos get longer. This matters because long-video depth consistency is what determines whether predicted depths can be used for 3D reconstruction and spatial reasoning, and the remedy requires no training data or model modification.

What carries the argument

The load-bearing machinery has two parts. Scale guidance: at each guided denoising step, the current window's predicted clean latent is decoded to depth, a least-squares scale $s$ and shift $t$ are fit to align the overlapping frames with the stored depth of the previous window, and the aligned depth $d_{\mathrm{aligned}} = s \cdot \hat{d}_{\mathrm{cur}} + t$ is re-encoded into latent space to serve as a pseudo-label for an MSE loss whose gradients update the noise prediction (forward guidance). Geometry guidance: off-the-shelf 2D tracking produces pixel correspondences, Perspective-n-Point (PnP) recovers camera poses from the current predicted depth, and the decoded depth is optimized by backward guidance against a weighted sum of a depth reprojection loss (warp one depth frame to a neighbor and compare), a tracking loss (3D distance between tracked points projected by the depths), a surface-normal alignment loss, and an edge-aware smoothness loss, with the depth reprojection term weighted highest. The two guidance terms act in the last two of five denoising steps, geometry first, scale second, with hand-set learning rates, iteration counts, and loss weights documented in the supplementary material.

What would settle it

A concrete test: on a long static video with heavy overlap occlusion or texture-poor regions, degrade the off-the-shelf tracks in half of the windows and run the full pipeline; if the PnP-derived poses are the load-bearing assumption, MFC and AbsRel should worsen relative to the base model because geometry guidance would be optimizing against a wrong target. The paper's own choice to withhold geometry guidance on the dynamic Bonn dataset marks this as the point where the assumption is most likely to break.

Watch

Extended reading notes

Core claim

The central claim is that multi-window depth scale drift in diffusion-based video depth prediction is best corrected by synchronizing scales at every guided denoising step rather than only at window initialization, and that within-window geometry should be enforced by explicit 3D constraints derived from the predicted depths themselves. Concretely, scale guidance decodes the current predicted clean latent to depth, fits a least-squares scale $s$ and shift $t$ over the overlapping frames against the previous window's stored depth, and re-encodes the aligned full-window depth as a pseudo-label for an MSE loss on the latent, so the denoiser is pushed toward the synchronized scale throughout the last steps of sampling. Geometry guidance instead estimates camera poses from off-the-shelf 2D tracks via PnP and optimizes the decoded depth by backward guidance against a weighted sum of depth reprojection, tracking, surface-normal, and edge-aware smoothness losses. The paper reports that the two terms together improve AbsRel, $\delta_1$, and multi-frame consistency (MFC) over all baselines on ScanNet, GMU Kitchen, KITTI, and Bonn; that each term used alone beats post-hoc optimization with the same losses; and that on the highly dynamic Bonn dataset, where only scale guidance is applied, the roughly 16% $\delta_1$ gain over DepthCrafter at 590 frames comes from cross-window scale synchronization alone.

Load-bearing premise

The geometry guidance presumes that camera poses recovered by PnP from the currently predicted depth and the off-the-shelf 2D tracks are accurate enough to define a useful optimization target, and that the hand-fixed guidance weights, learning rates, and iteration counts carry over to any new dataset without per-dataset tuning.

Editorial extensions

If this is right

  • Under the paper's global-alignment evaluation protocol, DepthSync's guided depths stay accurate at video lengths where the base models degrade, so long videos can be processed window-by-window without losing a common scale.
  • Poses derived from the guided depths by PnP outperform dedicated pose-estimation baselines (RelPose++, PoseDiffusion, RayDiffusion) on ScanNet under ATE and RPE metrics, so the same depth output directly supports monocular odometry.
  • Applying the regularization inside the denoising loop beats applying the identical losses after inference, indicating that the diffusion prior actively steers the geometric optimization rather than merely providing a starting point.
  • A lightweight variant that keeps scale guidance and drops geometry guidance runs in about 2% of the full method's time (18.1 s versus 961 s per 90 frames) while still improving over the base model, making cross-window scale synchronization the cheap and effective fix for long-video drift.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: because scale guidance only needs overlapping windows and a latent depth representation, it should transfer to any diffusion-based video depth model that encodes depth into latents; a testable prediction is that attaching it to a different base model reproduces similar AbsRel and MFC gains.
  • The paper withholds geometry guidance on the highly dynamic Bonn split; a natural follow-up is motion-masked geometry guidance that excludes tracked points on moving objects from the PnP fit, with the prediction that dynamic-scene MFC improves as well.
  • All guidance hyperparameters are fixed across datasets in the supplementary material, so the reported margins may be a floor; grid-searching the loss weights per dataset would likely widen the gap over DepthCrafter and would also reveal which of the four geometry losses carries the improvement.
  • The case study showing that per-window alignment largely restores accuracy implies that cheap post-hoc stitching of window scales is a strong baseline; the paper's comparison shows in-loop guidance beats that baseline, so the remaining gap quantifies the value of coupling the regularization to the diffusion trajectory.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper presents DepthSync, a training-free, inference-time guidance framework that wraps the DepthCrafter diffusion video-depth model. It keeps the sliding-window inference scheme and applies two guidance terms inside the last denoising steps: scale guidance, which least-squares aligns the depth scale of overlapping windows against the previous window, and geometry guidance, which uses off-the-shelf 2D tracks, PnP-derived camera poses, surface normals, and smoothness losses to regularize intra-window depth geometry. Experiments on ScanNet, GMU Kitchen, KITTI, and Bonn are reported for video lengths of 150–590 frames, with AbsRel, delta1, MFC, and derived-pose metrics. An ablation and a post-optimization comparison are included, along with supplementary ablations of loss terms, guidance start step, and inference cost.

Significance. If the reported gains are robust, DepthSync is a useful contribution: it improves cross-window scale consistency and intra-window geometry without retraining, and the scale-guidance mechanism is simple and effective across datasets. The paper includes a thoughtful comparison to post-optimization, and the supplementary ablations cover the main loss terms. The main weaknesses are the circular reliance of geometry guidance on poses estimated from the depth being optimized, the absence of error bars or sensitivity analysis for a method with many hand-set hyperparameters, and the limited-length post-optimization comparison. The MFC metric provides an independent (ground-truth-pose) signal, which strengthens the evidence, but the pose-based evaluation in Table 2 is not fully conclusive for the reasons above.

major comments (3)
  1. [Sec. 3.3 / Supp. A.2] The geometry guidance uses camera poses estimated by PnP from the current predicted depth (Supp. Eq. 7-9) and then optimizes that same depth with reprojection and tracking losses. This is a feedback loop: the optimization target is defined by the very quantity being optimized, so the losses enforce self-consistency rather than accuracy to true scene geometry. The pose-based evaluation in Table 2 is similarly self-confirming, because it derives poses from the optimized depth using the same off-the-shelf tracks and PnP pipeline. MFC (Sec. 4.1) is an independent check because it uses ground-truth poses, and the MFC gains are encouraging; however, the paper should add an ablation in which the poses are fixed from an external source (e.g., ground-truth ScanNet poses or an RGB-only SLAM estimate) or frozen from the baseline DepthCrafter depth, and show that the geometry guidance still improves accuracy and MFC. Without this, the central claim of 'geometry-consistent depth' is not fully established.
  2. [Sec. 4.2 / Table 1 / Supp. A.1] No error bars or statistical significance tests are reported for any result; every number is a point estimate on a fixed test split. The ablations in Table 3 and Supplement D are run on small subsets (first 35 and first 25 ScanNet test scenes), and the guidance hyperparameters (learning rates 0.01/0.02, 80 iterations per step, scale-guidance learning rates and 1000 iterations, guidance start at the second-to-last step, and loss weights alpha_d=35, alpha_t=2, alpha_n=0.1, alpha_s=1) are hand-fixed with no sensitivity analysis. Since the method relies on iterative optimization that could be sensitive to these choices and to the subset used for tuning, the paper should report variance across videos and a sensitivity study over at least the guidance start step, learning rates, and loss weights. This is necessary to support the claim of consistent, dataset-independent improvements.
  3. [Sec. 4.3 / Table 3] The comparison against post-optimization is performed only at a short video length (150 frames, 2 sliding windows) on a subset of ScanNet. This is precisely the regime where cross-window scale drift is least severe, as the paper itself demonstrates in Table 1 and Fig. 3. To substantiate the claim that injecting guidance into the denoising loop is superior to applying the same constraints post-hoc, the paper should include a post-optimization comparison at longer video lengths (e.g., 270 or 450 frames) and on at least one additional dataset. The current table does not rule out the possibility that post-optimization narrows the gap or even matches the guidance approach as video length grows.
minor comments (7)
  1. [Sec. 3.3] The reference 'as shown in Fig. Y' is an unresolved placeholder; it should point to a specific figure number (e.g., Fig. 1 or Fig. 5).
  2. [Supp. A.1] The base learning rates for scale guidance are reported as '6e4' and '1e6'; these values appear to be missing minus signs (likely 6e-4 and 1e-6 or similar). Please correct, since these exact numbers are needed to reproduce the method.
  3. [Table 3] The baseline AbsRel of 0.174 differs from the DepthCrafter ScanNet row in Table 1 (0.141 at length 150) because the ablation uses a different subset; this should be stated in the table caption or in the text.
  4. [Supp. Table 4] The citation placeholder '[?]' for 2DGS should be replaced with the proper reference.
  5. [Supp. Eq. (7)] The notation p_j ∼ K T_{i→j} d_i(p_i) K^{-1} p_i is projective and omits the depth division; define the relation precisely to avoid ambiguity.
  6. [Supp. A.2.1] 'solvePnPRansac inferface' is a typo for 'interface'.
  7. [Sec. 4.1] The order of the two guidance terms is clarified only in Supp. A.1 (geometry first, then scale); consider stating this order in the main text to avoid confusion.

Circularity Check

2 steps flagged · score 3.0 of 10

Geometry guidance and pose evaluation are self-referential via PnP poses computed from the depth being optimized, but scale guidance and GT-pose MFC provide independent support.

  1. self definitional [Sec. 3.3, Eq. (5); Supp. A.2.1, Eqs. (7)-(9)]
    "Specifically, with the assistance of an off-the-shelf tracking prediction network [25], we estimate the camera transformation from the RGB video and its corresponding depth frames using the Perspective-n-Point (PnP) method [13]. We then compute a depth reprojection loss and a tracking loss to ensure that different depth frames are aligned in 3D space, thereby enforcing geometric consistency between depth frames. ... pj ∼KT i→jdi(pi)K−1pi (7) ... Li,j d =|d i −d j→i| (8)"

    The poses T_i→j used in the reprojection and tracking losses are computed by PnP from 3D points lifted by the very depth maps d_i being optimized. PnP absorbs depth errors into the pose, so the losses measure self-consistency of the depth with its own pose estimate rather than agreement with externally fixed geometry. The optimization is a fixed-point loop d* = argmin_d L(d, PnP(d, tracks)), so a depth that is wrong but internally consistent can satisfy the guidance. This does not invalidate the external MFC test (which uses GT poses), but it means the geometry-guidance objective is partially defined by its own output.

  2. other [Sec. 4.2, Table 2; Supp. A.2.1 Eqs. (7)-(9)]
    "Given that video depths inherently represent 3D structure, we further evaluate the geometric quality by assessing poses derived from our depths. Using pixel correspondences between adjacent frames predicted from an offline tracking predictor [25], we compute poses via PnP-RANSAC and evaluate them using visual odometry metrics: absolute translation error (ATE), relative translation error (RPE trans), and relative rotation error (RPE rot) [63]."

    The pose-based validation in Table 2 uses the same tracking correspondences and the same PnP mechanism that defines the geometry-guidance objective. Because the optimized depth was explicitly pulled toward consistency with PnP poses estimated from itself, the derived poses are partly self-confirming: low reprojection error is guaranteed by the optimization target. ATE/RPE are still measured against ground-truth poses, so the comparison has independent content, but an ablation with externally fixed poses or an independent SLAM pose source would be needed to separate self-consistency gains from genuine geometric accuracy gains.

full rationale

DepthSync is largely self-contained: scale guidance is an alignment constraint evaluated with a shared least-squares fit to external ground truth, and MFC is computed with ground-truth poses, so the central scale-consistency claim does not reduce to a fitted input. The main circular element is the geometry guidance and its pose-based validation: camera poses are estimated by PnP from the current predicted depth and then used to regularize that same depth, creating a feedback loop that rewards self-consistency. The pose-evaluation table inherits this loop because it re-derives poses with the same correspondences and PnP routine. The paper does not ablate pose source (e.g., frozen baseline poses or independent RGB SLAM), so the magnitude of the geometry-guidance improvement attributable to true geometry, rather than self-consistency, is uncertain. However, the external MFC and depth metrics provide independent evidence, so the paper is not wholly circular; the issue is a partial, localized circularity in the geometry-validation chain.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two main pillars: the pretrained diffusion model (DepthCrafter) and the off-the-shelf trackers/normal predictors, plus a set of manually tuned hyperparameters. The geometric optimization includes a potential self-referential pose-from-depth loop. No new entities are postulated.

free parameters (4)
  • Geometry guidance loss weights (alpha_d, alpha_t, alpha_n, alpha_s) = 35, 2, 0.1, 1
    Set by hand for all experiments; no sensitivity analysis reported (Supp. A.2.3).
  • Scale guidance learning rate and iterations = 6e4 (step 4), 1e6 (step 5), 1000 iterations on final step, early termination <5e-4
    Manual tuning choices in Supp. A.1; likely selected on ScanNet ablation subset.
  • Geometry guidance learning rate and iteration count = 0.01 (step 4), 0.02 (step 5), 80 iterations, AdamW, cosine schedule
    Manual choices in Supp. A.1.
  • Number of final denoising steps with guidance = 2 of 5 Euler steps
    Selected based on supplementary ablation (Supp. D, Table 7) as efficiency/accuracy trade-off.
assumptions (4)
  • domain assumption DepthCrafter provides a strong enough pretrained diffusion prior for video depth, so that guidance can correct scale and geometry without retraining.
    The method inherits DepthCrafter's generalization; a weaker base model might not benefit.
  • domain assumption CoTracker3 and StableNormal supply sufficiently accurate 2D point tracks and surface normals for the evaluated scenes.
    Geometry guidance depends on these off-the-shelf predictors; their failure modes are not analyzed.
  • domain assumption Depth errors are well captured by a global scale/shift per video, so least-squares alignment on overlapping windows is an appropriate synchronization target.
    Standard in affine-invariant depth estimation, but assumes no higher-order drift.
  • ad hoc to paper The weighted combination of reprojection, tracking, normal, and smoothness losses converges to a geometrically consistent depth solution without degenerate collapse.
    No proof or theoretical guarantee is given; the paper relies on empirical ablation success.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DepthSync: Diffusion Guidance-Based Depth Synchronization for Scale- and Geometry-Consistent Video Depth Estimation." pith.science (2026). https://pith.science/paper/UVERKSNG

@misc{pith2026250701603,
  author       = {Pith},
  title        = {Pith review of: DepthSync: Diffusion Guidance-Based Depth Synchronization for Scale- and Geometry-Consistent Video Depth Estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UVERKSNG}},
  note         = {Machine review of arXiv:2507.01603}
}
read the original abstract

Diffusion-based video depth estimation methods have achieved remarkable success with strong generalization ability. However, predicting depth for long videos remains challenging. Existing methods typically split videos into overlapping sliding windows, leading to accumulated scale discrepancies across different windows, particularly as the number of windows increases. Additionally, these methods rely solely on 2D diffusion priors, overlooking the inherent 3D geometric structure of video depths, which results in geometrically inconsistent predictions. In this paper, we propose DepthSync, a novel, training-free framework using diffusion guidance to achieve scale- and geometry-consistent depth predictions for long videos. Specifically, we introduce scale guidance to synchronize the depth scale across windows and geometry guidance to enforce geometric alignment within windows based on the inherent 3D constraints in video depths. These two terms work synergistically, steering the denoising process toward consistent depth predictions. Experiments on various datasets validate the effectiveness of our method in producing depth estimates with improved scale and geometry consistency, particularly for long videos.

Figures

Figures reproduced from arXiv: 2507.01603 by the authors.

Figure 1
Figure 1. We present DepthSync, a framework that introduces cross-window depth scale synchronization and intra-window geometry alignment for long video depth predictions with enhanced scale and geometry consistency via diffusion guidance. Abstract Diffusion-based video depth estimation methods have achieved remarkable success. However, predicting depth for long videos remains challenging. Existing methods typ￾ically split vid… view at source ↗
Figure 2
Figure 2. Overview. Our DepthSync framework predicts depth for a monocular video by applying guidance to a pre-trained diffusion￾based depth estimation model, ensuring scale- and geometry-consistent depth predictions. Following common practice, the input video is split into overlapping windows and processed sequentially. During denoising, we derive depths from noise prediction, applying geometry guidance to align frames withi… view at source ↗
Figure 3
Figure 3. Existing methods suffer from scale inconsistency across windows as video length increases. We evaluate the same depth predictions on Bonn using two alignment strategies: align￾ing with a shared scale and shift factor for entire video versus per￾window alignment. The shared alignment causes a significant ac￾curacy drop, highlighting large scale variations between windows. random Gaussian noise. Additionally, the over… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Qualitative Comparison for Long Video Depth Estimation. We compare predictions on 450-frame ScanNet videos and 590- frame Bonn videos. Slicing along the timeline at the green-line position (first column), we concatenate results to visualize temporal changes in color an…
Figure 5
Figure 5. Figure 5: Qualitative Reconstruction Comparison on ScanNet. We project video depth into 3D space, demonstrating that our method produces more accurate geometry. Red boxes highlight misaligned walls in previous methods’ reconstructions. 5. Conclusion In this paper, we propose Dep…
Figure 6
Figure 6. Figure 6: Supplementary Qualitative Comparison on Long Video Depth Consistency. We compare predictions on 450-frame ScanNet videos and 590-frame Bonn videos. Slicing along the timeline at the green-line position (first column), we concatenate results to visualize temporal change…
Figure 7
Figure 7. Figure 7: Depth Error Map Comparison on Long Video Depth Consistency. Supplementary visualization of absolute error maps for qualitative analysis (see [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Supplementary Qualitative Examples for 3D Reconstruction Comparisons [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Supplementary Qualitative Examples for 3D Reconstruction Comparisons [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 39 canonical work pages

  1. [1]

    Universal guidance for diffusion models

    Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geip- ing, and Tom Goldstein. Universal guidance for diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 843–852,

    work page

  2. [2]

    Adabins: Depth estimation using adaptive bins

    Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4009–4018, 2021. 2

    2021

  3. [3]

    Zoedepth: Zero-shot trans- fer by combining relative and metric depth.arXiv preprint arXiv:2302.12288, 2023

    Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias M ¨uller. Zoedepth: Zero-shot trans- fer by combining relative and metric depth.arXiv preprint arXiv:2302.12288, 2023. 2

    arXiv 2023

  4. [4]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023. 1, 2, 3

    arXiv 2023

  5. [5]

    Self-supervised learning with geometric constraints in monocular video: Connecting flow, depth, and camera

    Yuhua Chen, Cordelia Schmid, and Cristian Sminchis- escu. Self-supervised learning with geometric constraints in monocular video: Connecting flow, depth, and camera. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7063–7072, 2019. 2

    work page 2019

  6. [6]

    Diffusion posterior sampling for general noisy inverse problems

    Hyungjin Chung, Jeongsol Kim, Michael T McCann, Marc L Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. 2023. 2

    work page 2023

  7. [7]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. 5

    2017

  8. [8]

    Diffusion models beat gans on image synthesis.Advances in neural informa- tion processing systems, 34:8780–8794, 2021

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis.Advances in neural informa- tion processing systems, 34:8780–8794, 2021. 3

    2021

Show all 64 references
  1. [9]

    Towards real-time monocular depth estimation for robotics: A survey.IEEE Transactions on Intelligent Transportation Systems, 23(10):16940–16961,

    Xingshuai Dong, Matthew A Garratt, Sreenatha G Ana- vatti, and Hussein A Abbass. Towards real-time monocular depth estimation for robotics: A survey.IEEE Transactions on Intelligent Transportation Systems, 23(10):16940–16961,

  2. [10]

    Mal: Motion-aware loss with temporal and distillation hints for self-supervised depth estimation

    Yue-Jiang Dong, Fang-Lue Zhang, and Song-Hai Zhang. Mal: Motion-aware loss with temporal and distillation hints for self-supervised depth estimation. In2024 IEEE Inter- national Conference on Robotics and Automation (ICRA), pages 7318–7324. IEEE, 2024. 1

    2024

  3. [11]

    Depth map prediction from a single image using a multi-scale deep net- work.Advances in neural information processing systems, 27, 2014

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep net- work.Advances in neural information processing systems, 27, 2014. 2

    2014

  4. [12]

    Diffusion self-guidance for control- lable image generation.Advances in Neural Information Processing Systems, 36:16222–16239, 2023

    Dave Epstein, Allan Jabri, Ben Poole, Alexei Efros, and Aleksander Holynski. Diffusion self-guidance for control- lable image generation.Advances in Neural Information Processing Systems, 36:16222–16239, 2023. 2

    2023

  5. [13]

    Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981

    Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981. 5, 12

    1981

  6. [14]

    Deep ordinal regression net- work for monocular depth estimation

    Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Bat- manghelich, and Dacheng Tao. Deep ordinal regression net- work for monocular depth estimation. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 2002–2011, 2018. 2

    2002

  7. [15]

    Geowiz- ard: Unleashing the diffusion priors for 3d geometry esti- mation from a single image

    Xiao Fu, Wei Yin, Mu Hu, Kaixuan Wang, Yuexin Ma, Ping Tan, Shaojie Shen, Dahua Lin, and Xiaoxiao Long. Geowiz- ard: Unleashing the diffusion priors for 3d geometry esti- mation from a single image. InEuropean Conference on Computer Vision, pages 241–258. Springer, 2025. 2

    2025

  8. [16]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In2012 IEEE conference on computer vision and pat- tern recognition, pages 3354–3361. IEEE, 2012. 5

    2012

  9. [17]

    Multiview rgb-d dataset for object instance detection

    Georgios Georgakis, Md Alimoor Reza, Arsalan Mousavian, Phi-Hung Le, and Jana Ko ˇseck´a. Multiview rgb-d dataset for object instance detection. In2016 Fourth international conference on 3D vision (3DV), pages 426–434. IEEE, 2016. 5

    2016

  10. [18]

    Diffusion models as plug-and-play pri- ors.Advances in Neural Information Processing Systems, 35:14715–14728, 2022

    Alexandros Graikos, Nikolay Malkin, Nebojsa Jojic, and Dimitris Samaras. Diffusion models as plug-and-play pri- ors.Advances in Neural Information Processing Systems, 35:14715–14728, 2022. 2

    2022

  11. [19]

    Pm-huber: Patchmatch with huber regularization for stereo matching

    Philipp Heise, Sebastian Klose, Brian Jensen, and Alois Knoll. Pm-huber: Patchmatch with huber regularization for stereo matching. InProceedings of the IEEE International Conference on Computer Vision, pages 2360–2367, 2013. 5, 13

    2013

  12. [20]

    Video dif- fusion models.Advances in Neural Information Processing Systems, 35:8633–8646, 2022

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models.Advances in Neural Information Processing Systems, 35:8633–8646, 2022. 1, 2

    2022

  13. [21]

    Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation.IEEE Transactions on Pattern Analysi...

    2024

  14. [22]

    Depthcrafter: Generating consistent long depth sequences for open-world videos

    Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, and Ying Shan. Depthcrafter: Generating consistent long depth sequences for open-world videos. pages 2005–2015, 2025. 1, 2, 3, 4, 5, 7, 12, 13, 14

    2005

  15. [23]

    2d gaussian splatting for geometrically ac- curate radiance fields

    Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically ac- curate radiance fields. InACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024. 1, 13

    2024

  16. [24]

    Ddp: Diffusion model for dense visual prediction

    Yuanfeng Ji, Zhe Chen, Enze Xie, Lanqing Hong, Xihui Liu, Zhaoqiang Liu, Tong Lu, Zhenguo Li, and Ping Luo. Ddp: Diffusion model for dense visual prediction. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 21741–21752, 2023. 2

    2023

  17. [25]

    Co- tracker3: Simpler and better point tracking by pseudo- labelling real videos.arXiv preprint arXiv:2410.11831,

    Nikita Karaev, Iurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Co- tracker3: Simpler and better point tracking by pseudo- labelling real videos.arXiv preprint arXiv:2410.11831,

    arXiv

  18. [26]

    Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35:26565–26577, 2022

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35:26565–26577, 2022. 5, 12

    2022

  19. [27]

    Denoising diffusion restoration models.Advances in Neural Information Processing Systems, 35:23593–23606,

    Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models.Advances in Neural Information Processing Systems, 35:23593–23606,

  20. [28]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9492– ...

    2024

  21. [29]

    Auto-encoding vari- ational bayes, 2013

    Diederik P Kingma, Max Welling, et al. Auto-encoding vari- ational bayes, 2013. 3

    2013

  22. [30]

    Ro- bust consistent video depth estimation

    Johannes Kopf, Xuejian Rong, and Jia-Bin Huang. Ro- bust consistent video depth estimation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1611–1621, 2021. 2

    2021

  23. [31]

    From big to small: Multi-scale local planar guidance for monocular depth estimation.arXiv preprint arXiv:1907.10326, 2019

    Jin Han Lee, Myung-Kyu Han, Dong Wook Ko, and Il Hong Suh. From big to small: Multi-scale local planar guidance for monocular depth estimation.arXiv preprint arXiv:1907.10326, 2019. 2

    1907 arXiv

  24. [32]

    Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds

    Jiahui Lei, Yijia Weng, Adam W Harley, Leonidas Guibas, and Kostas Daniilidis. Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds. pages 6165– 6177, 2025. 1

    2025

  25. [33]

    Relpose++: Recovering 6d poses from sparse-view ob- servations

    Amy Lin, Jason Y Zhang, Deva Ramanan, and Shubham Tul- siani. Relpose++: Recovering 6d poses from sparse-view ob- servations. In2024 International Conference on 3D Vision (3DV), pages 106–115. IEEE, 2024. 6

    2024

  26. [34]

    Wonder3d: Sin- gle image to 3d using cross-domain diffusion

    Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Sin- gle image to 3d using cross-domain diffusion. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pat...

    2024

  27. [35]

    Readout guidance: Learning con- trol from diffusion features

    Grace Luo, Trevor Darrell, Oliver Wang, Dan B Goldman, and Aleksander Holynski. Readout guidance: Learning con- trol from diffusion features. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8217–8227, 2024. 2

    2024

  28. [36]

    Consistent video depth estimation.ACM Transactions on Graphics (ToG), 39(4):71–1, 2020

    Xuan Luo, Jia-Bin Huang, Richard Szeliski, Kevin Matzen, and Johannes Kopf. Consistent video depth estimation.ACM Transactions on Graphics (ToG), 39(4):71–1, 2020. 2, 5

    2020

  29. [37]

    Palazzolo, J

    E. Palazzolo, J. Behley, P. Lottes, P. Gigu `ere, and C. Stach- niss. ReFusion: 3D Reconstruction in Dynamic Environ- ments for RGB-D Cameras Exploiting Residuals. 2019. 4, 5

    2019

  30. [38]

    Unidepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10106–10116, 2024. 2

    2024

  31. [39]

    Ren ´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer.IEEE transactions on pattern analysis and machine intelligence, 44(3):1623–1637, 2020. 2

    2020

  32. [40]

    The surprising effectiveness of diffusion models for optical flow and monocular depth estimation.Advances in Neural Information Processing Systems, 36, 2024

    Saurabh Saxena, Charles Herrmann, Junhwa Hur, Abhishek Kar, Mohammad Norouzi, Deqing Sun, and David J Fleet. The surprising effectiveness of diffusion models for optical flow and monocular depth estimation.Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  33. [41]

    Structure-from-motion revisited

    Johannes Lutz Sch ¨onberger and Jan-Michael Frahm. Structure-from-motion revisited. InConference on Com- puter Vision and Pattern Recognition (CVPR), 2016. 13

    2016

  34. [42]

    Learning temporally consistent video depth from video diffusion priors

    Jiahao Shao, Yuanbo Yang, Hongyu Zhou, Youmin Zhang, Yujun Shen, Vitor Guizilini, Yue Wang, Matteo Poggi, and Yiyi Liao. Learning temporally consistent video depth from video diffusion priors. pages 22841–22852, 2025. 1, 2, 3, 4, 5, 7, 14

    2025

  35. [43]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. 3

  36. [44]

    Deepv2d: Video to depth with differentiable structure from motion

    Zachary Teed and Jia Deng. Deepv2d: Video to depth with differentiable structure from motion. 2018. 2

    2018

  37. [45]

    Posediffusion: Solving pose estimation via diffusion-aided bundle adjustment

    Jianyuan Wang, Christian Rupprecht, and David Novotny. Posediffusion: Solving pose estimation via diffusion-aided bundle adjustment. InProceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 9773–9783,

  38. [46]

    Shape of motion: 4d reconstruc- tion from a single video.arXiv preprint arXiv:2407.13764,

    Qianqian Wang, Vickie Ye, Hang Gao, Jake Austin, Zhengqi Li, and Angjoo Kanazawa. Shape of motion: 4d reconstruc- tion from a single video.arXiv preprint arXiv:2407.13764,

  39. [47]

    Dust3r: Geometric 3d vi- sion made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vi- sion made easy. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20697– 20709, 2024. 13, 14

    2024

  40. [48]

    Zero-shot im- age restoration using denoising diffusion null-space model

    Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot im- age restoration using denoising diffusion null-space model. arXiv preprint arXiv:2212.00490, 2022. 2

    2022 arXiv

  41. [49]

    Neural video depth stabilizer

    Yiran Wang, Min Shi, Jiaqi Li, Zihao Huang, Zhiguo Cao, Jianming Zhang, Ke Xian, and Guosheng Lin. Neural video depth stabilizer. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision (ICCV), pages 9466– 9476, 2023. 5

    2023

  42. [50]

    Neural video depth stabilizer

    Yiran Wang, Min Shi, Jiaqi Li, Zihao Huang, Zhiguo Cao, Jianming Zhang, Ke Xian, and Guosheng Lin. Neural video depth stabilizer. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pages 9466–9476,

  43. [51]

    Depth any video with scalable synthetic data.arXiv preprint arXiv:2410.10815, 2024

    Honghui Yang, Di Huang, Wei Yin, Chunhua Shen, Haifeng Liu, Xiaofei He, Binbin Lin, Wanli Ouyang, and Tong He. Depth any video with scalable synthetic data.arXiv preprint arXiv:2410.10815, 2024. 2, 3, 13

    2024 arXiv

  44. [52]

    Depth any- thing v2.Advances in Neural Information Processing Sys- tems, 37:21875–21911, 2024

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth any- thing v2.Advances in Neural Information Processing Sys- tems, 37:21875–21911, 2024. 2, 5

    2024

  45. [53]

    Mamo: Leveraging memory and attention for monocular video depth estimation

    Rajeev Yasarla, Hong Cai, Jisoo Jeong, Yunxiao Shi, Risheek Garrepalli, and Fatih Porikli. Mamo: Leveraging memory and attention for monocular video depth estimation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 8754–8764, 2023. 2

    2023

  46. [54]

    Futuredepth: Learning to predict the future improves video depth estimation

    Rajeev Yasarla, Manish Kumar Singh, Hong Cai, Yunxiao Shi, Jisoo Jeong, Yinhao Zhu, Shizhong Han, Risheek Gar- repalli, and Fatih Porikli. Futuredepth: Learning to predict the future improves video depth estimation. pages 440–458,

  47. [55]

    Stablenormal: Reducing diffusion variance for stable and sharp normal.ACM Transactions on Graphics (TOG), 43(6):1–18, 2024

    Chongjie Ye, Lingteng Qiu, Xiaodong Gu, Qi Zuo, Yushuang Wu, Zilong Dong, Liefeng Bo, Yuliang Xiu, and Xiaoguang Han. Stablenormal: Reducing diffusion variance for stable and sharp normal.ACM Transactions on Graphics (TOG), 43(6):1–18, 2024. 5, 12

    2024

  48. [56]

    Freedom: Training-free energy-guided condi- tional diffusion model

    Jiwen Yu, Yinhuai Wang, Chen Zhao, Bernard Ghanem, and Jian Zhang. Freedom: Training-free energy-guided condi- tional diffusion model. InProceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 23174– 23184, 2023. 2

    2023

  49. [57]

    Neural window fully-connected crfs for monocu- lar depth estimation

    Weihao Yuan, Xiaodong Gu, Zuozhuo Dai, Siyu Zhu, and Ping Tan. Neural window fully-connected crfs for monocu- lar depth estimation. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 3916–3925, 2022. 2

    2022

  50. [58]

    A survey of autonomous driving: Common practices and emerging technologies.IEEE access, 8:58443– 58469, 2020

    Ekim Yurtsever, Jacob Lambert, Alexander Carballo, and Kazuya Takeda. A survey of autonomous driving: Common practices and emerging technologies.IEEE access, 8:58443– 58469, 2020. 1

    2020

  51. [59]

    Exploiting temporal consistency for real-time video depth estimation

    Haokui Zhang, Chunhua Shen, Ying Li, Yuanzhouhan Cao, Yu Liu, and Youliang Yan. Exploiting temporal consistency for real-time video depth estimation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 1725–1734, 2019. 2

    2019

  52. [60]

    Monst3r: A simple approach for estimat- ing geometry in the presence of motion.arXiv preprint arXiv:2410.03825, 2024

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jam- pani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming- Hsuan Yang. Monst3r: A simple approach for estimat- ing geometry in the presence of motion.arXiv preprint arXiv:2410.03825, 2024. 13, 14

    arXiv

  53. [61]

    Cameras as rays: Pose estimation via ray diffusion

    Jason Y Zhang, Amy Lin, Moneish Kumar, Tzu-Hsuan Yang, Deva Ramanan, and Shubham Tulsiani. Cameras as rays: Pose estimation via ray diffusion. 6

  54. [62]

    Consistent depth of moving objects in video.ACM Transactions on Graphics (ToG), 40(4):1–12,

    Zhoutong Zhang, Forrester Cole, Richard Tucker, William T Freeman, and Tali Dekel. Consistent depth of moving objects in video.ACM Transactions on Graphics (ToG), 40(4):1–12,

  55. [63]

    Structure and motion from casual videos

    Zhoutong Zhang, Forrester Cole, Zhengqi Li, Michael Ru- binstein, Noah Snavely, and William T Freeman. Structure and motion from casual videos. InEuropean Conference on Computer Vision, pages 20–37. Springer, 2022. 6

    2022

  56. [64]

    Unsupervised learning of depth and ego-motion from video

    Tinghui Zhou, Matthew Brown, Noah Snavely, and David G Lowe. Unsupervised learning of depth and ego-motion from video. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1851–1858, 2017. 1 DepthSync: Diffusion Guidance-Based Depth Synchroniz...

    2017

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.