Pith. sign in

REVIEW 3 major objections 5 minor 17 cited by

Video Depth Anything: Consistent Depth Estimation for Super-Long Videos

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Video Depth Anything claims that one feedforward model can produce temporally consistent depth for arbitrarily long videos while retaining the spatial accuracy of its monocular base model, without optical flow, camera poses, or…

desk verdict Efficient video depth with a strong architecture and wide geometric experiments, but the temporal-consistency flagship claim rests on a single TAE column and a sign-blind loss; still worth a serious referee. read the letter →

arxiv 2501.12375 v3 pith:C47QNCKD submitted 2025-01-21 cs.CV cs.AI

Sili Chen , Hengkai Guo , Shengnan Zhu , Feihu Zhang , Zilong Huang , Jiashi Feng , Bingyi Kang This is my paper · ORCID
classification cs.CVcs.AI
keywords monoculardepthestimationvideotemporalconsistencygradientmatchingspatiotemporalattentionlonginferencezero-shotgeneralizationAnythingV2
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video Depth Anything sets out to solve a practical bottleneck in monocular depth estimation: applying a strong image-depth model to video produces flicker, while existing video-depth methods only handle clips under ten seconds and trade quality for compute. The paper claims a single feedforward model, built on Depth Anything V2, can estimate temporally consistent depth for videos lasting several minutes with no drop in spatial accuracy or generalization. It achieves this with a spatiotemporal head inserted into the depth head, a temporal gradient matching loss that needs no optical flow or pose supervision, and a key-frame-based inference strategy for arbitrarily long videos. If these claims hold, the model would give robotics, AR, and video-editing pipelines a fast and consistent depth source for long footage.

What carries the argument

The load-bearing mechanism is the Temporal Gradient Matching (TGM) loss: at each pixel coordinate, the absolute change in predicted depth between adjacent frames is forced to match the absolute change in ground-truth depth, computed only where the ground-truth change is below 0.05 to avoid edges and dynamic objects. Because the loss uses the same coordinate rather than optical-flow correspondences, it needs no geometric priors. It is supported by a spatiotemporal head that applies temporal self-attention inside the DPT head, and by an inference strategy that feeds each new window together with overlapping frames and two key frames spaced 12 frames apart, then linearly interpolates predictions in the overlap region to avoid flicker and scale drift.

What would settle it

On a static-camera clip of an object moving toward the camera, check whether the model's depth at the object's pixels decreases over time; if it instead increases with the same absolute magnitude, TGM training has no loss term to correct it. A signed depth-change error computed on such dynamic clips would settle whether the consistency is genuine.

Watch

Extended reading notes

Core claim

The paper's central claim is that temporal consistency in video depth estimation can be obtained by training a spatial-temporal head on top of a frozen Depth Anything V2 encoder, using a simple loss that matches the absolute temporal gradient of predicted depth to that of ground truth, and then stitching inference windows with overlapping frames plus key frames from earlier clips. On five zero-shot video benchmarks the model reports state-of-the-art spatial accuracy on four datasets and best temporal consistency on all five, while retaining image-depth accuracy close to Depth Anything V2; the small variant runs at 30 FPS. The authors attribute this to the combination of a lightweight temporal attention head, the temporal gradient matching loss, joint training on video depth plus unlabeled single images, and the window-stitching scheme that inherits scale and shift information across windows.

Load-bearing premise

The training loss assumes that matching the absolute size of depth change at each pixel between adjacent frames, and only where ground-truth change is tiny, is enough to make depth temporally consistent, so wrong-direction changes and dynamic-object motion are never directly penalized.

Editorial extensions

If this is right

  • Videos of several minutes can be processed by a single forward pass per window without cumulative drift, removing the practical ceiling that limited earlier methods to clips under about ten seconds.
  • The smallest model's 30 FPS runtime makes temporally consistent depth a real-time option for robotics, AR, and 3D video conversion.
  • Zero-shot image depth accuracy is preserved almost unchanged, so one model can serve both still-image and video tasks.
  • Because no optical flow, camera-pose, or video-diffusion priors are used, the approach avoids the failure modes and compute cost those priors introduce.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The sign-agnostic form of TGM means a prediction that reverses the direction of depth change could match the loss as long as the magnitude is right; a benchmark measuring signed change on dynamic objects would reveal whether this matters.
  • The key-frame interval and overlap size are fixed hyperparameters; very fast camera motion or rapid scene changes could decorrelate the key frames from the current window, so adaptive spacing is a natural extension.
  • Because the encoder is frozen and only the head sees video data, the temporal reasoning capacity is limited; fine-tuning the backbone with more video data could improve dynamic-object consistency beyond the current results.
  • The reported efficiency advantage suggests depth from video could be used as a real-time input modality for downstream tasks such as point-cloud fusion, rather than as an offline post-processing step.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Video Depth Anything, a feed-forward video depth estimator built on the Depth Anything V2 image model. It replaces the DPT head with a spatio-temporal head that includes temporal self-attention layers, trains with a newly proposed temporal gradient matching (TGM) loss together with the scale-shift-invariant image loss, and develops a key-frame-based segment-wise inference strategy for long videos. The authors claim state-of-the-art zero-shot video depth estimation on five benchmarks in terms of geometric accuracy, temporal consistency, and inference efficiency, and they further claim that the model can be applied to arbitrarily long videos without compromising quality, consistency, or generalization. The paper provides quantitative results for geometric accuracy on KITTI, Scannet, Bonn, NYUv2, and Sintel; temporal consistency (TAE) results are reported only for a 170-frame Scannet subset in the main table. Ablations cover loss functions, inference strategies, window sizes, and image distillation.

Significance. If the result holds, this is a meaningful advance: it shows that a lightweight transformer with a small temporal head can match or beat diffusion-based video depth models on geometric accuracy while being substantially faster, and that long-video inference can be stabilized with a simple key-frame reference strategy. The empirical evaluation is extensive on the geometric side, the ablations are informative, and the latency numbers are compelling. However, the central temporal-consistency claim is under-supported because the only temporal metric, TAE, is reported for a single dataset, and the proposed TGM loss is sign-insensitive and masked on dynamic regions, so the training signal does not directly enforce correct temporal direction or dynamic-object consistency. These gaps need to be addressed before the SOTA claim can be fully accepted.

major comments (3)
  1. [Table 1, Sec. 4.2] Table 1 reports TAE, the only temporal-consistency metric, exclusively for the Scannet 170-frame subset; no TAE values are given for KITTI, Bonn, NYUv2, or Sintel. The sentence in Sec. 4.2 that the model 'outperforms all baselines on all datasets in terms of temporal consistency' is therefore not supported by the reported numerical evidence. This is particularly important because Bonn is a dynamic indoor dataset and Sintel contains large motions, exactly the conditions where temporal consistency and dynamic-object behavior need to be verified. Please report TAE on all five datasets, or at least on Bonn and Sintel, and include TAE as a function of frame length in Fig. 4 if the 'without compromising' claim is to be sustained.
  2. [Eq. (3), Sec. 3.2] The TGM loss in Eq. (3) is LTGM = (1/(N-1)) sum || |d_{i+1}-d_i| - |g_{i+1}-g_i| ||_1, computed only where |g_{i+1}-g_i| < 0.05. This formulation is invariant to the sign of the temporal depth change: a prediction that moves in the opposite direction from the ground truth with the same magnitude incurs no loss. Moreover, the threshold explicitly excludes dynamic-object and edge regions, so the model receives no direct supervision for the temporal consistency of moving objects. Since the paper's main temporal-consistency claim is credited to this loss, the sign-blindness and the mask are load-bearing. The ablation in Table 4 does not include a sign-aware or dynamic-region-aware variant, so the contribution of these design choices is untested. I recommend ablating a sign-sensitive version of the loss (e.g., enforcing signed gradient similarity) and reporting a directional temporal metric.
  3. [Sec. 4.2, Fig. 4, Fig. 7] The claim that the model can be applied to 'arbitrarily long videos without compromising quality, consistency, or generalization ability' is an extrapolation from the presented evidence. The longest quantitative evaluation is 500 frames (Fig. 4), and the only longer demonstration is the qualitative 7320-frame self-captured video in Fig. 7, where no quantitative consistency metric is reported. While the key-frame mechanism is plausible, the paper should either temper the 'arbitrarily long' phrasing or provide quantitative results at lengths beyond 500 frames, e.g., on the self-captured video or by reporting TAE every N frames for a longer sequence.
minor comments (5)
  1. [Sec. 3.2, Eq. (1)] The word 'wrapping' should be 'warping', and the summation index i starts at 2; please clarify whether the index range is intended to be i = 1 to N-1.
  2. [Table 1] 'ChoronDepth' is a typo for 'ChronoDepth' in the table header and row labels.
  3. [Sec. 4.3] 'Througout' should be 'Throughout'.
  4. [Fig. 1 (right)] The definition of consistency as 'the maximum TAE among all models minus the TAE of each individual model' is confusing as written; please define it clearly in the caption or text.
  5. [Supplementary Sec. 4] 'with a overlap' should be 'with an overlap' in the window-size discussion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's temporal-consistency and accuracy claims are supported by held-out video benchmarks and a training loss that supervises against ground-truth temporal gradients, with no prediction reducing to a fitted parameter or to a self-citation chain.

full rationale

The paper's central claims are empirical: Video Depth Anything achieves state-of-the-art spatial accuracy, temporal consistency, and efficiency on zero-shot video depth benchmarks. The derivation chain is a standard supervised learning pipeline: a frozen Depth Anything V2 encoder plus a new spatio-temporal head, trained with the TGM loss (Eq. 3) against ground-truth temporal depth gradients, the scale-shift-invariant loss, and image distillation, then evaluated on held-out KITTI, Scannet, Bonn, NYUv2, and Sintel. No equation in the paper defines a prediction in terms of the benchmark metric, and no reported result is a fitted parameter renamed as a prediction. The TGM loss uses ground-truth temporal gradients as supervision; it is a training objective, not an evaluation quantity, and the TAE metric is computed from ground-truth poses and depth on held-out videos, not from training-loss components. The use of Depth Anything V2 as a frozen backbone is a legitimate use of prior work, and several Depth Anything V2 authors overlap with the present authors, but the load-bearing evidence is the independent benchmark comparison in Table 1 and Fig. 4, not a uniqueness theorem or an unverified self-citation. The reviewer concerns that TAE is reported only for Scannet, that the TGM loss is sign-blind and masked to |g_{i+1}-g_i| < 0.05, and that 'arbitrarily long' is extrapolated from 500-frame evaluations are substantive support/correctness concerns, but they are not circularity: they question whether the evidence fully backs the claim, not whether the claim is equivalent to its inputs by construction. The paper even discloses its training-data limitations in the supplementary, which is inconsistent with hiding a circular derivation. Overall, the derivation is self-contained with respect to circularity, and the appropriate score is 0.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The paper's contribution rests on several assumptions: frozen encoder sufficiency, the validity of the sign-insensitive gradient loss, synthetic-to-real transfer, and evaluation protocol choices. These are typical for deep learning papers, but the TGM loss assumption is ad hoc and affects the core temporal consistency claim. The free parameters are hyperparameters that tune the loss and inference stitching.

free parameters (8)
  • TGM threshold = 0.05
    Regions with |g_{i+1}-g_i| < 0.05 are included in the TGM loss; this masks dynamic objects and edges and directly defines the training objective.
  • Loss weight alpha for LTGM = 10.0
    Balances temporal gradient loss against spatial SSI loss; chosen by the authors, affects the temporal/spatial trade-off.
  • Loss weight beta for Lssi = 1.0
    Weight for the scale-shift-invariant spatial loss.
  • Distillation loss weight = 0.5
    Weight for self-training on unlabeled images with pseudo-labels.
  • Inference window N = 32
    Number of frames per inference clip; ablations show 32 is preferred over 16 and 48.
  • Overlap frames To = 8
    Overlapping frames between consecutive windows for stitching; final model uses eight overlapping frames as described in Sec. 3.3.
  • Key frames Tk = 2
    Number of key frames sampled from previous clip at interval Delta_k.
  • Key frame interval Delta_k = 12
    Sampling stride for key frames going backward in the video.
assumptions (5)
  • domain assumption Frozen Depth Anything V2 encoder features are sufficient for video temporal reasoning; training only the head can adapt image features to video.
    Sec. 3.1 states the encoder is frozen during training to reduce cost and preserve features; if temporal dynamics require backbone-level adaptation, the head-only training would fail.
  • ad hoc to paper Temporal depth gradient matching at the same image coordinate is a valid proxy for temporal consistency, with the sign of depth change ignored.
    Eq. 3 defines the loss using absolute values; the paper posits that |d_{i+1}-d_i| should match |g_{i+1}-g_i|, discarding direction. This is an ad hoc assumption introduced in this paper.
  • domain assumption Synthetic video datasets (TartanAir, VKITTI, PointOdyssey, IRS) plus wild binocular data transfer zero-shot to real-world benchmarks.
    Sec. 4.2 and supplementary training details show the model is never trained on real video depth with dense ground truth.
  • domain assumption The Temporal Alignment Error metric with ground-truth camera poses measures temporal consistency as perceived flicker.
    Supplementary Eq. 5 shows TAE relies on camera poses and reprojection, which may not capture appearance-based flicker or dynamic-object consistency.
  • domain assumption A single scale-shift alignment over the entire 500-frame video is a fair normalization before computing spatial metrics.
    Supplementary Sec. 5 describes this evaluation protocol; it assumes the model's scale and shift are globally consistent over long videos.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Video Depth Anything: Consistent Depth Estimation for Super-Long Videos." pith.science (2026). https://pith.science/paper/C47QNCKD

@misc{pith2026250112375,
  author       = {Pith},
  title        = {Pith review of: Video Depth Anything: Consistent Depth Estimation for Super-Long Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C47QNCKD}},
  note         = {Machine review of arXiv:2501.12375}
}
read the original abstract

Depth Anything has achieved remarkable success in monocular depth estimation with strong generalization ability. However, it suffers from temporal inconsistency in videos, hindering its practical applications. Various methods have been proposed to alleviate this issue by leveraging video generation models or introducing priors from optical flow and camera poses. Nonetheless, these methods are only applicable to short videos (< 10 seconds) and require a trade-off between quality and computational efficiency. We propose Video Depth Anything for high-quality, consistent depth estimation in super-long videos (over several minutes) without sacrificing efficiency. We base our model on Depth Anything V2 and replace its head with an efficient spatial-temporal head. We design a straightforward yet effective temporal consistency loss by constraining the temporal depth gradient, eliminating the need for additional geometric priors. The model is trained on a joint dataset of video depth and unlabeled images, similar to Depth Anything V2. Moreover, a novel key-frame-based strategy is developed for long video inference. Experiments show that our model can be applied to arbitrarily long videos without compromising quality, consistency, or generalization ability. Comprehensive evaluations on multiple video benchmarks demonstrate that our approach sets a new state-of-the-art in zero-shot video depth estimation. We offer models of different scales to support a range of scenarios, with our smallest model capable of real-time performance at 30 FPS.

Figures

Figures reproduced from arXiv: 2501.12375 by the authors.

Figure 1
Figure 1. Left: Our model can generate consistent depth predictions for long videos with rich actions. The demo video shows a 196-second [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall pipeline and the spatio-temporal head. Left: Our model is composed of a backbone encoder from Depth Anything V2 and a newly proposed spatio-temporal head. We jointly train our model on video data using ground-truth depth labels for supervision and on unlabeled images with pseudo labels generated by a teacher model. During training, only the head is learned. Right: Our spatiotemporal head inserts several temp… view at source ↗
Figure 3
Figure 3. Inference strategy for long videos. N is the video clip lenght consumed by our model. Each inference video clip is built by N − To − Tk future frames, To overlapping/adjacent frames, and Tk key frames. The key frames are selected by taking every ∆k-th frame going backward. Then, the new depth predictions will be scale-shift-aligned to the previous frames based on the Tk overlapping frames. We use N = 32, To = 8, Tk … view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Video depth estimation accuracy for different frame length. We compare our model (VDA-L) with DepthCrafter [13] and DepthAnyVideo [41] from 110 to 500 frames on Bonn [24], Scannet [7], and NYUv2 [22]. compared to DAv2-L [43] across most datasets. This demon￾strates tha…
Figure 6
Figure 6. Figure 6: Qualitative comparison for in-the-wild short video depth estimation. We compare with Depth-Anything-V2 [43], DepthCrafter [13] and DepthAnyVideo [41] on videos with less than 100 frames from DAVIS [26]. Red boxes show incorrect depth estimation while blue boxes show in…
Figure 7
Figure 7. Figure 7: Qualitative comparisons of different inference strate￾gies. We compare overlap alignment (OA) with our proposed overlap interpolation and key-frame referencing (OI + KR) on a self-captured video with 7320 frames. Datasets Image-datasets Video-datasets AbsRel (↓) δ1 (↑)…
Figure 8
Figure 8. Figure 8: Qualitative comparison for static image depth esti￾mation. We compare our model with Depth-Anything-V2 [43], DepthCrafter [13], and Depth Any Video [41] on static image depth estimation. Our model demonstrates visualization results compara￾ble to those of Depth-Anythin…
Figure 9
Figure 9. Figure 9: Qualitative comparison for real-world long video depth estimation. We compare with Depth-Anything-V2 [43] and DepthCrafter [13] on 500-frames videos from Scannet [7] and Bonn [24] . We show changes in color and depth over time at the vertical red line in videos. White …
Figure 10
Figure 10. Figure 10: Temporal layer. The feature shape is adjusted for temporal attention. equally during training. Implementation Details The weights are initialized from Depth Anything V2 [43]. Training comprises two stages. In the first stage, synthetic and wild binocular data are used…
Figure 11
Figure 11. Figure 11: 3D Video Conversion. A video from the DAVIS dataset [26] is transformed into a 3D video using our model. frames spanning approximately 5 seconds from the KITTI dataset [11]. After obtaining the inferred inverse depth, we compute the global scale and shift by aligning …
Figure 12
Figure 12. Figure 12: Dense point cloud generation. We compare our model with DepthCrafter [13] and DepthAnyVideo [41] for dense point cloud generation on the KITTI dataset [11]. Our model generates a clean and regular point cloud from multiple frames spanning approximately 5 seconds. In c…
Figure 13
Figure 13. Figure 13: Point cloud generation for wild videos. We compare our method with DepthCrafter [13] and DepthAnyVideo [41] using three videos from DAVIS dataset [26]. Camera intrinsics, along with aligned scale and shift parameters, are derived from processing the first frame of eac…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. UniWorld-View: Large-Baseline View Synthesis via Video Diffusion Models

    cs.CV 2026-08 conditional novelty 6.0 of 10

    UniWorld-View couples an occlusion-aware point cloud renderer with a dual-stream video diffusion model to synthesize large-baseline novel views from monocular video.

  2. X-Lens: Real-Time Metric Depth Estimation with Heterogeneous Cameras

    cs.CV 2026-07 unverdicted novelty 6.0 of 10

    X-Lens fuses arbitrary calibrated fisheye and pinhole views into real-time metric depth at 41 FPS with a 0.04B-parameter model and a new 266K-frame synthetic dataset.

  3. Measuring 3D Spatial Geometric Consistency in Dynamic Video Generation

    cs.CV 2026-03 conditional novelty 6.0 of 10

    SGC quantifies 3D geometric consistency of generated videos by measuring divergence among local camera poses estimated only on static background sub-regions.

  4. Feedback Matters: Augmenting Autonomous Dissection with Visual and Topological Feedback

    cs.RO 2025-10 conditional novelty 6.0 of 10

    A stretch-based tissue connectivity estimator plus an exposure-maximizing controller and recovery planner raised autonomous dissection success on a da Vinci robot to 80%.

  5. Generative Visual Foresight Meets Task-Agnostic Pose Estimation in Robotic Table-Top Manipulation

    cs.RO 2025-08 conditional novelty 6.0 of 10

    GVF-TAPE predicts future RGB-D frames from an image and text, then extracts end-effector poses to control a robot, achieving strong success rates without action-labeled data.

  6. SpatialTrackerV2: 3D Point Tracking Made Easy

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A single feed-forward model jointly estimates video depth, camera poses, and 3D point trajectories from monocular video, setting a new state of the art on TAPVid-3D.

  7. RCG: Safety-Critical Scenario Generation for Robust Autonomous Driving via Real-World Crash Grounding

    cs.RO 2025-07 conditional novelty 6.0 of 10

    RCG replaces handcrafted adversarial scenario scoring with a crash-grounded embedding and k-NN selection, yielding a 9.2% average relative improvement in ego success.

  8. RaCalNet: Radar Calibration Network for Sparse-Supervised Metric Depth Estimation

    cs.CV 2025-06 reject novelty 6.0 of 10

    A radar-camera depth estimation framework that recalibrates sparse radar points and aligns a frozen monocular depth model using sparse LiDAR labels, claiming state-of-the-art accuracy with roughly 1% supervision density.

  9. IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation

    cs.CV 2025-06 reject novelty 6.0 of 10

    A diffusion video model that jointly uses HDR lighting, relit frames, and 3D point tracks to relight videos from text prompts.

  10. E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models

    cs.CV 2025-06 conditional novelty 6.0 of 10

    E3D-Bench compares 16 3D geometric foundation models on depth, reconstruction, pose, and view-synthesis tasks with a unified evaluation toolkit.

  11. UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimation

    cs.CV 2025-05 conditional novelty 6.0 of 10

    Fine-tuning a pretrained video diffusion transformer to predict geometry in one shared global frame produces consistent, camera-free surface normals and coordinates across entire video clips.

  12. Seeing World Dynamics in a Nutshell

    cs.CV 2025-02 conditional novelty 6.0 of 10

    NutWorld is a feed-forward model that represents a monocular video as structured dynamic 3D Gaussians in a canonical orthographic space, trained with depth and flow priors.

  13. RoboScape: Physics-informed Embodied World Model

    cs.CV 2025-06 conditional novelty 5.0 of 10

    RoboScape jointly learns RGB video, depth, and keypoint-token consistency in one autoregressive world model, improving video quality, geometry, action control, synthetic-data policy training, and policy evaluation for...

  14. FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulation

    cs.RO 2025-05 conditional novelty 5.0 of 10

    Explicitly predicting 3D scene flow before diffusion-based image generation improves future-frame prediction and visual planning in RGB-D robot manipulation world models.

  15. Reconstructing 4D Spatial Intelligence: A Survey

    cs.CV 2025-07 accept novelty 4.0 of 10

    A review that classifies 4D scene reconstruction methods into five progressive levels: low-level cues, scene components, dynamic scenes, interactions, and physics.

  16. MM-Gesture: Towards Precise Micro-Gesture Recognition through Multimodal Fusion

    cs.CV 2025-07 conditional novelty 4.0 of 10

    Combining joint, limb, RGB, Taylor-video, optical-flow, and depth streams with two video backbones and a validation-tuned weighted ensemble reaches 73.213% top-1 accuracy on iMiGUE, the best MiGA challenge result to date.

  17. The Fourth Monocular Depth Estimation Challenge

    cs.CV 2025-04 conditional novelty 4.0 of 10

    The fourth MDEC winner, HRI, achieved a 3D F-score of 23.05% on SYNS-Patches, a small improvement over the previous best of 22.58%, under a new two-degree-of-freedom alignment protocol.

Reference graph

Works this paper leans on

57 extracted references · 41 canonical work pages · cited by 17 Pith papers

  1. [1]

    Adabins: Depth estimation using adaptive bins

    Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4009–4018, 2021. 2

    work page 2021

  2. [2]

    Zoedepth: Zero-shot trans- fer by combining relative and metric depth

    Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias Müller. Zoedepth: Zero-shot trans- fer by combining relative and metric depth. arXiv preprint arXiv:2302.12288, 2023. 2

    arXiv 2023

  3. [3]

    Midas v3

    Reiner Birkl, Diana Wofk, and Matthias Müller. Midas v3. 1–a model zoo for robust monocular relative depth estimation. arXiv preprint arXiv:2307.14460, 2023. 1, 2, 4

    arXiv 2023

  4. [4]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 2

    arXiv 2023

  5. [5]

    Butler, Jonas Wulff, Garrett B

    Daniel J. Butler, Jonas Wulff, Garrett B. Stanley, and Michael J. Black. A Naturalistic Open Source Movie for Optical Flow Evaluation, page 611–625. Jan 2012. 5, 6, 13

    work page 2012

  6. [6]

    Vir- tual kitti 2

    Yohann Cabon, Naila Murray, and Martin Humenberger. Vir- tual kitti 2. arXiv preprint arXiv:2001.10773, 2020. 7, 11

    arXiv 2001

  7. [7]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017. 5, 6, 11, 12, 13

    work page 2017

  8. [8]

    Towards real-time monocular depth estimation for robotics: A survey

    Xingshuai Dong, Matthew A Garratt, Sreenatha G Anavatti, and Hussein A Abbass. Towards real-time monocular depth estimation for robotics: A survey. IEEE Transactions on In- telligent Transportation Systems, 23(10):16940–16961, 2022. 2

    work page 2022

Show all 57 references
  1. [9]

    Depth map prediction from a single image using a multi-scale deep network

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep network. Advances in neural information processing systems, 27, 2014. 2

    2014

  2. [10]

    Deep ordinal regression net- work for monocular depth estimation

    Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Bat- manghelich, and Dacheng Tao. Deep ordinal regression net- work for monocular depth estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2002–2011, 2018. 2

    2002

  3. [11]

    Vision meets robotics: The kitti dataset

    Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. The Inter- national Journal of Robotics Research, 32(11):1231–1237,

  4. [12]

    Fast depth densifi- cation for occlusion-aware augmented reality

    Aleksander Holynski and Johannes Kopf. Fast depth densifi- cation for occlusion-aware augmented reality. ACM Transac- tions on Graphics (ToG), 37(6):1–11, 2018. 2

    2018

  5. [13]

    Depthcrafter: Generating consistent long depth sequences for open-world videos

    Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, and Ying Shan. Depthcrafter: Generating consistent long depth sequences for open-world videos. arXiv preprint arXiv:2409.02095, 2024. 2, 3, 5, 6, 7, 8, 11, 12, 13, 14, 15, 16

    2024 arXiv

  6. [14]

    Wildavatar: Web-scale in-the-wild video dataset for 3d avatar creation

    Zihao Huang, ShouKang Hu, Guangcong Wang, Tianqi Liu, Yuhang Zang, Zhiguo Cao, Wei Li, and Ziwei Liu. Wildavatar: Web-scale in-the-wild video dataset for 3d avatar creation. arXiv preprint arXiv:2407.02165, 2024. 1

    2024 arXiv

  7. [15]

    Dai, AndreaF

    Vasiljevic Igor, Kolkin Nicholas, Shanyi Zhang, Ruotian Luo, Haochen Wang, FalconZ. Dai, AndreaF. Daniele, Moham- madreza Mostajabi, Steven Basart, MatthewR. Walter, and Gregory Shakhnarovich. Diode: A dense indoor and outdoor depth dataset. arXiv: Computer Vision and Pattern ...

  8. [16]

    Match- stereo-videos: Bidirectional alignment for consistent dynamic stereo matching

    Junpeng Jing, Ye Mao, and Krystian Mikolajczyk. Match- stereo-videos: Bidirectional alignment for consistent dynamic stereo matching. 2024. 12

    2024

  9. [17]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9492–9502,

  10. [18]

    Ro- bust consistent video depth estimation

    Johannes Kopf, Xuejian Rong, and Jia-Bin Huang. Ro- bust consistent video depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1611–1621, 2021. 2, 3

    2021

  11. [19]

    Towards practical consistent video depth estimation

    Pengzhi Li, Yikang Ding, Linge Li, Jingwei Guan, and Zhi- heng Li. Towards practical consistent video depth estimation. In Proceedings of the 2023 ACM International Conference on Multimedia Retrieval, ICMR ’23, page 388–397, New York, NY , USA, 2023. Association for Computing ...

    2023

  12. [20]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 13

    2017 arXiv

  13. [21]

    Consistent video depth estimation

    Xuan Luo, Jia-Bin Huang, Richard Szeliski, Kevin Matzen, and Johannes Kopf. Consistent video depth estimation. ACM Transactions on Graphics (ToG), 39(4):71–1, 2020. 2, 3

    2020

  14. [22]

    Indoor segmentation and support inference from rgbd images

    Pushmeet Kohli Nathan Silberman, Derek Hoiem and Rob Fergus. Indoor segmentation and support inference from rgbd images. In ECCV, 2012. 5, 6, 13

    2012

  15. [23]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 2

    2023 arXiv

  16. [24]

    Palazzolo, J

    E. Palazzolo, J. Behley, P. Lottes, P. Giguère, and C. Stachniss. ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals. 2019. 5, 6, 11, 12, 13

    2019

  17. [25]

    Controlnext: Powerful and effi- cient control for image and video generation

    Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming- Chang Yang, and Jiaya Jia. Controlnext: Powerful and effi- cient control for image and video generation. arXiv preprint arXiv:2408.06070, 2024. 2

    2024 arXiv

  18. [26]

    Perazzi, J

    F. Perazzi, J. Pont-Tuset, B. McWilliams, L. Van Gool, M. Gross, and A. Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In Computer Vision and Pattern Recognition, 2016. 7, 14, 16

    2016

  19. [27]

    Unidepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10106–10116, 2024. 2

    2024

  20. [28]

    Vi- sion transformers for dense prediction

    René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction. In Proceedings of the IEEE/CVF international conference on computer vision, pages 12179–12188, 2021. 2, 3, 4

    2021

  21. [29]

    Simplere- con: 3d reconstruction without 3d convolutions

    Mohamed Sayed, John Gibson, Jamie Watson, Victor Prisacariu, Michael Firman, and Clément Godard. Simplere- con: 3d reconstruction without 3d convolutions. In European Conference on Computer Vision, pages 1–19. Springer, 2022. 3

    2022

  22. [30]

    Structure- from-motion revisited

    Johannes L Schonberger and Jan-Michael Frahm. Structure- from-motion revisited. In Proceedings of the IEEE confer- ence on computer vision and pattern recognition, pages 4104– 4113, 2016. 2

    2016

  23. [31]

    Schonberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and An- dreas Geiger

    Thomas Schops, Johannes L. Schonberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, and An- dreas Geiger. A multi-view stereo benchmark with high- resolution images and multi-camera videos. In 2017 IEEE Conference on Computer Vision and Pattern Recogni...

    2017

  24. [32]

    Learning tempo- rally consistent video depth from video diffusion priors.arXiv preprint arXiv:2406.01493, 2024

    Jiahao Shao, Yuanbo Yang, Hongyu Zhou, Youmin Zhang, Yujun Shen, Matteo Poggi, and Yiyi Liao. Learning tempo- rally consistent video depth from video diffusion priors.arXiv preprint arXiv:2406.01493, 2024. 2, 3, 5, 6

    2024 arXiv

  25. [33]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pages 402–419. Springer, 2020. 3

    2020

  26. [34]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 3

    2017

  27. [35]

    Irs: A large naturalistic indoor robotics stereo dataset to train deep models for disparity and surface normal estimation, 2021

    Qiang Wang, Shizhen Zheng, Qingsong Yan, Fei Deng, Kaiy- ong Zhao, and Xiaowen Chu. Irs: A large naturalistic indoor robotics stereo dataset to train deep models for disparity and surface normal estimation, 2021. 11

    2021

  28. [36]

    Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision, 2024

    Ruicheng Wang, Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision, 2024. 16

    2024

  29. [37]

    Tartanair: A dataset to push the limits of visual slam

    Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. Tartanair: A dataset to push the limits of visual slam. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4909–4916...

    2020

  30. [38]

    Less is more: Consistent video depth estimation with masked frames modeling

    Yiran Wang, Zhiyu Pan, Xingyi Li, Zhiguo Cao, Ke Xian, and Jianming Zhang. Less is more: Consistent video depth estimation with masked frames modeling. In Proceedings of the 30th ACM International Conference on Multimedia, MM ’22, page 6347–6358. ACM, Oct. 2022. 3

    2022

  31. [39]

    Neural video depth stabilizer

    Yiran Wang, Min Shi, Jiaqi Li, Zihao Huang, Zhiguo Cao, Jianming Zhang, Ke Xian, and Guosheng Lin. Neural video depth stabilizer. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision , pages 9466–9476,

  32. [40]

    Gmflow: Learning optical flow via global matching

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. Gmflow: Learning optical flow via global matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8121–8130,

  33. [41]

    Depth any video with scalable synthetic data

    Honghui Yang, Di Huang, Wei Yin, Chunhua Shen, Haifeng Liu, Xiaofei He, Binbin Lin, Wanli Ouyang, and Tong He. Depth any video with scalable synthetic data. arXiv preprint arXiv:2410.10815, 2024. 2, 3, 5, 6, 7, 11, 12, 13, 14, 15, 16

    2024 arXiv

  34. [42]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In CVPR, 2024. 1, 2, 3

    2024

  35. [43]

    Depth anything v2

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. arXiv:2406.09414, 2024. 1, 2, 3, 5, 6, 7, 8, 11, 12, 13

    2024 arXiv

  36. [44]

    Mamo: Leveraging memory and attention for monocular video depth estimation

    Rajeev Yasarla, Hong Cai, Jisoo Jeong, Yunxiao Shi, Risheek Garrepalli, and Fatih Porikli. Mamo: Leveraging memory and attention for monocular video depth estimation. In Proceed- ings of the IEEE/CVF International Conference on Computer Vision, pages 8754–8764, 2023. 3

    2023

  37. [45]

    Metric3d: Towards zero-shot metric 3d prediction from a single image

    Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaix- uan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from a single image. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9043–9053, 2023. 2

    2023

  38. [46]

    Neural window fully-connected crfs for monocu- lar depth estimation

    Weihao Yuan, Xiaodong Gu, Zuozhuo Dai, Siyu Zhu, and Ping Tan. Neural window fully-connected crfs for monocu- lar depth estimation. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 3916–3925, 2022. 2

    2022

  39. [47]

    Controlvideo: Training- free controllable text-to-video generation

    Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. Controlvideo: Training- free controllable text-to-video generation. arXiv preprint arXiv:2305.13077, 2023. 2

    2023 arXiv

  40. [48]

    Consistent depth of moving objects in video

    Zhoutong Zhang, Forrester Cole, Richard Tucker, William T Freeman, and Tali Dekel. Consistent depth of moving objects in video. ACM Transactions on Graphics (ToG), 40(4):1–12,

  41. [49]

    Pointodyssey: A large-scale synthetic dataset for long-term point tracking

    Yang Zheng, Adam W Harley, Bokui Shen, Gordon Wetzstein, and Leonidas J Guibas. Pointodyssey: A large-scale synthetic dataset for long-term point tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 19855–19865, 2023. 11, 12 Video Depth ...

    2023

  42. [52]

    In-the-wild image results

    More Qualitative Results We present more qualitative comparisons among different approaches for static images and evaluation videos. In-the-wild image results. Static image depth estimation results are shown in Fig. 8. DepthCrafter [ 13] and Depth Any Video [41] exhibit poor p...

  43. [53]

    As shown in Tab

    Short video depth quantitative results We compare our model with DepthCrafter [13] and Depth Any Video [41] on the KITTI [ 11], Bonn [ 24], and Scan- net [7] datasets, with frame lengths of 110, 110, and 90, respectively, corresponding to the settings in [ 13]. As shown in Tab...

  44. [54]

    We believe that with more data, the model’s performance can be further improved, and the backbone network can be unlocked for fine-tuning

    Limitations and future work Our model is trained primarily on publicly available video depth datasets, which may limit its capabilities due to the data quantity. We believe that with more data, the model’s performance can be further improved, and the backbone network can be un...

  45. [55]

    Among the four temporal layers, two are inserted after the Reassemble layers at the two smallest resolutions, and the other two are inserted before the last two Fusion layers

    More Details of Pipeline Spatiotemporal head details. Among the four temporal layers, two are inserted after the Reassemble layers at the two smallest resolutions, and the other two are inserted before the last two Fusion layers. The shape of the feature is transformed into (B...

  46. [56]

    We use a total of five datasets for video depth evaluation: KITTI [ 11], Scannet [ 7], Bonn [24], NYUv2 [ 22], and Sintel [ 5]

    More Details of Evaluation Evaluation dataset details. We use a total of five datasets for video depth evaluation: KITTI [ 11], Scannet [ 7], Bonn [24], NYUv2 [ 22], and Sintel [ 5]. Specifically, we use Scannet [7] and NYUv2 [ 22] for static indoor scenes, Bonn [24] for dynam...

  47. [57]

    Applications Dense point cloud generation. By aligning single frame with metric depth, which can be obtained from a metric depth model or a sparse point cloud acquired through SLAM, our model can generate a depth point cloud for the entire en- vironment using camera informatio...

  48. [2013]

    5, 6, 11, 12, 13, 14, 15

  49. [2023]

    2, 3, 4, 5, 6, 7, 8, 13

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.