Pith. sign in

REVIEW 3 major objections 5 minor 25 references

Unsupervised Video Depth Estimation Based on Ego-motion and Disparity Consensus

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Stereo-pair reconstruction lets unsupervised depth networks train on far fewer videos

desk verdict An incremental stereo-plus-temporal depth estimator whose headline claims are undercut by missing prior work and a confounded efficiency comparison. read the letter →

arxiv 1909.01028 v1 pith:EZIRSTV4 submitted 2019-09-03 cs.CV

Lingtao Zhou , Jiaojiao Fang , Guizhong Liu This is my paper · ORCID
classification cs.CV
keywords unsuperviseddepthestimationmonocularvideoego-motionstereodisparityleft-rightconsistencyviewsynthesisself-supervisedlearningscene
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that an unsupervised monocular depth estimator can learn from unlabeled video plus ordinary stereo pairs, and that the extra view-reconstruction signal makes training much more data-efficient. The method builds on an existing temporal view-synthesis approach and adds two stereo-based supervision terms: reconstructing the left and right images from the predicted disparities, and enforcing consistency between the left and right disparity maps. On an autonomous driving benchmark, the proposed network reaches a mean absolute relative error of 0.195, better than the 0.208 of the temporal-only baseline and better than some supervised methods, while being trained on roughly 16,384 sequences instead of the baseline's 40,109. If this holds, depth estimation for driving scenes needs no ground-truth labels and substantially less unlabeled footage.

What carries the argument

The load-bearing object is the combined view-synthesis loss that sums the temporal ego-motion reconstruction term, the stereo left-right image reconstruction term, and the disparity-consistency term. The stereo terms are what make the method data-efficient: they convert each training sample into two additional reconstruction targets without needing labels, so the depth network receives denser per-pixel supervision. Differentiable bilinear sampling lets gradients flow through both warps, and an explainability mask plus an edge-aware smoothness term disciplines the otherwise under-constrained problem.

What would settle it

Train the proposed method and the temporal-only baseline on the exact same number of video frames and stereo pairs, with the same pre-training and the same optimizer, then compare depth error on the standardized test split; if the proposed method needs as many frames as the baseline when the extra supervision is controlled for, the fewer-data claim is falsified.

Watch

Extended reading notes

Core claim

The central claim is that combining two complementary image-reconstruction tasks in one loss — temporal view synthesis driven by ego-motion and spatial stereo reconstruction driven by disparity — gives a single network that outputs a dense depth map from one image, with no ground-truth depth and no known pose at test time. During training the left frame is reconstructed both from neighboring video frames and from the right stereo frame; the network simultaneously predicts left and right disparities, and a left-right consistency term forces them to agree. An edge-aware L2 smoothness penalty encourages piecewise-smooth depth without blurring object contours. The reported outcome is that this combined supervision improves depth accuracy (Abs Rel 0.195 versus 0.208 for the temporal-only baseline), reduces the amount of training video needed to roughly 60 percent of the baseline, and transfers to a different outdoor dataset without retraining.

Load-bearing premise

The central comparison assumes the extra stereo losses are what allow training on less data, but the proposed pipeline also uses stereo pairs as additional input and a larger pre-training dataset, so the fewer-frames claim has not been shown in isolation.

Editorial extensions

If this is right

  • If the claim is correct, an autonomous-driving depth system can be trained from ordinary dashcam video plus rectified stereo pairs, with no labeled depth and no measured ego-motion.
  • The same extra-stereo supervision should produce usable depth on scenes the network has not seen, since the model trained on one dataset transfers to another without retraining.
  • The training-data savings are large enough that a single GPU can train the model in tens of hours rather than requiring massive unlabeled collections.
  • Because the method learns one network with a fixed scale from stereo geometry, it avoids the global scale ambiguity that plagues purely monocular ego-motion depth estimators.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A conservative reading is that the reported speed-up is not yet isolated: the proposed model also consumes stereo pairs and is pre-trained on a larger city-driving dataset, so the fewer-data comparison mixes new losses with extra input signal and initialization.
  • The recipe suggests a general principle: any additional geometric view — another camera, a known pose, or a second time span — can be added as a reconstruction target to shrink the data needed by self-supervised depth learners.
  • A testable extension would apply the same disparity-consistency idea to unrectified or unknown-intrinsic video by predicting a lightweight epipolar correction, which the paper explicitly leaves to future work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes an unsupervised monocular video depth estimation method that extends the ego-motion-based view synthesis framework of Zhou et al. by adding a stereo left-right image reconstruction loss, a left-right disparity consistency loss, and an edge-aware smoothness term. The authors claim that the method achieves competitive results with the baseline using only 3/5 of the training data, and that it even outperforms some supervised methods. Experiments on KITTI and Make3D are reported, with KITTI results showing Abs Rel 0.195 versus 0.208 for the Zhou et al. baseline.

Significance. If the training-efficiency claim were convincingly supported, the paper would be a meaningful step toward reducing the data cost of unsupervised depth learning. The method combines two well-known ideas in a sensible way, and the reported KITTI results are directionally plausible. The strength of the paper is its use of a fully differentiable, standard geometric pipeline. However, the central efficiency claim is not supported by the current experimental design, and the arithmetic inconsistencies further weaken the contribution. The work could be substantial if the authors provide controlled comparisons that isolate the effect of the proposed losses from the extra stereo supervision and pretraining.

major comments (3)
  1. [Abstract; Section IV.A; Table 1; Section IV.B (Limitations)] The central claim that the method achieves competitive results with the baseline using only 3/5 of the training data is not supported because the comparison is confounded. The proposed method uses the right stereo image as an additional reconstruction target (Eqs. 2 and 8) and is pre-trained on Cityscapes, whereas the Zhou et al. baseline is trained on monocular KITTI sequences only. The improvement seen in Table 1 could therefore be due to the extra stereo supervision or pretraining rather than to a genuine need for fewer samples. The limitations section explicitly concedes that the method requires rectified and temporally aligned stereo pairs for training. A controlled experiment, such as training the baseline with the same stereo losses or training the proposed method without stereo supervision and without pretraining, is needed to substantiate the efficiency claim; without it, the claim collapses.
  2. [Section IV.A; Abstract; Introduction; Section V (Conclusion)] The numerical claim about training data is internally inconsistent. Section IV.A reports 16,384 training sequences versus 40,109 for the baseline, which is approximately 41% (about 2/5), while the abstract and conclusion state 3/5, the introduction states 'half,' and Section III.A mentions '1/2x.' The authors should correct this arithmetic and state the efficiency claim consistently with the actual experimental protocol.
  3. [Section III.A, Eq. (3)] The definition of the disparity consistency loss is unclear. The symbols d_l^t and d_r^t are predicted disparities, but the notation d̂_l^t and d̂_r^t is not precisely defined; the text says they are 'computed by the corresponding disparity map' without specifying the warping operation. For reproducibility, the authors should clearly state that d̂_l^t is the left disparity obtained by sampling d_r^t using the right-to-left warping and vice versa, consistent with Godard et al. [10].
minor comments (5)
  1. [Section IV.A] The paper reports a training time of roughly 32 hours on a single 1080Ti GPU but does not provide the baseline's training time or the number of parameters, making the efficiency claim even harder to evaluate.
  2. [Table 1 and Table 2] The numerical results for the Zhou et al. baseline differ between Table 1 and Table 2 (e.g., Abs Rel 0.208 vs 0.207, RMSE 6.856 vs 6.658). The authors should ensure that the baseline numbers are consistent across tables and clearly state the source of each set of numbers.
  3. [Section IV.B] The claim that the method 'even outperforms the classical supervised methods' is overstated: on Sq Rel and RMSE, the supervised results of Eigen et al. and Liu et al. are better than the proposed method's results.
  4. [Abstract; throughout] The manuscript contains numerous typos and grammatical errors (e.g., 'touch to acquire,' 'dept,' 'un' in the abstract; 'This results is trained' in Section IV.A). A thorough copy-editing pass is needed.
  5. [Section IV.A] The sentence 'This results is trained in a total of 22,801 sequences where 16,384 for training and 5,730 for validation' is ungrammatical and unclear about the origin of the 22,801 total; please rephrase.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: depth/pose are derived from photometric losses and evaluated against external KITTI metrics; the efficiency claim is confounded but not circular.

full rationale

The paper's depth and pose estimates are produced by minimizing photometric view-synthesis losses (Eqs. 2, 7, 8-11), with no fitted constants or parameters whose values define the reported KITTI metrics. The central comparison is against an external baseline (Zhou et al.) and the evaluation uses standard KITTI error/accuracy metrics computed from the authors' released predictions, so the reported numbers are not equivalent to the training objective by construction. There are no self-citations by the present authors carrying any load-bearing argument, no imported uniqueness theorem, and no renamed known result presented as a derivation. The '3/5 training data' efficiency claim is genuinely confounded: the proposed method additionally uses rectified stereo right-image supervision (Eq. 2, Eq. 8), is pre-trained on Cityscapes, and its own stated counts (16,384 vs 40,109 sequences, about 41%) contradict '3/5' in the abstract and 'half' in the introduction. However, this is an experimental-control and reporting-consistency problem, not a case of the paper's prediction reducing to its inputs by definition or by self-citation. Under the required standard of exhibiting a specific reduction Eq. X = Eq. Y by construction, no circular step is present.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new entities. Its losses combine existing photometric and smoothness terms; the only hand-tuned quantities are the four loss weights and a post-processing blend width. The central assumptions are standard for self-supervised depth: known camera intrinsics, static or maskable scenes, rectified stereo, and Lambertian appearance.

free parameters (5)
  • lambda_a (weight for stereo image reconstruction loss) = 0.5
    Set by hand in Section IV.A; no sensitivity analysis provided.
  • lambda_c (weight for disparity consistency loss) = 0.5
    Set by hand in Section IV.A.
  • lambda_s (weight for smoothness loss) = 0.2
    Set by hand in Section IV.A; scaled by 1/r per scale in Section III.C.
  • lambda_e (weight for mask regularization) = 0.2
    Set by hand in Section IV.A.
  • Post-processing blend width = 5%
    The leftmost and rightmost 5% of the disparity map are taken from the original and flipped predictions, Section IV.B. This is a hand-chosen constant.
assumptions (5)
  • domain assumption Pinhole camera model with known intrinsics K
    Used in Eqs. 4-6 to map pixels to 3D and reproject; the authors note in the conclusion that this forbids Internet videos with unknown cameras.
  • domain assumption Static or maskable scene (photometric consistency holds for modeled pixels)
    Inter-frame reconstruction loss in Eq. 9 assumes the scene appearance is explained by camera motion and depth; the mask network is introduced to discount non-modeled pixels, but this still assumes a rigid background.
  • domain assumption Stereo pairs are rectified and temporally aligned
    Required for horizontal disparity warping (Eq. 2) and joint temporal-spatial training; noted in Section IV.6 Limitations.
  • domain assumption Lambertian surfaces (pixel intensity is view-invariant)
    Photometric losses in Eqs. 2 and 7 compare raw pixel intensities; the limitations section admits specular and transparent surfaces produce inconsistent depths.
  • standard math Differentiable bilinear sampling provides gradients
    Used for warping, inherited from Jaderberg et al. [20].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unsupervised Video Depth Estimation Based on Ego-motion and Disparity Consensus." pith.science (2026). https://pith.science/paper/EZIRSTV4

@misc{pith2026190901028,
  author       = {Pith},
  title        = {Pith review of: Unsupervised Video Depth Estimation Based on Ego-motion and Disparity Consensus},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EZIRSTV4}},
  note         = {Machine review of arXiv:1909.01028}
}
read the original abstract

Unsupervised learning based depth estimation methods have received more and more attention as they do not need vast quantities of densely labeled data for training which are touch to acquire. In this paper, we propose a novel unsupervised monocular video depth estimation method in natural scenes by taking advantage of the state-of-the-art method of Zhou et al. which jointly estimates depth and camera motion. Our method advances beyond the baseline method by three aspects: 1) we add an additional signal as supervision to the baseline method by incorporating left-right binocular images reconstruction loss based on the estimated disparities, thus the left frame can be reconstructed by the temporal frames and right frames of stereo vision; 2) the network is trained by jointly using two kinds of view syntheses loss and left-right disparity consistency regularization to estimate depth and pose simultaneously; 3) we use the edge aware smooth L2 regularization to smooth the depth map while preserving the contour of the target. Extensive experiments on the KITTI autonomous driving dataset and Make3D dataset indicate the superiority of our algorithm in training efficiency. We can achieve competitive results with the baseline by only 3/5 times training data. The experimental results also show that our method even outperforms the classical supervised methods that using either ground truth depth or given pose for training.

Figures

Figures reproduced from arXiv: 1909.01028 by the authors.

Figure 1
Figure 1. Overview of the proposed pipeline for depth estimation and camera pose estimation. The depth network takes only the target view as input with the right binocular image for reconstruction supervision, and outputs a dense per-pixel depth map. The pose network takes both the target view and the nearby source views as input, and outputs the relative poses of camera from target image to the source images. mask mask Pose … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 21 canonical work pages

  1. [10]

    Godard, O

    C. Godard, O. M. Aodha and G. J. Brostow . Unsupervised Monocular Depth Estimation with Left -Right Consistency, In CVPR, 2017, pp. 6602-6611

    work page 2017

  2. [1]

    Saxena, M

    A. Saxena, M. Sun, and A. Y. Ng. Make3d: Learning 3d scene structure from a single still image. PAMI, 2009

    work page 2009

  3. [2]

    F. Liu, C. Shen, G. Lin, and I. Reid. Learning depth from single monocular images using deep convolutional neural fields. IEEE transactions on pattern analysis and machine intelligence, 38(10), 2024–2039,2016

    work page 2024

  4. [3]

    Karsch, C

    K. Karsch, C. Liu, and S. B. Kang. Depth transfer: Depth extraction from video using non-parametric sampling. PAMI, 2014

    work page 2014

  5. [4]

    Eigen, C

    D. Eigen, C. Puhrsch, and R. Fergus. Depth map prediction from a single image using a multi-scale deep network. In Advances in Neural Information Processing Systems, 2014

    work page 2014

  6. [5]

    Y. Cao, Z. Wu, and C. Shen. Estimating depth from monocular images as classification using deep fully convolutional residual networks. arXiv preprint arXiv:1605.02305, 2016

    work page Pith review arXiv 2016

  7. [6]

    Laina, C

    I. Laina, C. Rupprecht, V. Belagiannis, F. Tombari, and N. Navab. Deeper depth prediction with fully convolutional residual networks. In 3D Vision (3DV), 2016 Fourth International Conference on , pages 239–248. IEEE, 2016

    work page 2016

  8. [7]

    Flynn, I

    J. Flynn, I. Neulander, J. Philbin, and N. Snavely. D eepstereo: Learning to predict new views from the world’s imagery. In CVPR, 2016

    work page 2016

Show all 25 references
  1. [8]

    J. Xie, R. Girshick, and A. Farhadi. Deep3d: Fully automatic 2d-to-3d video conversion with deep convolutional neural networks. In ECCV, 2016

    2016

  2. [9]

    R. Garg, V. Kumar BG, and I. Reid. Unsupervised CNN for single view depth estimation: Geometry to the rescue. In ECCV, 2016

    2016

  3. [11]

    Ranftl, V

    R. Ranftl, V. Vineet, Q. Chen, and V. Koltun. Dense monocular depth estimation in complex dynamic scenes. In CVPR, 2016

    2016

  4. [12]

    Kendall, M

    A. Kendall, M. Grimes, and R. Cipolla, et al . PoseNet: a convolutional network for real-time 6-dof camera relocalization

  5. [13]

    Karsch, C

    K. Karsch, C. Liu, and S. B. Kang. Depth transfer: Depth extraction from video using non -parametric sampling. IEEE transactions on pattern analysis and machine intelligence , 36(11):2144–2158, 2014

    2014

  6. [14]

    T. Zhou, M. Brown, N. Snavely, and D. G. Lowe. Unsupervised learning of depth and ego-motion from video. In CVPR, vol. 2, pp. 7

  7. [15]

    Yin, and J

    Z. Yin, and J. Shi. Geonet: Unsupervised learning of dense depth, optical flow and camera pose. arXiv preprint arXiv:1803.02276

    arXiv

  8. [16]

    Mahjourian, M

    R. Mahjourian, M. Wicke, and A. Angelova. Un supervised learning of depth and ego motion from monocular video using 3d geometric constraints. arXiv preprint arXiv:1802.05522

    arXiv

  9. [17]

    Ranjan , V

    A . Ranjan , V. Jampani, L. Balles, et al. Adversarial collaboration: Joint unsupervised learning of depth, camera motion, optical flow and motion segmentation. arXiv preprint arXiv:1805.09806, 2018

    2018 arXiv

  10. [18]

    J. R. A. Moniz, C. Beckham, S. Rajotte, et al. Unsupervised Depth Estimation, 3D Face Rotation and Replacement[J]. 2018

    2018

  11. [19]

    Mayer, E

    N. Mayer, E. Ilg, P. Hausser, P. Fischer, D. Cremers, A. Dosovit- skiy, and T. Brox. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In CVPR, 2016. [20]M. Jaderberg, K. Simonyan, A. Zisserman, et al. Spatial transformer n...

    2016

  12. [21]

    Geiger, P

    A. Geiger, P. Lenz, and R. Urtasun, et al . Are we ready for autonomous driving? the KITTI vision benchmark suite. In CVPR, 2012, pp. 3354–3361

    2012

  13. [22]

    P. Adam, G. Sam, C. Soumith, C. Gregory, Y. Edward, et al. Automatic differentiation in PyTorch. In NIPS-W, 2017

    2017

  14. [23]

    Ioffe and C

    S. Ioffe and C. Szegedy. Batch normalization: Accelerating network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015

    2015 arXiv

  15. [24]

    D. P. Kingma, and J Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980,2014

    2014 arXiv

  16. [25]

    Cordts, M

    M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele. The Cityscapes dataset for semantic urban scene understanding. In CVPR, 2016

    2016

  17. [26]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016

    2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.