Pith. sign in

REVIEW 4 major objections 4 minor 89 references

Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokens

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A few trainable tokens let a frozen depth model estimate depth on fisheye images, without retraining, finetuning, or undistorting the input.

desk verdict Calibration tokens are a cheap, practical fix for fisheye depth on frozen perspective models, but the paper's zero-shot generalization to >180-degree cameras is not supported by a training set that never sees rays beyond the perspective FOV. read the letter →

arxiv 2508.04928 v5 pith:VD7KEV2A submitted 2025-08-06 cs.CV cs.AIcs.LG

Rit Gangopadhyay , Jung-Hee Kim , Xien Chen , Patrick Rim , Hyoungseob Park , Alex Wong This is my paper
classification cs.CVcs.AIcs.LG
keywords monoculardepthestimationfisheyecamerascalibrationtokensvisiontransformersself-supervisedlearningcovariateshiftKannala-Brandtdistortionfoundationalmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Foundational monocular depth estimators, trained on tens of millions of perspective images, produce poor depth maps on fisheye images because lens distortion shifts the distribution of their internal representations. The paper claims this shift can be corrected in latent space: appending a small set of trainable Calibration Tokens to each transformer encoder layer modulates fisheye embeddings back toward the perspective-image distribution, so the frozen model estimates depth accurately on fisheye images. The tokens are trained self-supervised, using only perspective images distorted by a polynomial fisheye model, with the loss computed after warping the predicted fisheye depth back into the perspective frame. With eight tokens per layer, the method adds less than one percent memory and latency, improves over prior fisheye depth baselines indoors and outdoors, and leaves perspective performance untouched since the tokens can simply be removed.

What carries the argument

Layer-wise Calibration Tokens $\Phi \in \mathbb{R}^{L \times M \times F}$: a distinct set of $M$ trainable token vectors, $F$-dimensional, appended to the patch-token sequence at each of the $L$ encoder layers of a frozen vision-transformer FMDE and discarded from the output. The attention mechanism lets these tokens modulate fisheye latent embeddings toward the perspective distribution. The training objective warps the fisheye depth prediction back to the perspective frame with the inverse synthetic distortion $T^{-1}$ and minimizes $\log(|\tilde{d} - T^{-1} \circ \hat{d}| + 1)$, so supervision comes from the FMDE's own high-fidelity perspective depth estimates.

What would settle it

Train the tokens with only mild synthetic distortions and test on KITTI-360's >180-degree fisheye images; if adding tokens then fails to improve over the frozen baseline's RMSE, the claim that the synthetic distortion range covers real fisheye cameras is falsified.

Watch

Extended reading notes

Core claim

Calibration Tokens are a set of $M$ trainable vectors appended to the patch-token sequence at every encoder layer of a vision-transformer FMDE, where they modulate the latent embeddings through attention. The paper's central discovery is that this latent-space recalibration is enough: the frozen FMDE, already capable of high-fidelity depth estimation on perspective images, also produces accurate depth on fisheye images once its embeddings are aligned to the perspective embedding distribution. The alignment is learned entirely from perspective images by synthesizing fisheye distortions with the Kannala-Brandt polynomial model, then minimizing the log-$L_1$ difference between the FMDE's perspe

Load-bearing premise

The synthetic fisheye distortions, created by randomly distorting perspective images with Kannala-Brandt polynomial coefficients in the range [-1.0, -0.01], are representative enough of real fisheye cameras that a few tokens trained on synthetic images transfer zero-shot to real fisheye images, including cameras with fields of view beyond 180 degrees.

Editorial extensions

If this is right

  • One frozen FMDE can serve both perspective and fisheye cameras: append Calibration Tokens for fisheye input, omit them for perspective input, with no change to the backbone.
  • No camera calibration parameters are needed at inference; the tokens, trained over a range of synthetic distortions, absorb the calibration information.
  • Training scale is not limited by scarce real fisheye data, since any calibrated perspective image dataset can be synthetically distorted for token training.
  • Because only tokens are trained, adapting to a newer FMDE is much cheaper than full-model finetuning, and the same token mechanism is portable to any transformer-based FMDE.
  • The method consistently improves over the frozen baseline across backbones (12% RMSE reduction for MiDaS, 17% for DepthAnything indoors, 13% for UniDepth indoors) while adding under 0.05% memory and under 1 ms of latency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same latent-alignment mechanism could in principle adapt FMDEs to other camera-induced covariate shifts, such as severe barrel or pincushion distortion, rolling shutter, or synthetic non-lens degradations, as long as they can be generated from perspective images; the paper tests only fisheye distortion.
  • Because supervision is the FMDE's own perspective output, the method's ceiling is the FMDE's perspective accuracy; in scenes where the FMDE misestimates perspective depth, Calibration Tokens cannot recover that error.
  • The zero-shot transfer to KITTI-360's field of view exceeding 180 degrees hints that the tokens learn something more general than the exact synthetic coefficient range used in training; a direct way to test that would be sweeping the synthetic distortion range and measuring the error curve, which the paper does not report.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Calibration Tokens, a set of trainable tokens appended to the encoder layers of frozen Transformer-based foundational monocular depth estimators (FMDEs), to adapt perspective-trained models to fisheye images. The training is self-supervised: perspective images are synthetically distorted with Kannala-Brandt parameters, the FMDE predicts depth for both the perspective and distorted views, and a log-L1 loss is computed in the perspective reference frame after undistorting the fisheye depth prediction. No real fisheye images or ground-truth depth are used for training. The method is evaluated on ScanNet++ (indoor) and KITTI-360 (outdoor) with MiDaS, DepthAnything, and UniDepth, reporting RMSE and δ1 improvements over no-token baselines and comparison to DepthAnyCamera and FoV A-Depth, while claiming a single token set works for both indoor and outdoor scenes.

Significance. If validated, the central idea is attractive and practical: a small set of trainable tokens (<0.1% parameter increase, <1% latency increase) can adapt a frozen FMDE to fisheye cameras without finetuning, while remaining backward-compatible with perspective images and not requiring camera intrinsics at inference. The paper provides code, evaluates on multiple FMDEs, and includes ablations of design choices. The main risks are evaluative rather than architectural: the alignment protocol for relative-depth models is unspecified, and the synthetic distortion distribution may not cover the extreme rays of >180-degree fisheye cameras. These issues are load-bearing for the paper's strongest claims but appear addressable with additional analysis.

major comments (4)
  1. [§4, Table 1, Supp. B.4] MiDaS and DepthAnything produce affine-invariant relative depth, but Table 1 reports RMSE against ground truth without specifying any scale/shift alignment. If raw predictions are compared directly, RMSE is not meaningful; if alignment is used, the procedure must be described. Please state exactly what alignment (per-image least-squares, global, none) was applied, and report scale-invariant metrics such as δ1 consistently. This is necessary to interpret the reported improvements.
  2. [§3.2, Eq. (1), Supp. B.3] Synthetic training data is generated by distorting perspective images, so every ray in the generated fisheye image satisfies θ < 90° relative to the optical axis. KITTI-360's fisheye cameras have FOV exceeding 180°, which includes rays with θ > 90°. Therefore the token set never encounters the geometry of the outermost fisheye regions during training, and the reported KITTI-360 improvements do not by themselves establish zero-shot generalization to >180° fisheye cameras. Please report results restricted to θ ≤ 90° versus θ > 90° (or a comparable FOV breakdown), and temper the generalization claim accordingly.
  3. [Table 1, KITTI-360 row] For DepthAnything on KITTI-360, RMSE improves from 2.214 to 2.043 but δ1 drops from 0.839 to 0.810. The text states that Calibration Tokens 'consistently improve accuracy' and that both MiDaS and DepthAnything improve on RMSE, but the δ1 decrease is not discussed. This is a direct inconsistency in the stated claim. Please address whether this is an alignment artifact or a real trade-off, and avoid claiming consistent improvement across all metrics.
  4. [§4.3, Table 4] The ablation study validates the design choices—LogL1 loss and layer-wise tokens—directly on the same real test sets (ScanNet++ and KITTI-360) that are used for final reporting, without a held-out validation split. This selection-on-test-set inflates the apparent advantage of the chosen configuration. Please add a validation split or explicitly acknowledge this as a limitation, and report the ablated variants on that split.
minor comments (4)
  1. [Eq. (2)] The formula for φ is written as arctan((y − cx)/(x − cy)); the roles of x, y and cx, cy appear swapped. Please correct the notation.
  2. [§5] The Discussion uses 'FMEs' instead of 'FMDEs' in the first sentence.
  3. [Supp. B.4] The text says the authors 'measure relative depth estimation performance to mitigate the gap introduced by fisheye images,' but the reported metrics RMSE and δ1 are defined in absolute/linear depth space. Please clarify how relative depth is handled in the evaluation.
  4. [Table 1, DepthAnyCamera rows] The rows for DepthAnyCamera trained on the 'Mix 200K' set show very large RMSE values, but no training details (optimizer, iterations, resolution) are given. Since this baseline is the main comparison, please include training hyperparameters or clarify whether the same protocol was used.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the training objective and the evaluation are decoupled, and the core improvement is measured against external fisheye ground truth.

full rationale

The paper's derivation chain is self-contained and not circular. Calibration Tokens are trained by synthesizing fisheye distortions from perspective images (Kannala-Brandt model, Eq. 1) and minimizing a self-supervised consistency loss (Eq. 3) between the FMDE's perspective output and the re-projected fisheye output. The use of the FMDE's own perspective estimates as pseudo-targets is a form of knowledge distillation, not a circular reduction, because the evaluation is performed on real fisheye images (ScanNet++, KITTI-360) with external ground truth that was never used in training. The citation of AugUndo [69] is inspirational rather than load-bearing, and the method does not depend on any unverified claim from the authors' prior work. The main caveats are (a) the ablation in Tab. 4 selects design choices (layer-wise tokens, LogL1) on the same test benchmarks used for the final claims, which can inflate results through selection bias, and (b) the synthetic fisheye training distribution, generated from perspective images, likely cannot cover rays with theta > 90 degrees, so generalization to FOV > 180 cameras like KITTI-360 is not fully established. Both are correctness/generalization concerns, not examples of a prediction being equivalent to its inputs by construction. Hence no circular step is identified.

Assumptions & free parameters 2 free parameters · 3 assumptions · 1 invented entities

The method's free parameters are the token count and the synthetic distortion range, both chosen by hand. The central assumptions are that the synthetic distortion model covers real fisheye lenses, that encoder-only modulation suffices, and that the FMDE's perspective outputs are reliable training targets. No external entities are postulated beyond the token mechanism itself.

free parameters (2)
  • Number of calibration tokens per layer (M) = 8
    Chosen by hand; not swept in the paper, and it directly controls the capacity of the adaptation mechanism.
  • Kannala-Brandt distortion coefficient sampling range = [-1.0, -0.01]
    Range used to synthesize fisheye images during training (Supp. B.3); determines the distribution the tokens are trained on and is not justified from real fisheye cameras.
assumptions (3)
  • domain assumption The Kannala-Brandt polynomial model (Eq. 1) can represent real fisheye distortions sufficiently well for training tokens that transfer to real cameras.
    Invoked at Sec 3.2 when synthesizing fisheye images; if real fisheye lenses, especially FOV > 180 degrees, are far from this model, the training signal is misaligned.
  • domain assumption Appending trainable tokens and modulating attention in the encoder is sufficient to transform fisheye embeddings into perspective-like embeddings; the decoder needs no modification.
    Sec 3.1 states tokens are only applied to encoder layers; if the decoder also encodes camera-specific bias, the alignment is incomplete.
  • domain assumption The FMDE's perspective depth estimates are high-fidelity enough to serve as training targets (pseudo ground truth).
    Loss Eq. 3 uses the FMDE's perspective output as target; systematic FMDE errors on perspective images will be reproduced in the fisheye adaptation.
invented entities (1)
  • Calibration Tokens
    purpose: Trainable parameter vectors appended to each encoder layer of a frozen transformer-based FMDE to shift fisheye image embeddings toward the perspective embedding distribution.
    The tokens are a new mechanism proposed by this paper; there is no independent falsifiable prediction outside the paper's own experiments. They are internal parameters, not a new physical entity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokens." pith.science (2026). https://pith.science/paper/VD7KEV2A

@misc{pith2026250804928,
  author       = {Pith},
  title        = {Pith review of: Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokens},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VD7KEV2A}},
  note         = {Machine review of arXiv:2508.04928}
}
read the original abstract

We propose a method to extend foundational monocular depth estimators (FMDEs), trained on perspective images, to fisheye images. Despite being trained on tens of millions of images, FMDEs are susceptible to the covariate shift introduced by changes in camera calibration (intrinsic, distortion) parameters, leading to erroneous depth estimates. Our method aligns the distribution of latent embeddings encoding fisheye images to those of perspective images, enabling the reuse of FMDEs for fisheye cameras without retraining or finetuning. To this end, we introduce a set of Calibration Tokens as a light-weight adaptation mechanism that modulates the latent embeddings for alignment. By exploiting the already expressive latent space of FMDEs, we posit that modulating their embeddings avoids the negative impact of artifacts and loss introduced in conventional recalibration or map projection to a canonical reference frame in the image space. Our method is self-supervised and does not require fisheye images but leverages publicly available large-scale perspective image datasets. This is done by recalibrating perspective images to fisheye images, and enforcing consistency between their estimates during training. We evaluate our approach with several FMDEs, on both indoors and outdoors, where we consistently improve over state-of-the-art methods using a single set of tokens for both. Code available at: https://github.com/JungHeeKim29/calibration-token; https://github.com/Suchisrit/CalibrationTokens.

Figures

Figures reproduced from arXiv: 2508.04928 by the authors.

Figure 1
Figure 1. Foundational monocular depth estimators fail on fisheye images. Despite being trained on large-scale datasets, foundational monocular depth estimators (FMDEs) models pro￾duces erroneous outputs. The inaccurate, blurry estimates are caused by a covariate shift that stem from fisheye distortion. eye or other wide-angle cameras, which allows for wider coverage of the 3D environment. However, images cap￾tured by these c… view at source ↗
Figure 2
Figure 2. Inference on different cameras. Calibration Tokens enable foundational monocular depth estimators to adapt to fisheye images while maintaining performance on perspective images. induce artificial distortion on perspective images to create pairs of perspective and synthetic fisheye images with di￾verse fisheye distortions. However, rather than doing the same in the output space, we undistort the fisheye depth maps to… view at source ↗
Figure 3
Figure 3. Overview of our method. We introduce a set of trainable Calibration Tokens, which is appended to the input sequence of the fisheye image tokens. The Calibration Tokens are trained to adapt the model to produce accurate depth maps for images with various fisheye distortions. A unique fisheye calibration token is appended to the input of each new layer of the encoder. 3.1. Extending FMDEs with Calibration Tokens Speci… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Comparison on ScanNet++(Indoor) [77] and KITTI-360 [36] dataset. Qualitative comparison results on ScanNet++ and KITTI-360 datasets. Here, +C. T. indicates prediction results by appending Calibration Tokens to patch embeddings of the model located above. Calibration To…
Figure 6
Figure 6. Figure 6: Comparison of attention maps for single and multi￾layer tokens. We visualize the attention map of Calibration To￾kens to the encoded patch embeddings. Calibration Tokens attend highly to distorted border regions: “Single Token” (top) has little effect in most layers du…
Figure 8
Figure 8. Figure 8: Validation on LogL1 loss. We evaluate the effectiveness of our LogL1 loss by comparing a single-layer token baseline with an additional LogL1 loss. Incorporating LogL1 loss helps model to mitigate artifacts in the highlighted border regions of fisheye images, leading t…
Figure 9
Figure 9. Figure 9: 3D reconstruction result of UniDepth predictions on ScanNet++ dataset. results [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Additional comparison results on ScanNet++ dataset [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Additional comparison results on KITTI-360 dataset [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

89 extracted references · 73 canonical work pages

  1. [1]

    Rotinvmtl: Rotation invariant multinet on fisheye images for autonomous driving applications

    Bruno Arsenali, Prashanth Viswanath, and Jelena Novosel. Rotinvmtl: Rotation invariant multinet on fisheye images for autonomous driving applications. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pages 0–0, 2019. 3

    2019

  2. [2]

    Adabins: Depth estimation using adaptive bins

    Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka. Adabins: Depth estimation using adaptive bins. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4009–4018, 2021. 3

    2021

  3. [3]

    Depth pro: Sharp monocular metric depth in less than a second

    Aleksei Bochkovskii, Ama ˜AG ¸ l Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second. In International Conference on Learning Representations, 2025. 3

    2025

  4. [4]

    Memory transformer

    Mikhail S Burtsev, Yuri Kuratov, Anton Peganov, and Grig- ory V Sapunov. Memory transformer. arXiv preprint arXiv:2006.11527, 2020. 4

    arXiv 2006

  5. [5]

    Transformer-based monocular depth estimation with attention supervision

    Wenjie Chang, Yueyi Zhang, and Zhiwei Xiong. Transformer-based monocular depth estimation with attention supervision. In 32nd British Machine Vision Conference (BMVC 2021), 2021. 3

    2021

  6. [6]

    Uncle: Bench- marking unsupervised continual learning for depth comple- tion

    Xien Chen, Suchisrit Gangopadhyay, Michael Chu, Patrick Rim, Hyoungseob Park, and Alex Wong. Uncle: Bench- marking unsupervised continual learning for depth comple- tion. arXiv preprint arXiv:2410.18074, 2024. 3

    arXiv 2024

  7. [7]

    Adaptive confidence thresholding for monocular depth esti- mation

    Hyesong Choi, Hunsang Lee, Sunkyung Kim, Sunok Kim, Seungryong Kim, Kwanghoon Sohn, and Dongbo Min. Adaptive confidence thresholding for monocular depth esti- mation. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 12808–12818, 2021. 3

    2021

  8. [8]

    Duncan, and Alex Wong

    Younjoon Chung, Hyoungseob Park, Patrick Rim, Xiaoran Zhang, Jihe He, Ziyao Zeng, Safa Cicek, Byung-Woo Hong, James S. Duncan, and Alex Wong. Eta: Energy-based test- time adaptation for depth completion. In Proceedings of the IEEE/CVF International Conference on Computer Vision ,

Show all 89 references
  1. [9]

    Vision transformers need registers

    Timoth ´ee Darcet, Maxime Oquab, Julien Mairal, and Pi- otr Bojanowski. Vision transformers need registers. arXiv preprint arXiv:2309.16588, 2023. 4

    2023 arXiv

  2. [10]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding. In North American Chapter of the Association for Computational Linguistics ,

  3. [11]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  4. [12]

    Close-range camera calibration

    C Brown Duane. Close-range camera calibration. Pho- togramm. Eng, 37(8):855–866, 1971. 3

    1971

  5. [13]

    Predicting depth, surface nor- mals and semantic labels with a common multi-scale con- volutional architecture

    David Eigen and Rob Fergus. Predicting depth, surface nor- mals and semantic labels with a common multi-scale con- volutional architecture. In Proceedings of the IEEE inter- national conference on computer vision , pages 2650–2658,

  6. [14]

    Depth map prediction from a single image using a multi-scale deep net- work

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep net- work. Advances in neural information processing systems , 27, 2014. 3, 15

    2014

  7. [15]

    All-day depth completion

    Vadim Ezhov, Hyoungseob Park, Zhaoyang Zhang, Rishi Upadhyay, Howard Zhang, Chethan Chinder Chandrappa, Achuta Kadambi, Yunhao Ba, Julie Dorsey, and Alex Wong. All-day depth completion. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE,

    2024

  8. [16]

    Geo- supervised visual depth prediction

    Xiaohan Fei, Alex Wong, and Stefano Soatto. Geo- supervised visual depth prediction. IEEE Robotics and Au- tomation Letters, 4(2):1661–1668, 2019. 3

    2019

  9. [17]

    Simfir: A simple framework for fisheye image rectification with self-supervised representation learn- ing

    Hao Feng, Wendi Wang, Jiajun Deng, Wengang Zhou, Li Li, and Houqiang Li. Simfir: A simple framework for fisheye image rectification with self-supervised representation learn- ing. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 12418–12427, ...

    2023

  10. [18]

    Deep ordinal regression net- work for monocular depth estimation

    Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Bat- manghelich, and Dacheng Tao. Deep ordinal regression net- work for monocular depth estimation. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 2002–2011, 2018. 3

    2002

  11. [19]

    Unsupervised cnn for single view depth estimation: Geom- etry to the rescue

    Ravi Garg, Vijay Kumar Bg, Gustavo Carneiro, and Ian Reid. Unsupervised cnn for single view depth estimation: Geom- etry to the rescue. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, Octo- ber 11-14, 2016, Proceedings, Part VIII 14, pages 7...

    2016

  12. [20]

    Unsupervised monocular depth estimation with left- right consistency

    Cl ´ement Godard, Oisin Mac Aodha, and Gabriel J Bros- tow. Unsupervised monocular depth estimation with left- right consistency. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 270–279,

  13. [21]

    Digging into self-supervised monocular depth estimation

    Cl ´ement Godard, Oisin Mac Aodha, Michael Firman, and Gabriel J Brostow. Digging into self-supervised monocular depth estimation. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 3828–3838,

  14. [22]

    Semantically-guided representation learning for self-supervised monocular depth

    Vitor Guizilini, Rui Hou, Jie Li, Rares Ambrus, and Adrien Gaidon. Semantically-guided representation learning for self-supervised monocular depth. In International Confer- ence on Learning Representations, 2019. 3

    2019

  15. [23]

    3d packing for self-supervised monocular depth estimation

    Vitor Guizilini, Rares Ambrus, Sudeep Pillai, Allan Raven- tos, and Adrien Gaidon. 3d packing for self-supervised monocular depth estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2485–2494, 2020. 3

    2020

  16. [24]

    Depth any camera: Zero-shot met- ric depth estimation from any camera

    Yuliang Guo, Sparsh Garg, S Mahdi H Miangoleh, Xinyu Huang, and Liu Ren. Depth any camera: Zero-shot met- ric depth estimation from any camera. arXiv preprint arXiv:2501.02464, 2025. 3, 5, 7

    2025 arXiv

  17. [25]

    Harris and M

    Christopher G. Harris and M. J. Stephens. A combined cor- ner and edge detector. In Alvey Vision Conference, 1988. 14

    1988

  18. [26]

    A generic camera calibration method for fish-eye lenses

    Juho Kannala and Sami Brandt. A generic camera calibration method for fish-eye lenses. In Proceedings of the 17th In- ternational Conference on Pattern Recognition, 2004. ICPR 2004., pages 10–13. IEEE, 2004. 3, 4, 15

    2004

  19. [27]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9492–9502,

  20. [28]

    A history of the photographic lens

    R Kingslake. A history of the photographic lens. University of Rochester NY, 145, 1989. 3

    1989

  21. [29]

    Syndistnet: Self-supervised monocular fisheye camera distance estima- tion synergized with semantic segmentation for autonomous driving

    Varun Ravi Kumar, Marvin Klingner, Senthil Yogamani, Ste- fan Milz, Tim Fingscheidt, and Patrick Mader. Syndistnet: Self-supervised monocular fisheye camera distance estima- tion synergized with semantic segmentation for autonomous driving. In Proceedings of the IEEE/CVF winte...

    2021

  22. [30]

    Deeper depth prediction with fully convolutional residual networks

    Iro Laina, Christian Rupprecht, Vasileios Belagiannis, Fed- erico Tombari, and Nassir Navab. Deeper depth prediction with fully convolutional residual networks. In 2016 Fourth international conference on 3D vision (3DV) , pages 239–

    2016

  23. [31]

    Sub-token vit embedding via stochastic reso- nance transformers

    Dong Lao, Yangchao Wu, Tian Yu Liu, Alex Wong, and Ste- fano Soatto. Sub-token vit embedding via stochastic reso- nance transformers. In International Conference on Machine Learning. PMLR, 2024

    2024

  24. [32]

    On the viabil- ity of monocular depth pre-training for semantic segmenta- tion

    Dong Lao, Fengyu Yang, Daniel Wang, Hyoungseob Park, Samuel Lu, Alex Wong, and Stefano Soatto. On the viabil- ity of monocular depth pre-training for semantic segmenta- tion. In European Conference on Computer Vision. Springer, 2024

    2024

  25. [33]

    Depth and surface normal estimation from monocular images using regression on deep features and hi- erarchical crfs

    Bo Li, Chunhua Shen, Yuchao Dai, Anton Van Den Hengel, and Mingyi He. Depth and surface normal estimation from monocular images using regression on deep features and hi- erarchical crfs. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1119–1127,

  26. [34]

    Depthformer: Exploiting long-range correlation and local in- formation for accurate monocular depth estimation.Machine Intelligence Research, 20(6):837–854, 2023

    Zhenyu Li, Zehui Chen, Xianming Liu, and Junjun Jiang. Depthformer: Exploiting long-range correlation and local in- formation for accurate monocular depth estimation.Machine Intelligence Research, 20(6):837–854, 2023. 3

    2023

  27. [35]

    Dr-gan: Automatic radial distortion rectification using con- ditional gan in real-time

    Kang Liao, Chunyu Lin, Yao Zhao, and Moncef Gabbouj. Dr-gan: Automatic radial distortion rectification using con- ditional gan in real-time. IEEE Transactions on Circuits and Systems for Video Technology, 30(3):725–733, 2019. 3

    2019

  28. [36]

    Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d

    Yiyi Liao, Jun Xie, and Andreas Geiger. Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(3):3292–3310, 2022. 5, 6, 7, 15

    2022

  29. [37]

    Fova-depth: Field-of-view agnostic depth es- timation for cross-dataset generalization

    Daniel Lichy, Hang Su, Abhishek Badki, Jan Kautz, and Orazio Gallo. Fova-depth: Field-of-view agnostic depth es- timation for cross-dataset generalization. In 2024 Interna- tional Conference on 3D Vision (3DV) , pages 1–10. IEEE,

    2024

  30. [38]

    Learning depth from single monocular images using deep convolutional neural fields

    Fayao Liu, Chunhua Shen, Guosheng Lin, and Ian Reid. Learning depth from single monocular images using deep convolutional neural fields. IEEE transactions on pattern analysis and machine intelligence, 38(10):2024–2039, 2015. 3

    2024

  31. [39]

    Monitored distillation for positive congruent depth completion

    Tian Yu Liu, Parth Agrawal, Allison Chen, Byung-Woo Hong, and Alex Wong. Monitored distillation for positive congruent depth completion. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part II , pages 35–53. Springer,

    2022

  32. [40]

    Hr-depth: High resolution self-supervised monocular depth estimation

    Xiaoyang Lyu, Liang Liu, Mengmeng Wang, Xin Kong, Lina Liu, Yong Liu, Xinxin Chen, and Yi Yuan. Hr-depth: High resolution self-supervised monocular depth estimation. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 2294–2301, 2021. 3

    2021

  33. [41]

    Un- supervised learning of depth and ego-motion from monocu- lar video using 3d geometric constraints

    Reza Mahjourian, Martin Wicke, and Anelia Angelova. Un- supervised learning of depth and ego-motion from monocu- lar video using 3d geometric constraints. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 5667–5675, 2018. 3

    2018

  34. [42]

    Fish eye lens

    Kenro Miyamoto. Fish eye lens. JOSA, 54(8):1060–1061,

  35. [43]

    Test- time adaptation for depth completion

    Hyoungseob Park, Anjali Gupta, and Alex Wong. Test- time adaptation for depth completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20519–20529, 2024. 3

    2024

  36. [44]

    Excavating the potential capacity of self- supervised monocular depth estimation

    Rui Peng, Ronggang Wang, Yawen Lai, Luyang Tang, and Yangang Cai. Excavating the potential capacity of self- supervised monocular depth estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 15560–15569, 2021. 5

    2021

  37. [45]

    Unidepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10106–10116, 2024. 1, 2, 3

    2024

  38. [46]

    Unidepthv2: Universal monocular metric depth estimation made simpler

    Luigi Piccinelli, Christos Sakaridis, Yung-Hsu Yang, Mat- tia Segu, Siyuan Li, Wim Abbeloos, and Luc Van Gool. Unidepthv2: Universal monocular metric depth estimation made simpler. arXiv preprint arXiv:2502.20110, 2025. 5, 7, 14

    2025 arXiv

  39. [47]

    Learning monocular depth estimation with unsupervised trinocular as- sumptions

    Matteo Poggi, Fabio Tosi, and Stefano Mattoccia. Learning monocular depth estimation with unsupervised trinocular as- sumptions. In 2018 International conference on 3d vision (3DV), pages 324–333. IEEE, 2018. 3

    2018

  40. [48]

    On the uncertainty of self-supervised monocular depth estimation

    Matteo Poggi, Filippo Aleotti, Fabio Tosi, and Stefano Mat- toccia. On the uncertainty of self-supervised monocular depth estimation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 3227–3237, 2020. 3

    2020

  41. [49]

    Vi- sion transformers for dense prediction

    Ren ´e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021. 1, 2, 3

    2021

  42. [50]

    Towards Robust Monocu- lar Depth Estimation: Mixing Datasets for Zero-Shot Cross- Dataset Transfer

    Rene Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards Robust Monocu- lar Depth Estimation: Mixing Datasets for Zero-Shot Cross- Dataset Transfer . IEEE Transactions on Pattern Analysis & Machine Intelligence, 44(03):1623–1637, 2022. 2, 3, 5, 7, 14

    2022

  43. [51]

    Radar-guided polynomial fitting for metric depth estimation

    Patrick Rim, Hyoungseob Park, Vadim Ezhov, Jeffrey Moon, and Alex Wong. Radar-guided polynomial fitting for metric depth estimation. arXiv preprint arXiv:2503.17182, 2025. 3

    2025 arXiv

  44. [52]

    Protodepth: Unsupervised contin- ual depth completion with prototypes

    Patrick Rim, Hyoungseob Park, Ziyao Zeng, Younjoon Chung, and Alex Wong. Protodepth: Unsupervised contin- ual depth completion with prototypes. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 6304–6316, 2025. 3

    2025

  45. [53]

    Hypersim: A photorealistic syn- thetic dataset for holistic indoor scene understanding

    Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M Susskind. Hypersim: A photorealistic syn- thetic dataset for holistic indoor scene understanding. In Proceedings of the IEEE/CVF international conference o...

    2021

  46. [54]

    Indoor segmentation and support inference from rgbd images

    Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In European Conference on Computer Vision,

  47. [55]

    Depth estimation from camera image and mmwave radar point cloud

    Akash Deep Singh, Yunhao Ba, Ankur Sarker, Howard Zhang, Achuta Kadambi, Stefano Soatto, Mani Srivastava, and Alex Wong. Depth estimation from camera image and mmwave radar point cloud. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages...

    2023

  48. [56]

    Nonparametric correction of distortion

    Daniel E Stevenson and Margaret M Fleck. Nonparametric correction of distortion. In Proceedings Third IEEE Work- shop on Applications of Computer Vision. WACV’96 , pages 214–219. IEEE, 1996. 3

    1996

  49. [57]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. In Proceed- ings of the IEEE/CVF conference on computer ...

    2020

  50. [58]

    Learning monocular depth estimation infusing tradi- tional stereo knowledge

    Fabio Tosi, Filippo Aleotti, Matteo Poggi, and Stefano Mat- toccia. Learning monocular depth estimation infusing tradi- tional stereo knowledge. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9799–9809, 2019. 3

    2019

  51. [59]

    Training data-efficient image transformers and distillation through at- tention

    Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou. Training data-efficient image transformers and distillation through at- tention. In Proceedings of the 38th International Conference on Machine Learning, pages 10347–10357. P...

    2021

  52. [60]

    Enhancing diffusion models with 3d perspec- tive geometry constraints

    Rishi Upadhyay, Howard Zhang, Yunhao Ba, Ethan Yang, Blake Gella, Sicheng Jiang, Alex Wong, and Achuta Kadambi. Enhancing diffusion models with 3d perspec- tive geometry constraints. ACM Transactions on Graphics (TOG), 42(6):1–15, 2023. 3

    2023

  53. [61]

    Learning depth from monocular videos using direct methods

    Chaoyang Wang, Jos ´e Miguel Buenaposada, Rui Zhu, and Simon Lucey. Learning depth from monocular videos using direct methods. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2022–2030,

    2022

  54. [62]

    Irs: A large naturalistic indoor robotics stereo dataset to train deep models for dis- parity and surface normal estimation

    Qiang Wang, Shizhen Zheng, Qingsong Yan, Fei Deng, Kaiyong Zhao, and Xiaowen Chu. Irs: A large naturalistic indoor robotics stereo dataset to train deep models for dis- parity and surface normal estimation. In 2021 IEEE Interna- tional Conference on Multimedia and Expo (ICME) ...

    2021

  55. [63]

    Self-supervised monocular depth hints

    Jamie Watson, Michael Firman, Gabriel J Brostow, and Daniyar Turmukhambetov. Self-supervised monocular depth hints. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 2162–2171, 2019. 3

    2019

  56. [64]

    Bilateral cyclic con- straint and adaptive regularization for unsupervised monoc- ular depth prediction

    Alex Wong and Stefano Soatto. Bilateral cyclic con- straint and adaptive regularization for unsupervised monoc- ular depth prediction. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 5644–5653, 2019

    2019

  57. [65]

    Targeted ad- versarial perturbations for monocular depth prediction

    Alex Wong, Safa Cicek, and Stefano Soatto. Targeted ad- versarial perturbations for monocular depth prediction. Ad- vances in neural information processing systems , 33:8486– 8497, 2020

    2020

  58. [66]

    Unsupervised depth completion from visual iner- tial odometry

    Alex Wong, Xiaohan Fei, Stephanie Tsuei, and Stefano Soatto. Unsupervised depth completion from visual iner- tial odometry. IEEE Robotics and Automation Letters, 5(2): 1899–1906, 2020. 5, 14

    1906

  59. [67]

    Learning topol- ogy from synthetic data for unsupervised depth comple- tion

    Alex Wong, Safa Cicek, and Stefano Soatto. Learning topol- ogy from synthetic data for unsupervised depth comple- tion. IEEE Robotics and Automation Letters , 6(2):1495– 1502, 2021

    2021

  60. [68]

    An adaptive framework for learning unsupervised depth completion

    Alex Wong, Xiaohan Fei, Byung-Woo Hong, and Stefano Soatto. An adaptive framework for learning unsupervised depth completion. IEEE Robotics and Automation Letters, 6 (2):3120–3127, 2021. 3

    2021

  61. [69]

    Augundo: Scaling up augmentations for monocular depth completion and estima- tion

    Yangchao Wu, Tian Yu Liu, Hyoungseob Park, Stefano Soatto, Dong Lao, and Alex Wong. Augundo: Scaling up augmentations for monocular depth completion and estima- tion. In European Conference on Computer Vision , pages 274–293. Springer, 2024. 3, 5

    2024

  62. [70]

    Quadric representations for lidar odometry, mapping and localization

    Chao Xia, Chenfeng Xu, Patrick Rim, Mingyu Ding, Nan- ning Zheng, Kurt Keutzer, Masayoshi Tomizuka, and Wei Zhan. Quadric representations for lidar odometry, mapping and localization. IEEE Robotics and Automation Letters , 8 (8):5023–5030, 2023. 3

    2023

  63. [71]

    Sparsefusion: Fusing multi-modal sparse rep- resentations for multi-sensor 3d object detection

    Yichen Xie, Chenfeng Xu, Marie-Julie Rakotosaona, Patrick Rim, Federico Tombari, Kurt Keutzer, Masayoshi Tomizuka, and Wei Zhan. Sparsefusion: Fusing multi-modal sparse rep- resentations for multi-sensor 3d object detection. InProceed- ings of the IEEE/CVF International Confer...

    2023

  64. [72]

    Multi-scale continuous crfs as sequential deep networks for monocular depth estimation

    Dan Xu, Elisa Ricci, Wanli Ouyang, Xiaogang Wang, and Nicu Sebe. Multi-scale continuous crfs as sequential deep networks for monocular depth estimation. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 5354–5362, 2017. 3

    2017

  65. [73]

    Binding touch to everything: Learning unified multimodal tactile rep- resentations

    Fengyu Yang, Chao Feng, Ziyang Chen, Hyoungseob Park, Daniel Wang, Yiming Dou, Ziyao Zeng, Xien Chen, Rit Gangopadhyay, Andrew Owens, and Alex Wong. Binding touch to everything: Learning unified multimodal tactile rep- resentations. In Proceedings of the IEEE/CVF Conference on...

    2024

  66. [74]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 1, 2, 3, 5, 7, 14

    2024

  67. [75]

    Dense depth posterior (ddp) from single image and sparse range

    Yanchao Yang, Alex Wong, and Stefano Soatto. Dense depth posterior (ddp) from single image and sparse range. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3353–3362, 2019. 3

    2019

  68. [76]

    Lego: Learning edge with geometry all at once by watching videos

    Zhenheng Yang, Peng Wang, Yang Wang, Wei Xu, and Ram Nevatia. Lego: Learning edge with geometry all at once by watching videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 225–234,

  69. [77]

    Scannet++: A high-fidelity dataset of 3d in- door scenes

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high-fidelity dataset of 3d in- door scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12–22, 2023. 5, 6, 7, 14

    2023

  70. [78]

    En- forcing geometric constraints of virtual normal for depth pre- diction

    Wei Yin, Yifan Liu, Chunhua Shen, and Youliang Yan. En- forcing geometric constraints of virtual normal for depth pre- diction. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 5684–5693, 2019. 3

    2019

  71. [79]

    Fisheyerecnet: A multi-context collaborative deep network for fisheye image rectification

    Xiaoqing Yin, Xinchao Wang, Jun Yu, Maojun Zhang, Pas- cal Fua, and Dacheng Tao. Fisheyerecnet: A multi-context collaborative deep network for fisheye image rectification. In Proceedings of the European conference on computer vision (ECCV), pages 469–484, 2018. 4

    2018

  72. [80]

    Fisheyebevseg: Surround view fisheye cameras based bird’s-eye view segmentation for au- tonomous driving

    Senthil Yogamani, David Unger, Venkatraman Narayanan, and Varun Ravi Kumar. Fisheyebevseg: Surround view fisheye cameras based bird’s-eye view segmentation for au- tonomous driving. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 1...

    2024

  73. [81]

    Neural window fully-connected crfs for monocu- lar depth estimation

    Weihao Yuan, Xiaodong Gu, Zuozhuo Dai, Siyu Zhu, and Ping Tan. Neural window fully-connected crfs for monocu- lar depth estimation. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 3916–3925, 2022. 3

    2022

  74. [82]

    Priordiffusion: Leverage language prior in diffu- sion models for monocular depth estimation

    Ziyao Zeng, Jingcheng Ni, Daniel Wang, Patrick Rim, Youn- joon Chung, Fengyu Yang, Byung-Woo Hong, and Alex Wong. Priordiffusion: Leverage language prior in diffu- sion models for monocular depth estimation. arXiv preprint arXiv:2411.16750, 2024. 3

    2024

  75. [83]

    Wordepth: Vari- ational language prior for monocular depth estimation

    Ziyao Zeng, Daniel Wang, Fengyu Yang, Hyoungseob Park, Stefano Soatto, Dong Lao, and Alex Wong. Wordepth: Vari- ational language prior for monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 9708–9719, 2024

    2024

  76. [84]

    Rsa: Resolving scale ambiguities in monoc- ular depth estimators through language descriptions

    Ziyao Zeng, Yangchao Wu, Hyoungseob Park, Daniel Wang, Fengyu Yang, Stefano Soatto, Dong Lao, Byung-Woo Hong, and Alex Wong. Rsa: Resolving scale ambiguities in monoc- ular depth estimators through language descriptions. Ad- vances in neural information processing systems , 37...

    2024

  77. [85]

    Unsupervised learn- ing of monocular depth estimation and visual odometry with deep feature reconstruction

    Huangying Zhan, Ravi Garg, Chamara Saroj Weerasekera, Kejie Li, Harsh Agarwal, and Ian Reid. Unsupervised learn- ing of monocular depth estimation and visual odometry with deep feature reconstruction. In Proceedings of the IEEE con- ference on computer vision and pattern recog...

    2018

  78. [86]

    Lite-mono: A lightweight cnn and transformer architecture for self-supervised monocular depth estimation

    Ning Zhang, Francesco Nex, George V osselman, and Nor- man Kerle. Lite-mono: A lightweight cnn and transformer architecture for self-supervised monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18537–18546, 2023. 3

    2023

  79. [87]

    Monovit: Self-supervised monocular depth estimation with a vision transformer

    Chaoqiang Zhao, Youmin Zhang, Matteo Poggi, Fabio Tosi, Xianda Guo, Zheng Zhu, Guan Huang, Yang Tang, and Stefano Mattoccia. Monovit: Self-supervised monocular depth estimation with a vision transformer. In 2022 Inter- national Conference on 3D Vision (3DV) , pages 668–678. IE...

    2022

  80. [88]

    Fisheyedepth: A real scale self-supervised depth estimation model for fisheye camera

    Guoyang Zhao, Yuxuan Liu, Weiqing Qi, Fulong Ma, Ming Liu, and Jun Ma. Fisheyedepth: A real scale self-supervised depth estimation model for fisheye camera. arXiv preprint arXiv:2409.15054, 2024. 3

    2024 arXiv

  81. [89]

    Unsupervised learning of depth and ego-motion from video

    Tinghui Zhou, Matthew Brown, Noah Snavely, and David G Lowe. Unsupervised learning of depth and ego-motion from video. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1851–1858, 2017. 3 Extending Foundational Monocular Depth Estimators t...

    2017

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.