Pith. sign in

REVIEW 4 major objections 4 minor 107 references

ZeroVO: Visual Odometry with Minimal Assumptions

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Zero-shot metric-scale camera motion, no calibration or fine-tuning required.

desk verdict ZeroVO is a well-engineered and honest extension of XVO with real architectural novelty, but the headline claim of zero-shot metric-scale generalization is softer than advertised once you look at rotation errors and the unvalidated reliance on off-the-shelf depth and intrinsics. read the letter →

arxiv 2506.08005 v1 pith:FCWWMSB4 submitted 2025-06-09 cs.CV

Lei Lai , Zekai Yin , Eshed Ohn-Bar This is my paper
classification cs.CV
keywords visualodometryzero-shotgeneralizationmetric-scaleposeestimationcalibration-freecameracross-attentiontransformermonoculardepthlanguagepriorssemi-supervisedlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ZeroVO claims that a two-frame network can predict metric-scale relative camera motion on unseen roads without any camera calibration and without fine-tuning. The paper argues that by fusing estimated depth, estimated camera intrinsics, optical flow, and language-derived scene descriptions through a cross-attention transformer, the network outperforms established baselines on KITTI, nuScenes, Argoverse 2, and a new GTA-based synthetic benchmark. If correct, the result matters because visual odometry could be deployed across heterogeneous camera fleets at scale, removing a per-domain recalibration bottleneck.

What carries the argument

The central mechanism is a transformer-based fusion module that uses cross-attention over four modalities: MaskFlownet optical flow, Metric3Dv2 metric depth, WildCamera estimated intrinsics encoded as an image-sized map, and language features from LLaVA-NeXT encoded by Sentence Transformers. The depth map is unprojected into a pseudo-3D point cloud and the 2D flow is lifted into scene flow using the estimated intrinsics, giving the network physically coherent but noisy geometric structure. A two-branch MLP decoder then regresses metric translation and a matrix Fisher distribution over rotation, while the language prior refines the depth and flow features used for this decoding.

What would settle it

Collect a test set of unseen driving sequences with ground-truth poses where the lens is deliberately dirty or covered with raindrops, and measure scale error and ATE against ground truth. If ZeroVO's advantage over the M+DS baseline shrinks or reverses on those sequences relative to clean-lens runs of the same routes, the claim that the language prior rescues noisy depth and intrinsics would not be supported.

Watch

Extended reading notes

Core claim

ZeroVO's central claim is that pseudo-3D geometry built from noisy single-image estimates, combined with language-conditioned cross-attention, yields a representation from which a two-frame regression head can recover metric-scale translation and rotation in unseen driving scenes. The model encodes estimated intrinsics into an image-sized map, refines flow and depth features with language-guided cross-attention, unprojects depth and flow into 3D scene flow, and decodes ego-motion with an MLP whose rotation head fits a matrix Fisher distribution. Semi-supervised training on unlabeled web video, with geometry- and language-guided pseudo-label filtering, further lifts zero-shot results. The paper reports over 30% improvement over prior methods and the lowest absolute trajectory error among the compared baselines, including a multi-frame SLAM pipeline (M+DS) that uses the same estimated depth.

Load-bearing premise

The load-bearing premise is that the off-the-shelf depth and intrinsic estimators remain accurate enough in the unseen test environments, because metric scale and geometry come from those estimates. The supplementary acknowledges persistent depth failures under lens artifacts and reflections, so this premise is already known to be violated in the very conditions the paper targets.

Editorial extensions

If this is right

  • Metric-scale visual odometry can be achieved from two frames without ground-truth intrinsics or fine-tuning, because scale is inherited from off-the-shelf metric depth estimates.
  • Language-based scene descriptions act as a regularizer that keeps pose estimation stable when depth and flow are unreliable, such as at night, in rain, or under strong light reflections.
  • Semi-supervised training on unlabeled web video with multimodal pseudo-label filtering can improve zero-shot generalization beyond supervised training alone.
  • Injecting noise into estimated intrinsics degrades but does not break ZeroVO, indicating partial tolerance to calibration error relative to baseline methods.
  • A lighter variant without the language-conditioning module still outperforms several baselines at 5 FPS, showing a concrete speed-accuracy tradeoff in the same architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the central claim holds, calibration-free VO could become a practical default for heterogeneous vehicle fleets, since per-camera intrinsics would no longer need to be known or stored ahead of time.
  • A testable extension is to run ZeroVO on non-driving footage, such as handheld or aerial video; the language prior is trained on driving captions, so its benefit may shrink or invert for scene types outside that distribution.
  • Because scale is inherited from monocular depth, improvements in metric depth estimation should transfer directly to VO accuracy, making the depth backbone an attractive target for future work.
  • The paper's own failure analysis suggests that lens artifacts and reflections are the main remaining bottleneck, so a dedicated evaluation with deliberately soiled lenses would likely expose where the approach stops generalizing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces ZeroVO, a two-frame monocular visual odometry network that predicts relative pose at metric scale without requiring known camera calibration. The method extracts optical flow, metric depth (Metric3Dv2), estimated camera intrinsics (WildCamera), and language features (LLaVA-NeXT plus Sentence Transformers), fuses them through cross-attention, unprojects depth and flow into pseudo-3D scene flow, and decodes translation and a probabilistic rotation via a matrix Fisher distribution. A semi-supervised variant, ZeroVO+, is trained on in-the-wild YouTube videos with pseudo-labels filtered by geometric warping consistency (normSSIM) and language subspace similarity. Experiments on KITTI, nuScenes, Argoverse 2, and a newly introduced GTA dataset report lower translation error and ATE than the compared baselines XVO, M+DS, TartanVO, and DPVO, alongside extensive ablations and qualitative trajectory plots.

Significance. If the claims are fully supported, ZeroVO would be a valuable step toward calibration-free, metric-scale, zero-shot visual odometry, and the proposed language-geometry fusion in a two-frame regressor is a timely and interesting architectural idea. The release of a challenging GTA dataset with lens corruption and adverse weather is a useful community resource. The paper also provides thorough ablations and comparisons, including a base model that is trained without privileged camera parameters. However, the headline claims of zero-shot generalization and 'over 30% improvement' currently outrun the evidence: the metric-scale output relies on unvalidated upstream depth and intrinsic estimators, the rotation results are worse than a baseline on two datasets, and all quantitative results are single-run with no variance estimates. These gaps are addressable but require additional experiments and qualified claims.

major comments (4)
  1. [§3.1, Supplementary §1.1, §2.1] The central claim of zero-shot metric-scale generalization depends on the accuracy of the frozen upstream estimators. In Sec. 3.1, the pseudo-3D unprojection and scene flow are constructed from the depth map D_hat and intrinsics K_hat estimated by Metric3Dv2 and WildCamera, so systematic errors in those estimates enter the translation prediction directly. The paper provides no quantitative evaluation of Metric3Dv2 or WildCamera on the test domains, and Supplementary Sec. 2.1 explicitly concedes 'persistent failures' in depth under lens and reflection artifacts and states that 'errors then propagate to the VO decoder both in baselines and ZeroVO.' Tables 11-15 illustrate these failures on the very night/rain/reflection conditions the paper targets. Consequently, the reported translation/ATE improvements on those splits could be inflated by the accuracy of the upstream models rather than by ZeroVO's fusion. I request a direct validation: report depth and intrinsic errors on the test splits, include ablations with ground-truth depth and intrinsics (or with oracle scale), and provide per-condition analysis of where the upstream estimates fail.
  2. [Table 1, §4.2] The text claims 'superior performance across nearly all metrics on the four datasets,' but Table 1 shows that ZeroVO+'s rotation error is worse than M+DS on nuScenes (4.37 vs. 1.46) and on Argoverse 2 (2.83 vs. 1.79). The abstract's 'over 30% improvement against prior methods' is also unqualified; it holds for translation and ATE against some baselines, but not for rotation. Please specify the metrics and baselines used for the headline number, and discuss why rotation is substantially worse on these datasets despite better translation.
  3. [Tables 1-3, Supplementary Tables 5-8] All quantitative results are reported as single runs with no standard deviations, confidence intervals, or multiple seeds. Given the stochastic nature of deep network training and the small differences between some variants (e.g., KITTI ATE 104.69 for ZeroVO+ vs. 105.07 for ZeroVO in Table 1), statistical significance is not established. The paper should report mean and standard deviation over at least three seeds, or otherwise justify why single runs are sufficient.
  4. [Supplementary §1.2, Eq. (9)] The pseudo-label selection thresholds are internally inconsistent. Supplementary Sec. 1.2 states that samples are filtered out if 'subspace-sim<5', but Eq. (9) defines subspace-sim as sin(arccos(trace(Λ)))^2, which lies in [0,1] for valid arccos arguments; a threshold of 5 is out of range. Additionally, the definition of Λ as the 'eigenvalues matrix obtained via Singular Value Decomposition over Q_i^T Q_{i+H}' is unclear, and trace(Λ) may fall outside the domain of arccos. Please clarify the actual computation and the threshold used, as this directly affects reproducibility of the semi-supervised training.
minor comments (4)
  1. [Table 12] The title of Table 12 says 'Failure Cases on GTA' but the caption text says the figure illustrates failure cases on Argoverse; this mismatch should be corrected.
  2. [Abstract] The abstract lists 'three standard benchmarks——KITTI, nuScenes, and Argoverse 2——as well as a newly introduced, high-fidelity synthetic dataset derived from Grand Theft Auto (GTA)'; this wording implies GTA is a standard benchmark, which it is not. Please rephrase to distinguish public benchmarks from the new synthetic dataset.
  3. [Table 3] Table 3 reports condition-wise ATE but does not state the number of sequences in each condition or the precise criteria for the 'Light' category beyond 'severe light reflections'; adding sample sizes and selection criteria would help interpretation.
  4. [Eq. (9)] The notation 'sin(arccos(trace(Λ))) 2' is mathematically ambiguous; it should be written as sin^2(arccos(trace(Λ))) with a precise definition of Λ and the principal-angle computation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: evaluation is anchored to external ground-truth benchmarks, and the semi-supervised self-training stage is a standard bootstrap rather than a definitional reduction.

full rationale

ZeroVO's central claims are validated on external benchmarks (KITTI, nuScenes, Argoverse 2, and a new GTA dataset) using ground-truth poses, and supervised training uses ground-truth poses from nuScenes. The predicted relative poses are therefore not defined in terms of the evaluation targets. The only self-referential component is the semi-supervised stage, where ZeroVO+ is trained on pseudo-labels generated by the model itself and filtered using hand-set SSIM and subspace-similarity thresholds. This is standard self-training and is explicitly treated by the paper as noisy supervision; the gains from this stage are measured on external test sets, so the result is not forced by construction. Self-citations to XVO establish the experimental protocol, YouTube data sourcing, and the scale-error metric, but the core architecture (depth, flow, language, intrinsics fusion) is implemented and evaluated independently, and no load-bearing argument reduces to a self-citation. The supplementary admission of persistent depth-estimation failures under lens and reflection artifacts is a real robustness concern about the external Metric3Dv2 and WildCamera estimators, but it is a correctness/assumption risk, not a circularity. No prediction in the paper is equivalent to its input by construction, and no fitted parameter is renamed as a prediction. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The method's zero-shot and metric-scale claims depend on a chain of pretrained models and hand-set training choices. The model does not learn depth or intrinsics from scratch; it consumes estimates from Metric3Dv2, WildCamera, MaskFlowNet, and LLaVA-NeXT. If any of these priors degrades in a new domain, the pseudo-3D unprojection and scale recovery degrade. The pseudo-label filtering thresholds are hand-selected rather than derived or validated across a sweep. There are no invented physical entities; the main contributions are a new architecture plus a GTA dataset.

free parameters (3)
  • normSSIM threshold = 0.5
    Hand-set threshold for geometry-based pseudo-label filtering; samples with normSSIM below 0.5 are excluded. No sensitivity analysis is reported.
  • subspace-sim threshold = 5
    Hand-set threshold for language-based pseudo-label diversity filtering; samples with subspace-sim below 5 are excluded. No sensitivity analysis is reported.
  • temporal window H = 10
    Hand-set window length used in the subspace-sim computation for language-guided selection; chosen without reported ablation.
assumptions (6)
  • domain assumption Metric3Dv2 provides metric-scale depth with acceptable accuracy in unseen driving domains.
    Depth is unprojected into pseudo-3D and provides metric scale in Sec. 3.1. The supplementary acknowledges persistent depth failures in challenging conditions.
  • domain assumption WildCamera estimates camera intrinsics accurately enough for unprojection and scale recovery.
    The calibration-free claim relies on WildCamera outputs; robustness is tested with injected noise but not with the full failure distribution.
  • domain assumption LLaVA-NeXT captions provide geometrically relevant semantic information.
    Language features condition attention and are used to filter pseudo-labels; the paper does not validate that captions capture scale or layout information.
  • domain assumption MaskFlowNet optical flow generalizes across the test datasets.
    Flow is a primary input for correspondence and scene flow; no cross-dataset validation of flow accuracy is provided.
  • domain assumption The YouTube web-video dataset used for semi-supervised training is sufficiently diverse and free of systematic bias.
    The model is trained on this unlabeled dataset with pseudo-labels; its composition and quality are inherited from prior work and not analyzed here.
  • domain assumption GTA V can provide ground-truth poses and images that are valid proxies for real-world driving conditions.
    The new GTA dataset is used for evaluation, but the manuscript does not describe how ground-truth poses were obtained or validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ZeroVO: Visual Odometry with Minimal Assumptions." pith.science (2026). https://pith.science/paper/FCWWMSB4

@misc{pith2026250608005,
  author       = {Pith},
  title        = {Pith review of: ZeroVO: Visual Odometry with Minimal Assumptions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FCWWMSB4}},
  note         = {Machine review of arXiv:2506.08005}
}
read the original abstract

We introduce ZeroVO, a novel visual odometry (VO) algorithm that achieves zero-shot generalization across diverse cameras and environments, overcoming limitations in existing methods that depend on predefined or static camera calibration setups. Our approach incorporates three main innovations. First, we design a calibration-free, geometry-aware network structure capable of handling noise in estimated depth and camera parameters. Second, we introduce a language-based prior that infuses semantic information to enhance robust feature extraction and generalization to previously unseen domains. Third, we develop a flexible, semi-supervised training paradigm that iteratively adapts to new scenes using unlabeled data, further boosting the models' ability to generalize across diverse real-world scenarios. We analyze complex autonomous driving contexts, demonstrating over 30% improvement against prior methods on three standard benchmarks, KITTI, nuScenes, and Argoverse 2, as well as a newly introduced, high-fidelity synthetic dataset derived from Grand Theft Auto (GTA). By not requiring fine-tuning or camera calibration, our work broadens the applicability of VO, providing a versatile solution for real-world deployment at scale.

Figures

Figures reproduced from arXiv: 2506.08005 by the authors.

Figure 1
Figure 1. Multimodal and Geometry-Guided Network Overview. Given a pair of input images, our model computes a rich multimodal embedding through a transformer-based fusion module. The embedding is then passed to a two-branch decoder MLP that outputs real￾world translation and rotation. Our architecture (Sec. 3.1) leverages cross-attention to fuse complementary cues, including flow, depth, camera intrinsics, and language-based … view at source ↗
Figure 2
Figure 2. Qualitative Results on KITTI. We show trajectory prediction results across the four most complex driving sequences (00, 02, 05, and 08) from the KITTI dataset. Each subplot illustrates the trajectories generated by our proposed model and the baseline models alongside the ground truth trajectory. The qualitative results demonstrate that our approach achieves the highest alignment with the ground truth, particularly i… view at source ↗
Figure 3
Figure 3. Example from Our GTA Dataset. We leverage GTA to construct a diverse and challenging generalization dataset for validation. to the initial frame, i.e., expressed in the camera coordinate system of the first frame: Gi→1 = Qi j=2 Tj for ∀i > 1, with G1→1 defined as the 4 × 4 identity matrix. Once the trajectory is recovered, evaluation metrics can be computed as follows: Translation and Rotation Errors: We follow stan… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

107 extracted references · 67 canonical work pages

  1. [1]

    Visual odometry in smoke occluded environments

    Aditya Agarwal, Daniel Maturana, and Sebastian Scherer. Visual odometry in smoke occluded environments. Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, Tech. Rep. CMU-RI-TR-15-07, 2014. 1

    2014

  2. [2]

    Vqa: Visual question answering

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. InCVPR, 2015. 2

    2015

  3. [3]

    Recursive estima- tion of motion, structure, and focal length.PAMI, 1995

    Ali Azarbayejani and Alex P Pentland. Recursive estima- tion of motion, structure, and focal length.PAMI, 1995. 1

    1995

  4. [4]

    Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring.arXiv preprint arXiv:1911.09785, 2019

    David Berthelot, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, and Colin Raffel. Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring.arXiv preprint arXiv:1911.09785, 2019. 2

    arXiv 1911

  5. [5]

    Mixmatch: A holistic approach to semi-supervised learning.NeurIPS,

    David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning.NeurIPS,

  6. [6]

    Unsuper- vised scale-consistent depth and ego-motion learning from monocular video

    Jiawang Bian, Zhichao Li, Naiyan Wang, Huangying Zhan, Chunhua Shen, Ming-Ming Cheng, and Ian Reid. Unsuper- vised scale-consistent depth and ego-motion learning from monocular video. InNeurIPS, 2019. 14

    2019

  7. [7]

    On the mathematical properties of the structural similarity in- dex.T-IP, 2011

    Dominique Brunet, Edward R Vrscay, and Zhou Wang. On the mathematical properties of the structural similarity in- dex.T-IP, 2011. 5

    2011

  8. [8]

    The euroc micro aerial vehicle datasets.IJRR, 2016

    Michael Burri, Janosch Nikolic, Pascal Gohl, Thomas Schneider, Joern Rehder, Sammy Omari, Markus W Achte- lik, and Roland Siegwart. The euroc micro aerial vehicle datasets.IJRR, 2016. 14

    2016

Show all 107 references
  1. [9]

    Past, present, and future of simultaneous localiza- tion and mapping: Toward the robust-perception age.T-RO,

    Cesar Cadena, Luca Carlone, Henry Carrillo, Yasir Latif, Davide Scaramuzza, Jos ´e Neira, Ian Reid, and John J Leonard. Past, present, and future of simultaneous localiza- tion and mapping: Toward the robust-perception age.T-RO,

  2. [10]

    nuscenes: A multi- modal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. InCVPR, 2020. 6, 13, 14

    2020

  3. [11]

    Pseudo-labeling for scalable 3d object detection

    Benjamin Caine, Rebecca Roelofs, Vijay Vasudevan, Ji- quan Ngiam, Yuning Chai, Zhifeng Chen, and Jonathon Shlens. Pseudo-labeling for scalable 3d object detection. InarXiv preprint arXiv:2103.02093, 2021. 2, 5

    2021 arXiv

  4. [12]

    Orb-slam3: An ac- curate open-source library for visual, visual–inertial, and multimap slam.T-RO, 2021

    Carlos Campos, Richard Elvira, Juan J G ´omez Rodr´ıguez, Jos´e MM Montiel, and Juan D Tard ´os. Orb-slam3: An ac- curate open-source library for visual, visual–inertial, and multimap slam.T-RO, 2021. 1

    2021

  5. [13]

    Rgb cameras fail- ures and their effects in autonomous driving applications

    Andrea Ceccarelli and Francesco Secci. Rgb cameras fail- ures and their effects in autonomous driving applications. T-DSC, 2022. 1

    2022

  6. [14]

    Llava-interactive: An all-in-one demo for image chat, segmentation, generation and editing

    Wei-Ge Chen, Irina Spiridonova, Jianwei Yang, Jianfeng Gao, and Chunyuan Li. Llava-interactive: An all-in-one demo for image chat, segmentation, generation and editing. arXiv preprint arXiv:2311.00571, 2023. 2

    2023 arXiv

  7. [15]

    Structure from motion causally integrated over time.PAMI, 2002

    Alessandro Chiuso, Paolo Favaro, Hailin Jin, and Stefano Soatto. Structure from motion causally integrated over time.PAMI, 2002. 1

    2002

  8. [16]

    Semi-supervised sequence modeling with cross-view training.arXiv preprint arXiv:1809.08370,

    Kevin Clark, Minh-Thang Luong, Christopher D Man- ning, and Quoc V Le. Semi-supervised sequence modeling with cross-view training.arXiv preprint arXiv:1809.08370,

    arXiv

  9. [17]

    Contrastive learning for image cap- tioning.NeurIPS, 30, 2017

    Bo Dai and Dahua Lin. Contrastive learning for image cap- tioning.NeurIPS, 30, 2017. 2

    2017

  10. [18]

    Springer, 2007

    Ernst Dieter Dickmanns.Dynamic vision for perception and control of motion. Springer, 2007. 1

    2007

  11. [19]

    Carla: An open urban driv- ing simulator

    Alexey Dosovitskiy, German Ros, Felipe Codevilla, Anto- nio Lopez, and Vladlen Koltun. Carla: An open urban driv- ing simulator. InCoRL, 2017. 6

    2017

  12. [20]

    An im- age is worth 16x16 words: Transformers for image recog- nition at scale.ICLR, 2021

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An im- age is worth 16x16 words: Transformers for image recog- nitio...

    2021

  13. [21]

    Direct sparse odometry

    Jakob Engel, Vladlen Koltun, and Daniel Cremers. Direct sparse odometry. InPAMI, 2017. 1

    2017

  14. [22]

    Visual odometry: Part i: The first 30 years and fundamentals

    Friedrich Fraundorfer and Davide Scaramuzza. Visual odometry: Part i: The first 30 years and fundamentals. RAM, 2011. 1

    2011

  15. [23]

    Note-rcnn: Noise tolerant ensemble rcnn for semi-supervised object detection

    Jiyang Gao, Jiang Wang, Shengyang Dai, Li-Jia Li, and Ram Nevatia. Note-rcnn: Noise tolerant ensemble rcnn for semi-supervised object detection. InCVPR, 2019. 2

    2019

  16. [24]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. InCVPR, 2012. 6, 14

    2012

  17. [25]

    Learning optical flow, depth, and scene flow with- out real-world labels.RA-L, 2022

    Vitor Guizilini, Kuan-Hui Lee, Rares ¸ Ambrus ¸, and Adrien Gaidon. Learning optical flow, depth, and scene flow with- out real-world labels.RA-L, 2022. 13, 14

    2022

  18. [26]

    Towards zero-shot scale-aware monoc- ular depth estimation

    Vitor Guizilini, Igor Vasiljevic, Dian Chen, Rares, Ambrus,, and Adrien Gaidon. Towards zero-shot scale-aware monoc- ular depth estimation. InCVPR, 2023. 1

    2023

  19. [27]

    Variational pretraining for semi-supervised text classification.arXiv preprint arXiv:1906.02242, 2019

    Suchin Gururangan, Tam Dang, Dallas Card, and Noah A Smith. Variational pretraining for semi-supervised text classification.arXiv preprint arXiv:1906.02242, 2019. 2

    1906 arXiv

  20. [28]

    Don’t stop pretraining: Adapt language models to domains and tasks.arXiv preprint arXiv:2004.10964, 2020

    Suchin Gururangan, Ana Marasovi ´c, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. Don’t stop pretraining: Adapt language models to domains and tasks.arXiv preprint arXiv:2004.10964, 2020. 2

    2004 arXiv

  21. [29]

    Deep visual mpc-policy learning for navigation.RA-L, 4(4), 2019

    Noriaki Hirose, Fei Xia, Roberto Mart ´ın-Mart´ın, Amir Sadeghian, and Silvio Savarese. Deep visual mpc-policy learning for navigation.RA-L, 4(4), 2019. 2

    2019

  22. [30]

    Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation.PAMI, 2024

    Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, Hao Chen, Kaixuan Wang, Gang Yu, Chunhua Shen, and Shaojie Shen. Metric3d v2: A versatile monocular geomet- ric foundation model for zero-shot metric depth and surface normal estimation.PAMI, 2024. 1, 2, 4, 5, 6, 7, 8, 13, 16, 17

    2024

  23. [31]

    Generalized product quantization network for semi-supervised image retrieval

    Young Kyun Jang and Nam Ik Cho. Generalized product quantization network for semi-supervised image retrieval. InCVPR, 2020. 2

    2020

  24. [32]

    Consistency-based semi-supervised learning for object de- tection

    Jisoo Jeong, Seungeui Lee, Jeesoo Kim, and Nojun Kwak. Consistency-based semi-supervised learning for object de- tection. InNeurIPS, 2019. 5

    2019

  25. [33]

    Self-supervised geometry-guided ini- tialization for robust monocular visual odometry.arXiv preprint arXiv:2406.00929, 2024

    Takayuki Kanai, Igor Vasiljevic, Vitor Guizilini, and Kazuhiro Shintani. Self-supervised geometry-guided ini- tialization for robust monocular visual odometry.arXiv preprint arXiv:2406.00929, 2024. 2, 14

    2024

  26. [34]

    Mdn-vo: Estimating visual odometry with confidence

    Nimet Kaygusuz, Oscar Mendez, and Richard Bowden. Mdn-vo: Estimating visual odometry with confidence. In IROS, 2021. 1

    2021

  27. [35]

    Geometric loss func- tions for camera pose regression with deep learning

    Alex Kendall and Roberto Cipolla. Geometric loss func- tions for camera pose regression with deep learning. In CVPR, 2017. 1, 14

    2017

  28. [36]

    Posenet: A convolutional network for real-time 6-dof cam- era relocalization

    Alex Kendall, Matthew Grimes, and Roberto Cipolla. Posenet: A convolutional network for real-time 6-dof cam- era relocalization. InICCV, 2015. 1

    2015

  29. [37]

    Motion diversification networks

    Hee Jae Kim and Eshed Ohn-Bar. Motion diversification networks. InCVPR, 2024. 2

    2024

  30. [38]

    Robust visual odometry to irregular illumination changes with rgb-d cam- era

    Pyojin Kim, Hyon Lim, and H Jin Kim. Robust visual odometry to irregular illumination changes with rgb-d cam- era. InIROS, 2015. 1

    2015

  31. [39]

    Principal an- gles between subspaces in an a-based scalar product: algo- rithms and perturbation estimates.SIAM Journal on Scien- tific Computing, 2002

    Andrew V Knyazev and Merico E Argentati. Principal an- gles between subspaces in an a-based scalar product: algo- rithms and perturbation estimates.SIAM Journal on Scien- tific Computing, 2002. 5

    2002

  32. [40]

    XVO: Generalized visual odometry via cross- modal self-training

    Lei Lai, Zhongkai Shangguan, Jimuyang Zhang, and Eshed Ohn-Bar. XVO: Generalized visual odometry via cross- modal self-training. InICCV, 2023. 1, 2, 4, 5, 6, 7, 8, 14, 15, 16

    2023

  33. [41]

    Uncertainty-guided never-ending learning to drive

    Lei Lai, Eshed Ohn-Bar, Sanjay Arora, and John Seon Keun Yi. Uncertainty-guided never-ending learning to drive. In CVPR, 2024. 2

    2024

  34. [42]

    Pseudo-label: The simple and effi- cient semi-supervised learning method for deep neural net- works

    Dong-Hyun Lee et al. Pseudo-label: The simple and effi- cient semi-supervised learning method for deep neural net- works. InICMLW, 2013. 5, 13

    2013

  35. [43]

    Label efficient semi-supervised learning via graph filtering

    Qimai Li, Xiao-Ming Wu, Han Liu, Xiaotong Zhang, and Zhichao Guan. Label efficient semi-supervised learning via graph filtering. InCVPR, 2019. 5

    2019

  36. [44]

    Undeepvo: Monocular visual odometry through unsuper- vised deep learning

    Ruihao Li, Sen Wang, Zhiqiang Long, and Dongbing Gu. Undeepvo: Monocular visual odometry through unsuper- vised deep learning. InICRA, 2018. 5, 14, 15

    2018

  37. [45]

    Self-supervised deep visual odometry with online adaptation

    Shunkai Li, Xin Wang, Yingdian Cao, Fei Xue, Zike Yan, and Hongbin Zha. Self-supervised deep visual odometry with online adaptation. InCVPR, 2020. 14

    2020

  38. [46]

    Deep patch visual slam

    Lahav Lipson, Zachary Teed, and Jia Deng. Deep patch visual slam. InECCV, 2024. 1, 2

    2024

  39. [47]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In CVPR, 2024. 1, 2

    2024

  40. [48]

    Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Im- proved reasoning, ocr, and world knowledge, 2024. 4, 13

    2024

  41. [49]

    Visual instruction tuning.NeurIPS, 2024

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.NeurIPS, 2024. 1, 2

    2024

  42. [50]

    Llava-plus: Learning to use tools for creating multi- modal agents.arXiv preprint arXiv:2311.05437, 2023

    Shilong Liu, Hao Cheng, Haotian Liu, Hao Zhang, Feng Li, Tianhe Ren, Xueyan Zou, Jianwei Yang, Hang Su, Jun Zhu, et al. Llava-plus: Learning to use tools for creating multi- modal agents.arXiv preprint arXiv:2311.05437, 2023. 2

    2023 arXiv

  43. [51]

    Un- supervised learning of depth and ego-motion from monoc- ular video using 3d geometric constraints

    Reza Mahjourian, Martin Wicke, and Anelia Angelova. Un- supervised learning of depth and ego-motion from monoc- ular video using 3d geometric constraints. InCVPR, 2018. 5

    2018

  44. [52]

    Kanti V Mardia, Peter E Jupp, and KV Mardia.Directional statistics. 2000. 4

    2000

  45. [53]

    Reinforcement learning meets visual odometry.ECCV, 2024

    Nico Messikommer, Giovanni Cioffi, Mathias Gehrig, and Davide Scaramuzza. Reinforcement learning meets visual odometry.ECCV, 2024. 2

    2024

  46. [54]

    Ecker, Matthias Bethge, and Wieland Brendel

    Claudio Michaelis, Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexander S. Ecker, Matthias Bethge, and Wieland Brendel. Benchmarking ro- bustness in object detection: Autonomous driving when winter is coming.arXiv preprint arXiv:1907.07484, 2019. 14, 17

    1907 arXiv

  47. [55]

    Probabilistic orientation estimation with matrix fisher dis- tributions

    David Mohlin, Josephine Sullivan, and G ´erald Bianchi. Probabilistic orientation estimation with matrix fisher dis- tributions. InNeurIPS, 2020. 4

    2020

  48. [56]

    Orb-slam2: An open- source slam system for monocular, stereo, and rgb-d cam- eras.T-RO, 2017

    Raul Mur-Artal and Juan D Tard ´os. Orb-slam2: An open- source slam system for monocular, stereo, and rgb-d cam- eras.T-RO, 2017. 1, 3

    2017

  49. [57]

    Orb-slam: a versatile and accurate monocular slam system.T-RO, 2015

    Raul Mur-Artal, Jose Maria Martinez Montiel, and Juan D Tardos. Orb-slam: a versatile and accurate monocular slam system.T-RO, 2015. 1

    2015

  50. [58]

    Unidepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mat- tia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation. In CVPR, 2024. 1, 2

    2024

  51. [59]

    A visual odometry framework robust to motion blur

    Alberto Pretto, Emanuele Menegatti, Maren Bennewitz, Wolfram Burgard, and Enrico Pagello. A visual odometry framework robust to motion blur. InICRA, 2009. 1

    2009

  52. [60]

    Sentence-bert: Sentence embeddings using siamese bert-networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. InEMNLP. 1, 4, 5, 13

  53. [61]

    In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning

    Mamshad Nayeem Rizve, Kevin Duarte, Yogesh S Rawat, and Mubarak Shah. In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning. InICLR, 2021. 5

    2021

  54. [62]

    The 8-point algorithm as an inductive bias for relative pose pre- diction by vits

    Chris Rockwell, Justin Johnson, and David F Fouhey. The 8-point algorithm as an inductive bias for relative pose pre- diction by vits. In3DV. IEEE, 2022. 1

    2022

  55. [63]

    The tum vi benchmark for evaluating visual-inertial odometry

    David Schubert, Thore Goll, Nikolaus Demmel, Vladyslav Usenko, J ¨org St ¨uckler, and Daniel Cremers. The tum vi benchmark for evaluating visual-inertial odometry. In IROS, 2018. 14

    2018

  56. [64]

    Re- thinking semi-supervised learning with language models

    Zhengxiang Shi, Francesco Tonolini, Nikolaos Aletras, Emine Yilmaz, Gabriella Kazai, and Yunlong Jiao. Re- thinking semi-supervised learning with language models. arXiv preprint arXiv:2305.13002, 2023. 2

    2023 arXiv

  57. [65]

    Semi supervised semantic segmentation using generative adversarial network

    Nasim Souly, Concetto Spampinato, and Mubarak Shah. Semi supervised semantic segmentation using generative adversarial network. InICCV, 2017. 5

    2017

  58. [66]

    BA-net: Dense bundle ad- justment network.arXiv preprint arXiv:1806.04807, 2018

    Chengzhou Tang and Ping Tan. BA-net: Dense bundle ad- justment network.arXiv preprint arXiv:1806.04807, 2018. 3

    2018 arXiv

  59. [67]

    Humble teachers teach better students for semi-supervised object detection

    Yihe Tang, Weifeng Chen, Yijun Luo, and Yuting Zhang. Humble teachers teach better students for semi-supervised object detection. InCVPR, 2021. 2

    2021

  60. [68]

    DROID-SLAM: Deep visual slam for monocular, stereo, and rgb-d cameras.NeurIPS,

    Zachary Teed and Jia Deng. DROID-SLAM: Deep visual slam for monocular, stereo, and rgb-d cameras.NeurIPS,

  61. [69]

    Deep patch visual odometry.NeurIPS, 2023

    Zachary Teed, Lahav Lipson, and Jia Deng. Deep patch visual odometry.NeurIPS, 2023. 1, 2, 3, 6, 7, 17

    2023

  62. [70]

    Tokenize the world into object-level knowledge to address long-tail events in autonomous driving.arXiv preprint arXiv:2407.00959, 2024

    Ran Tian, Boyi Li, Xinshuo Weng, Yuxiao Chen, Edward Schmerling, Yue Wang, Boris Ivanovic, and Marco Pavone. Tokenize the world into object-level knowledge to address long-tail events in autonomous driving.arXiv preprint arXiv:2407.00959, 2024. 2

    2024 arXiv

  63. [71]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. InNeurIPS,

  64. [72]

    Learning of structure and motion from video

    Sudheendra Vijayanarasimhan, Susanna Ricco, Cordelia Schmid, Rahul Sukthankar, and Katerina Fragkiadaki. Learning of structure and motion from video. InCVPR,

  65. [73]

    3DIoUMatch: Leveraging IoU pre- diction for semi-supervised 3D object detection

    He Wang, Yezhen Cong, Or Litany, Yue Gao, and Leonidas J Guibas. 3DIoUMatch: Leveraging IoU pre- diction for semi-supervised 3D object detection. InCVPR,

  66. [74]

    Deepvo: Towards end-to-end visual odometry with deep re- current convolutional neural networks

    Sen Wang, Ronald Clark, Hongkai Wen, and Niki Trigoni. Deepvo: Towards end-to-end visual odometry with deep re- current convolutional neural networks. InICRA, 2017. 1, 15

    2017

  67. [75]

    Tartanair: A dataset to push the limits of visual slam

    Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. Tartanair: A dataset to push the limits of visual slam. InIROS, 2020. 1, 14

    2020

  68. [76]

    Tar- tanvo: A generalizable learning-based vo

    Wenshan Wang, Yaoyu Hu, and Sebastian Scherer. Tar- tanvo: A generalizable learning-based vo. InCoRL, 2021. 1, 2, 3, 4, 6, 7, 14, 15, 16, 17

    2021

  69. [77]

    Pseudo- lidar from visual depth estimation: Bridging the gap in 3d object detection for autonomous driving

    Yan Wang, Wei-Lun Chao, Divyansh Garg, Bharath Hari- haran, Mark Campbell, and Kilian Q Weinberger. Pseudo- lidar from visual depth estimation: Bridging the gap in 3d object detection for autonomous driving. InCVPR, 2019. 4, 13

    2019

  70. [78]

    SEA-RAFT: Simple, efficient, accurate raft for optical flow

    Yihan Wang, Lahav Lipson, and Jia Deng. SEA-RAFT: Simple, efficient, accurate raft for optical flow. InECCV,

  71. [79]

    Ar- goverse 2: Next generation datasets for self-driving percep- tion and forecasting, 2023

    Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, Deva Ramanan, Peter Carr, and James Hays. Ar- goverse 2: Next generation datasets for self-driving percep- tion an...

    2023

  72. [80]

    Beyond tracking: Selecting mem- ory and refining poses for deep visual odometry

    Fei Xue, Xin Wang, Shunkai Li, Qiuyuan Wang, Junqiu Wang, and Hongbin Zha. Beyond tracking: Selecting mem- ory and refining poses for deep visual odometry. InCVPR,

  73. [81]

    Billion-scale semi-supervised learning for image classification.arXiv preprint arXiv:1905.00546,

    I Zeki Yalniz, Herv ´e J ´egou, Kan Chen, Manohar Paluri, and Dhruv Mahajan. Billion-scale semi-supervised learning for image classification.arXiv preprint arXiv:1905.00546,

    1905 arXiv

  74. [82]

    Unified contrastive learning in image-text-label space

    Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Bin Xiao, Ce Liu, Lu Yuan, and Jianfeng Gao. Unified contrastive learning in image-text-label space. InCVPR, 2022. 2

    2022

  75. [83]

    Generalized predictive model for autonomous driving

    Jiazhi Yang, Shenyuan Gao, Yihang Qiu, Li Chen, Tianyu Li, Bo Dai, Kashyap Chitta, Penghao Wu, Jia Zeng, Ping Luo, et al. Generalized predictive model for autonomous driving. InCVPR, 2024. 5, 14

    2024

  76. [84]

    Depth anything: Un- leashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Ji- ashi Feng, and Hengshuang Zhao. Depth anything: Un- leashing the power of large-scale unlabeled data. InCVPR,

  77. [85]

    D3vo: Deep depth, deep pose and deep uncer- tainty for monocular visual odometry

    Nan Yang, Lukas von Stumberg, Rui Wang, and Daniel Cremers. D3vo: Deep depth, deep pose and deep uncer- tainty for monocular visual odometry. InCVPR, 2020. 1

    2020

  78. [86]

    A survey on deep semi-supervised learning.arXiv preprint arXiv:2103.00550, 2021

    Xiangli Yang, Zixing Song, Irwin King, and Zenglin Xu. A survey on deep semi-supervised learning.arXiv preprint arXiv:2103.00550, 2021. 5

    2021 arXiv

  79. [87]

    Pvo: Panoptic visual odometry

    Weicai Ye, Xinyue Lan, Shuo Chen, Yuhang Ming, Xingyuan Yu, Hujun Bao, Zhaopeng Cui, and Guofeng Zhang. Pvo: Panoptic visual odometry. InCVPR, 2023. 2

    2023

  80. [88]

    Met- ric3d: Towards zero-shot metric 3d prediction from a single image

    Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Met- ric3d: Towards zero-shot metric 3d prediction from a single image. InCVPR, 2023. 1

    2023

  81. [89]

    Geonet: Unsupervised learn- ing of dense depth, optical flow and camera pose

    Zhichao Yin and Jianping Shi. Geonet: Unsupervised learn- ing of dense depth, optical flow and camera pose. InCVPR,

  82. [90]

    Image captioning with semantic attention

    Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo. Image captioning with semantic attention. In CVPR, 2016. 2

    2016

  83. [91]

    Wordepth: Variational language prior for monocular depth estimation

    Ziyao Zeng, Daniel Wang, Fengyu Yang, Hyoungseob Park, Stefano Soatto, Dong Lao, and Alex Wong. Wordepth: Variational language prior for monocular depth estimation. InCVPR, 2024. 2

    2024

  84. [92]

    Unsupervised learn- ing of monocular depth estimation and visual odometry with deep feature reconstruction

    Huangying Zhan, Ravi Garg, Chamara Saroj Weerasekera, Kejie Li, Harsh Agarwal, and Ian Reid. Unsupervised learn- ing of monocular depth estimation and visual odometry with deep feature reconstruction. InCVPR, 2018. 5

    2018

  85. [93]

    Df-vo: What should be learnt for visual odometry?arXiv preprint arXiv:2103.00933, 2021

    Huangying Zhan, Chamara Saroj Weerasekera, Jia-Wang Bian, Ravi Garg, and Ian Reid. Df-vo: What should be learnt for visual odometry?arXiv preprint arXiv:2103.00933, 2021. 1

    2021 arXiv

  86. [94]

    Camera calibration from images of spheres.PAMI, 2007

    Hui Zhang, K Wong Kwan-yee, and Guoqiang Zhang. Camera calibration from images of spheres.PAMI, 2007. 2

    2007

  87. [95]

    Selfd: Self-learning large-scale driving policies from the web

    Jimuyang Zhang, Ruizhao Zhu, and Eshed Ohn-Bar. Selfd: Self-learning large-scale driving policies from the web. In CVPR, 2022. 2

    2022

  88. [96]

    Feedback-guided autonomous driving

    Jimuyang Zhang, Zanming Huang, Arijit Ray, and Eshed Ohn-Bar. Feedback-guided autonomous driving. InCVPR,

  89. [97]

    A flexible online camera calibration using line seg- ments.Journal of Sensors, 2016

    Yueqiang Zhang, Langming Zhou, Haibo Liu, and Yang Shang. A flexible online camera calibration using line seg- ments.Journal of Sensors, 2016. 2

    2016

  90. [98]

    Z. Zhang. A flexible new technique for camera calibration. PAMI, 2000

    2000

  91. [99]

    Camera calibration with one- dimensional objects.PAMI, 2004

    Zhengyou Zhang. Camera calibration with one- dimensional objects.PAMI, 2004. 2

    2004

  92. [100]

    Maskflownet: Asymmetric feature matching with learnable occlusion mask

    Shengyu Zhao, Yilun Sheng, Yue Dong, Eric I Chang, Yan Xu, et al. Maskflownet: Asymmetric feature matching with learnable occlusion mask. InCVPR, 2020. 4, 5, 13

    2020

  93. [101]

    Unsupervised learning of depth and ego- motion from video

    Tinghui Zhou, Matthew Brown, Noah Snavely, and David G Lowe. Unsupervised learning of depth and ego- motion from video. InCVPR, 2017. 5, 14, 15

    2017

  94. [102]

    Learning to drive anywhere

    Ruizhao Zhu, Peng Huang, Eshed Ohn-Bar, and Venkatesh Saligrama. Learning to drive anywhere. InCoRL, 2023. 2

    2023

  95. [103]

    Tame a wild camera: in-the-wild monocular camera calibration.NeurIPS, 2024

    Shengjie Zhu, Abhinav Kumar, Masa Hu, and Xiaoming Liu. Tame a wild camera: in-the-wild monocular camera calibration.NeurIPS, 2024. 2, 4, 5, 13

    2024

  96. [104]

    Nicer-slam: Neural implicit scene encoding for rgb slam

    Zihan Zhu, Songyou Peng, Viktor Larsson, Zhaopeng Cui, Martin R Oswald, Andreas Geiger, and Marc Pollefeys. Nicer-slam: Neural implicit scene encoding for rgb slam. In3DV, 2024. 2 Supplementary Material for ZeroVO: Visual Odometry with Minimal Assumptions Lei Lai∗ Zekai Yin∗ E...

    2024

  97. [106]

    Network Architecture Image Encoders:To ensurebroad generalizationof our findings, in our work we focus on thepure two-frame VO task

    Implementation Details 1.1. Network Architecture Image Encoders:To ensurebroad generalizationof our findings, in our work we focus on thepure two-frame VO task. Thus, our findings can be potentially applicable to diverse autonomous driving scenarios, e.g., to scale over scenar...

    1920

  98. [107]

    Additional Ablations Pseud-Label Selection:Table5shows an ablation over the proposed semi-supervised training framework

    Detailed Analysis 2.1. Additional Ablations Pseud-Label Selection:Table5shows an ablation over the proposed semi-supervised training framework. We find both the language and geometry consistency-based mechanisms to complement, resulting in best overall performance. This is int...

  99. [2021]

    1, 2, 3, 6, 14, 16, 17

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.