Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

Rig3R: Rig-Aware Conditioning for Learned 3D Reconstruction

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Rig3R introduces a single-pass transformer that conditions on optional multi-camera rig metadata and learns to infer rig structure from unordered images, achieving leading results in reconstruction, pose estimation, and rig discovery.

desk verdict A solid rig-aware extension of DUSt3R whose most novel claim—rig discovery on unseen rigs—is not yet backed by a clean held-out experiment. read the letter →

arxiv 2506.02265 v1 pith:4FEDSBBY submitted 2025-06-02 cs.CV

Samuel Li , Pujith Kachana , Prajwal Chidananda , Saurabh Nair , Yasutaka Furukawa , Matthew Brown This is my paper
classification cs.CV
keywords rig-aware3Dreconstructionmulti-camerarigraymaprepresentationcameraposeestimationcalibrationdiscoverytransformerautonomousdrivingpointmapregression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Rig3R sets out to show that multi-camera rig structure is a usable prior for learned 3D reconstruction, not a preprocessing step. It conditions a transformer on optional camera ID, timestamp, and rig pose, and adds a rig-relative raymap prediction head so the model can also infer the rig from images alone. On real-world driving rigs the method reports leading pose accuracy, improving over classical and learned baselines by 17 to 45 mAA, with reconstruction and rig discovery also ahead, all in a single forward pass. The contribution is a unified representation: pointmaps, global pose raymaps, and rig raymaps trained jointly so geometry, ego-motion, and fixed rig structure reinforce each other.

What carries the argument

The load-bearing mechanism is the raymap, a dense field of unit ray directions anchored at a shared camera center, produced by two heads: a pose raymap in the global frame and a rig raymap in a rig-centric frame. Rig raymaps decouple fixed camera geometry from ego-motion, so clustering them across frames reveals camera identity and rig calibration. On the input side, optional metadata embeddings, namely camera ID, timestamp, and rig raymap patches, are added to image patch tokens and dropped independently with 50 percent probability during training, forcing the model to use rig cues when present and to infer them when absent. The map from raymaps to camera parameters is closed form: focal lengths come from pixel-ray angular constraints and rotations from SVD alignment of camera-frame and global-frame rays, which is why pose estimates stay stable in regions where pointmap depth is unreliable.

What would settle it

Run Rig3RUnstr on a sequence from a rig geometry absent from the training distribution, such as a six-camera asymmetric rig with unusual relative rotations; if rig-ID clustering accuracy and rig-relative pose mAA stay near chance levels while the calibrated variant remains accurate, the rig-discovery and generalization claims are refuted.

Watch

Extended reading notes

Core claim

Rig3R is a single-pass transformer that predicts, for every input image, a dense pointmap, a global pose raymap in the first frame's coordinate system, and a rig raymap in a rig-centric frame that is constant across time. The rig raymap is the paper's key device: because it encodes each camera's fixed position and orientation within the rig, the model can condition on that structure when calibration metadata is provided and can recover it from image content when it is not. Camera ID, timestamp, and rig pose are injected as optional embeddings with 50 percent dropout during training, so the same weights handle unordered images, partially calibrated rigs, and fully calibrated rigs. The paper's claim is that this rig-aware latent space is what allows one forward pass, with no bundle adjustment or iterative refinement, to outperform both classical rig-aware COLMAP and learned feedforward baselines on real driving data, and to discover rig calibration from unordered image sets.

Load-bearing premise

The claim that Rig3R generalizes to unseen rig geometries and discovers rig structure rests on the training data containing enough variety in rig configurations, which the paper itself identifies as its main limitation.

Editorial extensions

If this is right

  • Autonomous driving pipelines could obtain dense scene geometry and camera poses in a single transformer pass, removing the need for explicit feature matching and bundle adjustment on rig data.
  • Rig calibration could be discovered from unordered, timestamp-free image sets, a task the paper says no classical or learned method previously addressed.
  • Pose estimation from raymaps should remain accurate in sky, reflective, and dynamic regions where pointmap-based pose estimation degrades sharply.
  • Partial or noisy metadata is usable: models trained with dropped or corrupted rig embeddings outperform unstructured baselines and can be finetuned to hold up under calibration noise.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Going beyond the paper, the rig-raymap representation could generalize to non-driving camera networks such as drones or distributed fixed cameras, where calibration is unknown but relative rig geometry is approximately constant.
  • A natural test of the stated data-diversity limitation is to train with procedurally augmented continuous rig configurations; if generalization is data-limited, rig-discovery accuracy on novel geometries should rise accordingly.
  • Rig3R's clean separation of ego-motion and rig geometry suggests combining it with dynamic-scene models, which currently build pose priors from pointmaps and struggle in moving regions.
  • Because rig discovery works from unordered images, it could serve as an unsupervised self-calibration signal for camera networks before any metric reconstruction is attempted.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. Rig3R is a single-pass transformer model for multi-view 3D reconstruction and camera pose estimation from multi-camera rigs. In addition to predicting dense pointmaps, it predicts two types of raymaps: a global pose raymap and a rig-relative raymap. The model optionally conditions on camera ID, timestamp, and rig-relative calibration, with dropout during training to tolerate missing metadata; when metadata is absent, the rig raymap head is used to discover rig structure directly from images. The paper reports state-of-the-art pose and pointmap accuracy on Waymo and WayveScenes101, ablations of metadata and auxiliary heads, robustness to noisy calibration, and rig-discovery results on Argoverse2 across 1-, 3-, 5-, and 7-camera configurations. The central claims are that rig-aware conditioning improves reconstruction and pose estimation, and that the model can discover rig calibration from unordered images on previously unseen rig configurations.

Significance. If the findings hold, Rig3R would be a meaningful step beyond unstructured multi-view models: it explicitly exploits rig geometry when available, and it introduces a rig-relative raymap representation with closed-form pose and intrinsic recovery. The experimental package is substantial: real-world driving datasets, multiple baselines including DUSt3R-GA and Fast3R, clear ablations of each metadata component, and a useful Appendix C showing that raymap-based pose estimation dramatically outperforms pointmap-based PnP (Table 5). The closed-form solver in Appendix B and the use of external LiDAR/COLMAP ground truth reduce circularity concerns. However, the most distinctive claim—generalization of rig discovery to unseen rig configurations—is not yet supported by a clean held-out evaluation, and this is the main reason the contribution cannot be accepted as stated.

major comments (3)
  1. [Section 4.3 and Figure 5] The rig-discovery results are evaluated only on the Argoverse2 validation set, even though Argoverse2 is explicitly listed in the training data mix in Section 3.4. The 1-, 3-, 5-, and 7-camera settings in Figure 5 are subsampled versions of the same seven-camera rig geometry seen during training, so high Rig ID accuracy and rig mAA may reflect memorization of a familiar rig layout rather than generalization to a novel rig. The only held-out dataset, WayveScenes101, appears in Tables 1 and 2 but has no rig-discovery metrics. Since the abstract and Section 1 claim the first method to support rig calibration discovery from unordered images, the paper needs either rig-discovery metrics on WayveScenes101 (or another dataset with a genuinely different rig layout) or a clear demonstration of generalization to a rig configuration not present in training. The paper's own Section 5 limitation statement, which names rig-configuration diversity as the main performance limitation, reinforces that this evidence is missing.
  2. [Section 3.4 and Tables 1-2] The training mix includes 'an internal dataset' that is not described, and the evaluation on WayveScenes101 is presented as an unseen-dataset test. Given that the authors are from Wayve and WayveScenes101 is also a Wayve dataset, the manuscript must state whether the internal training data overlaps with WayveScenes101 or any of its scenes. Without an explicit non-overlap statement, the 'unseen' status of WayveScenes101 is not verifiable, and the generalization claims built on Tables 1 and 2 are weakened.
  3. [Table 1 and Section 4.1] On WayveScenes101, the unstructured variant Rig3RUnstr achieves only 25.7 mAA, substantially below DUSt3R-GA's 48.6 mAA. The text focuses on outperforming feedforward baselines and 'performing comparably to Rig COLMAP' (29.2 mAA), but this selective comparison does not address the paper's novelty claim: that rig structure can be inferred from unordered images on an unseen rig. A direct discussion of why Rig3RUnstr falls behind a global-alignment classical baseline on the held-out rig, and ideally rig-discovery metrics on that same data, is required to support the claimed capability.
minor comments (5)
  1. [Figure 5] The y-axis labels in Figure 5 range from 10 to 100 without units; add '%' or a stated metric unit for clarity.
  2. [Table 3] The checkmarks in the ablation table are ambiguous: specify that each row corresponds to adding that metadata component to the unstructured baseline, and define the baseline row explicitly.
  3. [Appendix B, Eq. (2)] The notation \(\tilde{u}\), \(\tilde{u}'\) is introduced only in text; define these as homogeneous image coordinates in the equation or its preceding sentence.
  4. [Abstract and Section 4] The claim of outperforming baselines by 17-45% mAA is stated without a pointer to a specific table or to which Rig3R variant is meant; clarify whether this range refers to Rig3RCalib, Rig3RUnstr, or a combination.
  5. [Section 4] All tables report single runs without error bars, confidence intervals, or repeated-seed statistics. Given the large reported gaps this is not disqualifying, but for a rigorous comparison please state that these are single runs and, where feasible, provide repeated-seed statistics or at least acknowledge the absence.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's predictions are evaluated against external ground truth and recovered via fixed closed-form solvers, with no fitted parameter or self-citation chain defining the reported metrics.

full rationale

I examined the derivation chain for circular reductions. The central outputs are dense pointmaps, global pose raymaps, and rig-relative raymaps. Camera poses are recovered from predicted raymaps using a fixed closed-form solver (Appendix B: intrinsic recovery from angular constraints and extrinsic recovery via SVD alignment), not by fitting parameters to the evaluation targets. Pose and reconstruction metrics are computed against external ground truth: Waymo LiDAR poses and WayveScenes101 COLMAP reconstructions, with classical and learned baselines evaluated under the same protocol. Rig discovery is measured by clustering the predicted rig raymaps and matching clusters to known rig labels via the Hungarian algorithm; the rig raymap head is supervised by external rig calibration ground truth, and the rig ID accuracy and rig mAA metrics are not parameters that were fit to produce those numbers. The use of Argoverse2 validation for rig-configuration generalization is a potential evaluation-scope concern, since Argoverse2 appears in the training data mix, but the paper refers to a validation split and this is not a circular derivation. Self-citations, such as initializing from DUSt3R and following VGGT for training stability, are not load-bearing for the reported results, and no uniqueness theorem is imported to force the design. The stated limitation about data diversity and rig-configuration variety is an honest external constraint, not a circular step. Overall, the claimed predictions are not equivalent to their inputs by construction.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The paper introduces no free-floating physical postulates. Its new ingredients are an output representation (rig raymap), a conditioning scheme (metadata embeddings with dropout), and a training recipe. The main load-bearing assumptions are the accuracy of ground-truth geometry, the rigidity and synchronization of the rig, and the sufficiency of the training data's rig diversity; the last is explicitly acknowledged as a limitation.

free parameters (5)
  • lambda_p (pose raymap loss weight) = not reported
    Hand-chosen weight balancing Lp_rmap in Ltotal; value not given in the main text.
  • lambda_r (rig raymap loss weight) = not reported
    Hand-chosen weight balancing Lr_rmap in Ltotal; value not given in the main text.
  • alpha (confidence regularization weight in Lpmap) = not reported
    Follows DUSt3R [11]; used in the pointmap loss but not specified in this paper.
  • beta (camera center loss weight) = not reported
    Hand-chosen weight for the center term in Lrmap; value not specified.
  • metadata dropout probability = 0.5
    Each metadata field dropped with 50% probability during training; a hand-set hyperparameter central to robustness.
assumptions (5)
  • domain assumption Pinhole camera model with principal point fixed at the image center for raymap recovery
    Invoked in Appendix B, Equation 1, to recover focal lengths and extrinsics from predicted ray directions; real cameras often have off-center principal points.
  • domain assumption Rig is synchronized and rigid across time
    Rig raymaps are defined in a rig-centric frame 'consistent across time' (Section 3), assuming fixed relative poses and synchronized capture.
  • domain assumption Ground-truth geometry from LiDAR and COLMAP is accurate
    Training supervision and the WayveScenes101 evaluation both depend on external reconstructions being reliable enough to serve as targets and metrics.
  • domain assumption Training data diversity covers the rig configurations seen at test time
    The claimed generalization to unseen rigs and rig discovery relies on the data mix including varied rig layouts; the authors list this as the main limitation in Section 5.
  • domain assumption Initializing the image encoder from DUSt3R provides a beneficial starting point
    Section 3.3 initializes from DUSt3R, asserting performance is not sensitive to this choice but providing no ablation for the full model.
invented entities (1)
  • Rig raymap independent evidence
    purpose: A per-pixel 6-DoF ray representation in a rig-centric frame, encoding each camera's pose relative to the rig and enabling rig structure discovery from images.
    This is a new output representation rather than a physical entity. It has an external handle: the rig discovery evaluation (Fig. 5a) clusters rig raymaps and matches them against known camera rig labels.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rig3R: Rig-Aware Conditioning for Learned 3D Reconstruction." pith.science (2026). https://pith.science/paper/4FEDSBBY

@misc{pith2026250602265,
  author       = {Pith},
  title        = {Pith review of: Rig3R: Rig-Aware Conditioning for Learned 3D Reconstruction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4FEDSBBY}},
  note         = {Machine review of arXiv:2506.02265}
}
read the original abstract

Estimating agent pose and 3D scene structure from multi-camera rigs is a central task in embodied AI applications such as autonomous driving. Recent learned approaches such as DUSt3R have shown impressive performance in multiview settings. However, these models treat images as unstructured collections, limiting effectiveness in scenarios where frames are captured from synchronized rigs with known or inferable structure. To this end, we introduce Rig3R, a generalization of prior multiview reconstruction models that incorporates rig structure when available, and learns to infer it when not. Rig3R conditions on optional rig metadata including camera ID, time, and rig poses to develop a rig-aware latent space that remains robust to missing information. It jointly predicts pointmaps and two types of raymaps: a pose raymap relative to a global frame, and a rig raymap relative to a rig-centric frame consistent across time. Rig raymaps allow the model to infer rig structure directly from input images when metadata is missing. Rig3R achieves state-of-the-art performance in 3D reconstruction, camera pose estimation, and rig discovery, outperforming both traditional and learned methods by 17-45% mAA across diverse real-world rig datasets, all in a single forward pass without post-processing or iterative refinement.

Figures

Figures reproduced from arXiv: 2506.02265 by the authors.

Figure 1
Figure 1. Rig3R is the first learned 3D vision model to leverage rig constraints when available, and the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Rig3R jointly predicts pointmaps, global raymaps, and rig-relative raymaps, with dropout [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Qualitative results of baselines vs Rig3R with and without rig embeddings. Fast3R fails to [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Qualitative results from Rig3R across diverse rig configurations, showing pointmaps and [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Rig3R generalizes across rig configurations. (a) Rig-relative pose mAA and Rig ID [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Visualizations of the qualitative effects of rig metadata embeddings on the Waymo validation [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Rig scene reconstructions on Argoverse [ [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Robustness to rig metadata noise before and after finetuning on noisy embeddings. We plot [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TRIG: Trajectory-Rig Decoupled Metric Geometry Learning

    cs.CV 2026-07 unverdicted novelty 6.0 of 10

    TRIG factorizes multi-camera poses into ego-trajectory and static rig geometry, with decoupled supervision and sparse temporal-spatial attention, claiming SOTA metric depth, pose, and 3D reconstruction on five driving...

  2. OmniNWM: Omniscient Driving Navigation World Models

    cs.CV 2025-10 conditional novelty 6.0 of 10

    OmniNWM jointly generates long panoramic multi-modal driving videos, controls them precisely via normalized Plücker ray-maps, and derives dense driving rewards from generated 3D occupancy.

  3. E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models

    cs.CV 2025-06 conditional novelty 6.0 of 10

    E3D-Bench compares 16 3D geometric foundation models on depth, reconstruction, pose, and view-synthesis tasks with a unified evaluation toolkit.

Reference graph

Works this paper leans on

79 extracted references · 40 canonical work pages · cited by 3 Pith papers

  1. [1]

    Multiple View Geometry in Computer Vision

    Richard Hartley and Andrew Zisserman. Multiple View Geometry in Computer Vision. Cam- bridge University Press, 2003

    work page 2003

  2. [2]

    Raúl Mur-Artal and Juan D. Tardós. ORB-SLAM2: an open-source SLAM system for monocu- lar, stereo and RGB-D cameras. IEEE Transactions on Robotics, 33(5):1255–1262, 2017. doi: 10.1109/TRO.2017.2705103

    arXiv 2017

  3. [3]

    Loam: Lidar odometry and mapping in real-time

    Ji Zhang and Sanjiv Singh. Loam: Lidar odometry and mapping in real-time. 07 2014. doi: 10.15607/RSS.2014.X.007

    doi:10.15607/rss.2014.x.007 2014

  4. [4]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis, 2020. URL https://arxiv.org/abs/2003.08934

    arXiv 2020

  5. [5]

    3d gaussian splatting for real-time radiance field rendering, 2023

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering, 2023. URL https://arxiv.org/abs/2308. 04079

    2023

  6. [6]

    Instant neural graphics primitives with a multiresolution hash encoding

    Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics, 41(4), 2022. ISSN 1557-7368. doi: 10.1145/3528223.3530127. URL http://dx.doi.org/10.1145/ 3528223.3530127

    arXiv 2022

  7. [7]

    Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction

    Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2106.10689, 2021

    arXiv 2021

  8. [8]

    Schonberger and Jan-Michael Frahm

    Johannes L. Schonberger and Jan-Michael Frahm. Structure-from-motion revisited. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    work page 2016

Show all 79 references
  1. [9]

    Raúl Mur-Artal, J. M. M. Montiel, and Juan D. Tardós. Orb-slam: A versatile and accurate monocular slam system. IEEE Transactions on Robotics, 31(5):1147–1163, 2015

    2015

  2. [10]

    Accurate, dense, and robust multiview stereopsis

    Yasutaka Furukawa and Jean Ponce. Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8):1362–1376, 2010

    2010

  3. [11]

    Dust3r: Geometric 3d vision made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  4. [12]

    Liang, Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, and Matt Feiszli

    Jianing Yang, Alexander Sax, Kevin J. Liang, Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, and Matt Feiszli. Fast3r: Towards 3d reconstruction of 1000+ images in one forward pass. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2025

  5. [13]

    Mv-dust3r+: Single-stage scene reconstruction from sparse views in 2 seconds

    Zhenggang Tang, Yuchen Fan, Dilin Wang, Hongyu Xu, Rakesh Ranjan, Alexander Schwing, and Zhicheng Yan. Mv-dust3r+: Single-stage scene reconstruction from sparse views in 2 seconds. arXiv preprint arXiv:2412.06974, 2024

    2024 arXiv

  6. [14]

    Probabilistic structure matching for visual slam with a multi-camera rig

    Michael Kaess and Frank Dellaert. Probabilistic structure matching for visual slam with a multi-camera rig. Computer Vision and Image Understanding, 114(2):286–296, 2010

    2010

  7. [15]

    Leveraging image-based localization for infrastructure-based calibration of a multi-camera rig

    Lionel Heng, Paul Furgale, and Marc Pollefeys. Leveraging image-based localization for infrastructure-based calibration of a multi-camera rig. Journal of Field Robotics, 32(5):775–802, 2015

    2015

  8. [16]

    Gerardo Carrera, Adrien Angeli, and Andrew J. Davison. Slam-based automatic extrinsic calibration of a multi-camera rig. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 2652–2659. IEEE, 2011

    2011

  9. [17]

    Schönberger

    Linfei Pan, Dániel Baráth, Marc Pollefeys, and Johannes L. Schönberger. Global structure- from-motion revisited, 2024. URL https://arxiv.org/abs/2407.20219. 13

    2024 arXiv

  10. [18]

    Walter, Vitor Campagnolo Guizilini, and Greg Shakhnarovich

    Jiahao Li, Haochen Wang, Muhammad Zubair Irshad, Igor Vasiljevic, Matthew R. Walter, Vitor Campagnolo Guizilini, and Greg Shakhnarovich. Fastmap: Revisiting dense and scalable structure from motion, 2025. URL https://arxiv.org/abs/2505.04612

    2025 arXiv

  11. [19]

    Superpoint: Self-supervised interest point detection and description, 2018

    Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superpoint: Self-supervised interest point detection and description, 2018. URL https://arxiv.org/abs/1712.07629

    2018 arXiv

  12. [20]

    Superglue: Learning feature matching with graph neural networks, 2020

    Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks, 2020. URL https://arxiv.org/ abs/1911.11763

    2020 arXiv

  13. [21]

    Back to the feature: Learning robust camera localization from pixels to pose, 2021

    Paul-Edouard Sarlin, Ajaykumar Unagar, Måns Larsson, Hugo Germain, Carl Toft, Viktor Larsson, Marc Pollefeys, Vincent Lepetit, Lars Hammarstrand, Fredrik Kahl, and Torsten Sattler. Back to the feature: Learning robust camera localization from pixels to pose, 2021. URL https://...

    2021 arXiv

  14. [22]

    LoFTR: Detector-free local feature matching with transformers

    Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. LoFTR: Detector-free local feature matching with transformers. CVPR, 2021

    2021

  15. [23]

    RoMa: Robust Dense Feature Matching

    Johan Edstedt, Qiyu Sun, Georg Bökman, Mårten Wadenbäck, and Michael Felsberg. RoMa: Robust Dense Feature Matching. arXiv preprint arXiv:2305.15404, 2023

    2023 arXiv

  16. [24]

    Ricardo Martin-Brualla, Noha Radwan, Mehdi S. M. Sajjadi, Jonathan T. Barron, Alexey Dosovitskiy, and Daniel Duckworth. Nerf in the wild: Neural radiance fields for unconstrained photo collections, 2021. URL https://arxiv.org/abs/2008.02268

    2021 arXiv

  17. [25]

    Das3r: Dynamics-aware gaussian splatting for static scene reconstruction, 2024

    Kai Xu, Tze Ho Elden Tse, Jizong Peng, and Angela Yao. Das3r: Dynamics-aware gaussian splatting for static scene reconstruction, 2024. URL https://arxiv.org/abs/2412.19584

    2024 arXiv

  18. [26]

    MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-view Images , page 370–386

    Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-view Images , page 370–386. Springer Nature Switzerland, October 2024. ISBN 9783031726644. doi: 10.1...

    2024 doi

  19. [27]

    Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision

    Ruicheng Wang, Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. arXiv preprint arXiv:2410.19115, 2024

    arXiv

  20. [28]

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin ElNouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabb...

    2024

  21. [29]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  22. [30]

    Vggsfm: Visual geometry grounded deep structure from motion

    Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny. Vggsfm: Visual geometry grounded deep structure from motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  23. [31]

    Efros, and Angjoo Kanazawa

    Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A. Efros, and Angjoo Kanazawa. Continuous 3d perception model with persistent state, 2025. URL https://arxiv.org/abs/ 2501.12387

    2025 arXiv

  24. [32]

    3d reconstruction with spatial memory, 2024

    Hengyi Wang and Lourdes Agapito. 3d reconstruction with spatial memory, 2024. URL https://arxiv.org/abs/2408.16061

    2024 arXiv

  25. [33]

    Monst3r: A simple approach for estimating geometry in the presence of motion, 2025

    Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. Monst3r: A simple approach for estimating geometry in the presence of motion, 2025. URL https://arxiv.org/abs/2410.03825. 14

    2025 arXiv

  26. [34]

    Stereo4d: Learning how things move in 3d from internet stereo videos, 2025

    Linyi Jin, Richard Tucker, Zhengqi Li, David Fouhey, Noah Snavely, and Aleksander Holynski. Stereo4d: Learning how things move in 3d from internet stereo videos, 2025. URL https: //arxiv.org/abs/2412.09621

    2025 arXiv

  27. [35]

    Easi3r: Estimating disentangled motion from dust3r without training, 2025

    Xingyu Chen, Yue Chen, Yuliang Xiu, Andreas Geiger, and Anpei Chen. Easi3r: Estimating disentangled motion from dust3r without training, 2025. URL https://arxiv.org/abs/ 2503.24391

    2025

  28. [36]

    Slam3r: Real-time dense scene reconstruction from monocular rgb videos,

    Yuzheng Liu, Siyan Dong, Shuzhe Wang, Yingda Yin, Yanchao Yang, Qingnan Fan, and Baoquan Chen. Slam3r: Real-time dense scene reconstruction from monocular rgb videos,

  29. [38]

    Riku Murai, Eric Dexheimer, and Andrew J. Davison. Mast3r-slam: Real-time dense slam with 3d reconstruction priors, 2024. URL https://arxiv.org/abs/2412.12392

    2024 arXiv

  30. [39]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025

    2025

  31. [40]

    Pow3r: Empowering unconstrained 3d reconstruction with camera and scene priors, 2025

    Wonbong Jang, Philippe Weinzaepfel, Vincent Leroy, Lourdes Agapito, and Jerome Revaud. Pow3r: Empowering unconstrained 3d reconstruction with camera and scene priors, 2025. URL https://arxiv.org/abs/2503.17316

    2025 arXiv

  32. [41]

    Epnp: An accurate o(n) solution to the pnp problem

    Vincent Lepetit, Francesc Moreno-Noguer, and Pascal Fua. Epnp: An accurate o(n) solution to the pnp problem. International Journal of Computer Vision, 81(2):155–166, 2009

    2009

  33. [42]

    Fischler and Robert C

    Martin A. Fischler and Robert C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381–395, 1981

    1981

  34. [43]

    Posenet: A convolutional network for real-time 6-dof camera relocalization, 2016

    Alex Kendall, Matthew Grimes, and Roberto Cipolla. Posenet: A convolutional network for real-time 6-dof camera relocalization, 2016. URL https://arxiv.org/abs/1505.07427

    2016 arXiv

  35. [44]

    Zhang, Amy Lin, Moneish Kumar, Tzu-Hsuan Yang, Deva Ramanan, and Shubham Tulsiani

    Jason Y . Zhang, Amy Lin, Moneish Kumar, Tzu-Hsuan Yang, Deva Ramanan, and Shubham Tulsiani. Cameras as rays: Pose estimation via ray diffusion, 2024. URL https://arxiv. org/abs/2402.14817

    2024 arXiv

  36. [45]

    Reloc3r: Large-scale training of relative camera pose regression for generalizable, fast, and accurate visual localization, 2025

    Siyan Dong, Shuzhe Wang, Shaohui Liu, Lulu Cai, Qingnan Fan, Juho Kannala, and Yanchao Yang. Reloc3r: Large-scale training of relative camera pose regression for generalizable, fast, and accurate visual localization, 2025. URL https://arxiv.org/abs/2412.08376

    2025 arXiv

  37. [46]

    Matching 2d images in 3d: Metric relative pose from metric correspondences, 2024

    Axel Barroso-Laguna, Sowmya Munukutla, Victor Adrian Prisacariu, and Eric Brachmann. Matching 2d images in 3d: Metric relative pose from metric correspondences, 2024. URL https://arxiv.org/abs/2404.06337

    2024 arXiv

  38. [47]

    Tinghui Zhou, Matthew Brown, Noah Snavely, and David G. Lowe. Unsupervised learning of depth and ego-motion from video, 2017. URL https://arxiv.org/abs/1704.07813

    2017 arXiv

  39. [48]

    Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras, 2022

    Zachary Teed and Jia Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras, 2022. URL https://arxiv.org/abs/2108.10869

    2022 arXiv

  40. [49]

    Erik Sandström, Yue Li, Luc Van Gool, and Martin R. Oswald. Point-slam: Dense neural point cloud-based slam. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023

    2023

  41. [50]

    Oswald, and Marc Pollefeys

    Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin R. Oswald, and Marc Pollefeys. Nice-slam: Neural implicit scalable encoding for slam. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022. 15

    2022

  42. [51]

    Splatam: Splat, track & map 3d gaussians for dense rgb-d slam

    Nikhil Keetha, Jay Karhade, Krishna Murthy Jatavallabhula, Gengshan Yang, Sebastian Scherer, Deva Ramanan, and Jonathon Luiten. Splatam: Splat, track & map 3d gaussians for dense rgb-d slam. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    2024

  43. [52]

    Grounding image matching in 3d with mast3r

    Vincent Leroy, Yohann Cabon, and Jérôme Révaud. Grounding image matching in 3d with mast3r. arXiv preprint arXiv:2406.09756, 2024

    2024 arXiv

  44. [53]

    Scene coordinate reconstruction: Posing of image collections via incremental learning of a relocalizer

    Eric Brachmann, Jamie Wynn, Shuai Chen, Tommaso Cavallari, Áron Monszpart, Daniyar Turmukhambetov, and Victor Adrian Prisacariu. Scene coordinate reconstruction: Posing of image collections via incremental learning of a relocalizer. In ECCV, 2024

    2024

  45. [54]

    Somani Arun, Thomas S

    K. Somani Arun, Thomas S. Huang, and Steven D. Blostein. Least-squares fitting of two 3-d point sets. IEEE Transactions on Pattern Analysis and Machine Intelligence , (5):698–700, 1987

    1987

  46. [55]

    An image is worth 16x16 words: Transformers for image recognition at scale, 2021

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021 arXiv

  47. [56]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. URL https://arxiv. org/abs/1706.03762

    2023 arXiv

  48. [57]

    Vision transformers for dense prediction,

    René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction,

  49. [58]

    Modelling uncertainty in deep learning for camera relo- calization

    Alex Kendall and Roberto Cipolla. Modelling uncertainty in deep learning for camera relo- calization. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pages 4762–4769. IEEE, 2016

    2016

  50. [59]

    Capturing the geometry of object categories from video supervision

    David Novotny, Diane Larlus, and Andrea Vedaldi. Capturing the geometry of object categories from video supervision. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2018

    2018

  51. [60]

    Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction

    Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In International Conference on Computer Vision, 2021

    2021

  52. [61]

    Blendedmvs: A large-scale dataset for generalized multi-view stereo networks.Computer Vision and Pattern Recognition (CVPR), 2020

    Yao Yao, Zixin Luo, Shiwei Li, Jingyang Zhang, Yufan Ren, Lei Zhou, Tian Fang, and Long Quan. Blendedmvs: A large-scale dataset for generalized multi-view stereo networks.Computer Vision and Pattern Recognition (CVPR), 2020

    2020

  53. [62]

    Map-free visual relocalization: Metric pose relative to a single image

    Eduardo Arnold, Jamie Wynn, Sara Vicente, Guillermo Garcia-Hernando, Áron Monszpart, Victor Adrian Prisacariu, Daniyar Turmukhambetov, and Eric Brachmann. Map-free visual relocalization: Metric pose relative to a single image. In ECCV, 2022

    2022

  54. [63]

    Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner

    Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2017

    2017

  55. [64]

    Mvimgnet: A large-scale dataset of multi-view images

    Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Tianyou Liang, Guanying Chen, Shuguang Cui, and Xiaoguang Han. Mvimgnet: A large-scale dataset of multi-view images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patter...

    2023

  56. [65]

    Harley, Bokui Shen, Gordon Wetzstein, and Leonidas J

    Yang Zheng, Adam W. Harley, Bokui Shen, Gordon Wetzstein, and Leonidas J. Guibas. Pointodyssey: A large-scale synthetic dataset for long-term point tracking, 2023. URL https://arxiv.org/abs/2307.15055

    2023 arXiv

  57. [66]

    Virtual kitti 2, 2020

    Yohann Cabon, Naila Murray, and Martin Humenberger. Virtual kitti 2, 2020. URL https: //arxiv.org/abs/2001.10773. 16

    2020 arXiv

  58. [67]

    Tartanair: A dataset to push the limits of visual slam

    Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. Tartanair: A dataset to push the limits of visual slam. 2020

    2020

  59. [68]

    Pandaset: Advanced sensor suite dataset for autonomous driving

    Pengchuan Xiao, Zhenlei Shao, Steven Hao, Zishuo Zhang, Xiaolin Chai, Judy Jiao, Zesong Li, Jian Wu, Kai Sun, Kun Jiang, et al. Pandaset: Advanced sensor suite dataset for autonomous driving. In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) , pag...

    2021

  60. [69]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3354–3361. IEEE, 2012

    2012

  61. [70]

    Argoverse 2: Next generation datasets for self-driving perception and forecasting.arXiv preprint arXiv:2301.00493, 2023

    Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, et al. Argoverse 2: Next generation datasets for self-driving perception and forecasting.arXiv preprint arXiv:2301...

    2023 arXiv

  62. [71]

    Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom

    Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. arXiv preprint arXiv:1903.11027, 2019

    1903 arXiv

  63. [72]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al. Scalability in perception for autonomous driving: Waymo open dataset. In Proceedings of the IEEE/CVF Conference on Computer Vi...

    2020

  64. [73]

    Omnire: Omni urban scene reconstruction

    Ziyu Chen, Jiawei Yang, Jiahui Huang, Riccardo de Lutio, Janick Martinez Esturo, Boris Ivanovic, Or Litany, Zan Gojcic, Sanja Fidler, Marco Pavone, Li Song, and Yue Wang. Omnire: Omni urban scene reconstruction. arXiv preprint arXiv:2408.16760, 2024

    2024 arXiv

  65. [74]

    Wayvescenes101: A dataset and benchmark for novel view synthesis in autonomous driving, 2024

    Jannik Zürn, Paul Gladkov, Sofía Dudas, Fergal Cotter, Sofi Toteva, Jamie Shotton, Vasiliki Simaiaki, and Nikhil Mohan. Wayvescenes101: A dataset and benchmark for novel view synthesis in autonomous driving, 2024. URL https://arxiv.org/abs/2407.08280

    2024 arXiv

  66. [75]

    Posediffusion: Solving pose esti- mation via diffusion-aided bundle adjustment, 2024

    Jianyuan Wang, Christian Rupprecht, and David Novotny. Posediffusion: Solving pose esti- mation via diffusion-aided bundle adjustment, 2024. URL https://arxiv.org/abs/2306. 15667

    2024

  67. [76]

    Relpose: Predicting probabilistic relative rotation for single objects in the wild

    Jason Y Zhang, Deva Ramanan, and Shubham Tulsiani. Relpose: Predicting probabilistic relative rotation for single objects in the wild. In European Conference on Computer Vision (ECCV), pages 592–611. Springer, 2022

    2022

  68. [77]

    Image matching across wide baselines: From paper to practice

    Yuhe Jin, Dmytro Mishkin, Anastasiia Mishchuk, Jiri Matas, Pascal Fua, Kwang Moo Yi, and Eduard Trulls. Image matching across wide baselines: From paper to practice. International Journal of Computer Vision, 129(2):517–547, 2021

    2021

  69. [78]

    A Consistently Fast and Globally Optimal Solution to the Perspective-n-Point Problem, pages 478–494

    George Terzakis and Manolis Lourakis. A Consistently Fast and Globally Optimal Solution to the Perspective-n-Point Problem, pages 478–494. 11 2020. ISBN 978-3-030-58451-1. doi: 10.1007/978-3-030-58452-8_28. 17

    2020 doi

  70. [2021]

    URL https://arxiv.org/abs/2103.13413

    arXiv

  71. [2025]

    URL https://arxiv.org/abs/2412.09401

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.