Pith. sign in

REVIEW 5 major objections 7 minor 56 references

Learned flow, not recurrent architecture, drives deep SLAM success

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-08 19:05 UTC pith:SY66XWQ6

load-bearing objection Controlled ablation showing learned data association + uncertainty matter more than recurrence in V-SLAM; conclusion is slightly overstated due to confounded backend differences the 5 major comments →

arxiv 2607.06023 v1 pith:SY66XWQ6 submitted 2026-07-07 cs.CV cs.RO

Why does Deep Learning Improve Visual SLAM?

Giovanni Cioffi , Davide Scaramuzza This is my paper
classification cs.CV cs.RO
keywords Visual SLAMdeep learningoptical flowlearned uncertaintybundle adjustmentdata associationORB-SLAM3DROID-SLAM
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks a precise question: what makes deep-learning-based Visual SLAM systems outperform classical geometry-based ones? The candidates are learned 2D data association (optical flow), learned uncertainty estimation, and the recurrent architecture that iterates between flow prediction and bundle adjustment. The authors answer by performing a controlled substitution: they take ORB-SLAM3, a mature classical feed-forward SLAM pipeline, and replace only its handcrafted descriptor-matching module with the optical-flow network from DROID-SLAM, producing ORB-SLAM3-OF. They then add the flow network's confidence estimates as weights in bundle adjustment, producing ORB-SLAM3-OF-U. Everything else in the classical pipeline — tracking, local mapping, keyframe culling, loop closure — stays unchanged. On two challenging benchmarks (TartanAir and UZH-FPV), ORB-SLAM3-OF-U matches state-of-the-art deep SLAM systems on in-distribution data and outperforms them on out-of-distribution data. The conclusion: the recurrent architecture is not necessary; learned data association and learned uncertainty are the load-bearing components.

Core claim

The central mechanism is a controlled ablation by substitution. By swapping only the 2D correspondence engine and the residual weighting scheme in a classical SLAM pipeline — leaving the feed-forward architecture, the bundle adjustment backend, and all heuristics intact — the authors isolate the contribution of learned optical flow and learned uncertainty from the contribution of the recurrent differentiable architecture used in systems like DROID-SLAM. The result is that a classical feed-forward pipeline with learned correspondence and uncertainty achieves state-of-the-art trajectory accuracy, demonstrating that recurrence is not the essential ingredient. The learned uncertainty proves most

What carries the argument

ORB-SLAM3-OF-U: ORB-SLAM3 with its descriptor matching replaced by DROID-SLAM's optical-flow network (flow + confidence), and bundle-adjustment residuals weighted by the network's predicted uncertainty instead of uniform or pyramid-level weights.

Load-bearing premise

The claim that the recurrent architecture is unnecessary rests on the assumption that ORB-SLAM3's classical multi-stage pipeline — with its keyframe culling, local mapping, loop closure, and chi-squared outlier rejection — is a fair representative of a non-recurrent architecture, when in fact these structural redundancies may be doing significant work that DROID-SLAM's simpler differentiable backend does not have.

What would settle it

If a recurrent architecture with the same learned flow and uncertainty components were added on top of the ORB-SLAM3 pipeline and produced materially better trajectory accuracy on the same benchmarks, the claim that recurrence is unnecessary would be weakened.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Next-generation V-SLAM systems can use classical feed-forward pipelines with learned frontends for data association and uncertainty, avoiding the computational and implementation complexity of recurrent differentiable architectures.
  • Learned uncertainty estimation is most critical under domain shift and high-noise conditions, where classical outlier rejection (chi-squared tests, Huber loss) fails to adequately filter bad correspondences.
  • The robustness of learned optical flow in low-texture, low-illumination, and high-motion-blur scenarios suggests that the bottleneck in classical SLAM is the frontend correspondence engine, not the backend optimizer.
  • Practitioners can improve existing classical SLAM deployments by swapping in a learned correspondence module without redesigning their entire system architecture.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The comparison between ORB-SLAM3-OF-U and recurrent systems like DROID-SLAM conflates two variables: feed-forward vs. recurrent architecture, and ORB-SLAM3's multi-stage outlier-rejection pipeline vs. DROID-SLAM's differentiable BA backend. The classical pipeline's structural redundancies (keyframe culling, local mapping, loop closure) may contribute to the result, meaning the claim that recurrenc
  • If the flow network is the bottleneck component, then improvements in optical-flow models (e.g., from larger training datasets or transformer-based architectures) should transfer directly to SLAM performance without architectural changes to the pipeline — a testable prediction.
  • The finding that learned uncertainty matters most under domain shift (UZH-FPV) suggests that uncertainty calibration could be a productive research direction: a well-calibrated uncertainty estimate may matter more than a marginally better flow prediction.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. This paper investigates which components of deep learning-based visual SLAM systems are responsible for their superior performance: learned 2D data association, learned uncertainty, or the recurrent architecture. The authors take ORB-SLAM3 and replace its descriptor-based matching with the optical flow network from DROID-SLAM, creating ORB-SLAM3-OF, and additionally incorporate the network's uncertainty estimates into bundle adjustment, creating ORB-SLAM3-OF-U. They evaluate these variants against DROID-SLAM, DPVO, and DPV-SLAM on the TartanAir and UZH-FPV benchmarks. The main finding is that learned data association and uncertainty are the key drivers of performance, while the recurrent architecture is not strictly necessary, as ORB-SLAM3-OF-U achieves state-of-the-art results using a classical feed-forward pipeline.

Significance. The paper addresses a well-motivated and timely question for the V-SLAM community. The modular experimental design—swapping one component at a time within a fixed classical pipeline—is a clean way to isolate causal factors. The finding that a classical feed-forward backend with a learned frontend can match or exceed recurrent architectures on out-of-distribution data is a useful design principle. The authors provide falsifiable empirical predictions across two challenging benchmarks with multiple baselines. The promise of open-source code release is noted as a positive factor for reproducibility.

major comments (5)
  1. Sec. V.3 and the Conclusion claim that the recurrent architecture is 'not strictly necessary' to achieve state-of-the-art accuracy. However, the comparison between ORB-SLAM3-OF-U and the recurrent systems (DROID-SLAM, DPVO, DPV-SLAM) does not isolate recurrence as a variable. The systems differ along multiple axes: (1) backend type (classical BA with iterative chi-squared outlier rejection vs. differentiable dense/sparse BA), (2) loop closure (ORB-SLAM3 has full loop closing; DPVO has none; DPV-SLAM's classical loop closure was disabled per Sec. III.B), (3) keyframe management, and (4) frame-to-map vs. frame-to-frame tracking. The authors acknowledge this in Sec. V.3, noting that ORB-SLAM3 'incorporates structural redundancies that enable accurate trajectory estimation even in the presence of a significant number of outlier measurements.' This acknowledgment undercuts the strong claim: a
  2. The claim that recurrence is unnecessary rests heavily on the UZH-FPV results (Table II), where ORB-SLAM3-OF-U achieves the best average ATE_T (0.63 vs. 1.08 for DPV-SLAM). But on TartanAir (Table I), ORB-SLAM3-OF-U (0.34 ATE_T) underperforms DPVO (0.24) and DROID-SLAM (0.28). Since the flow network is pre-trained on TartanAir, the TartanAir evaluation is in-distribution for the learned frontend. The UZH-FPV results, where ORB-SLAM3-OF-U excels, are out-of-distribution—and it is precisely in the OOD setting that the classical backend's outlier-rejection machinery (loop closure, keyframe culling, chi-squared tests) is most likely to compensate for noisy flow predictions. The paper should explicitly acknowledge that the evidence for the 'recurrence is unnecessary' claim is strongest in the OOD setting where backend differences are most confounded, and should soften the conclusion to match.
  3. Sec. III.C.2 states that for keyframe matching, the system uses flow-based matching within a 5-frame temporal window and reverts to BoW-based matching beyond that. This means that for longer-baseline matches (which are critical for loop closure and scale correction), the system falls back to the classical ORB descriptor pipeline. This hybrid matching strategy is not a pure replacement of data association with learned flow; it is a mixed system. The paper should clarify how this fallback affects the interpretation of the results, particularly for sequences where loop closure is triggered.
  4. Table I: MH004 is excluded from the TartanAir evaluation (footnote 1: 'rendering artifacts, such as missing surfaces'). However, DPV-SLAM reports a result for MH004 (4.98/20.08), suggesting it was evaluated. If MH004 is excluded only for some systems but not others, the average ATE comparison is not on equal footing. Please clarify whether MH004 is excluded from all systems' averages or only from ORB-SLAM3 variants. If the latter, the averages are not directly comparable.
  5. Sec. III.C.3: The paper states that 'combining classical ORB-based matches with this learned uncertainty is methodologically inconsistent.' This is an important observation, but it is unclear whether the ORB-SLAM3-OF-U system ever uses ORB-based matches (e.g., in the BoW fallback for keyframes beyond the 5-frame window). If so, those matches would carry uniform or pyramid-level weights while flow-based matches carry learned uncertainty weights, creating a mixed weighting scheme within the same BA. Please clarify whether this occurs and discuss its potential impact.
minor comments (7)
  1. Sec. III.C.1: The flow convergence threshold (0.0625) and maximum iterations (20) are listed as fixed parameters. Were these tuned per dataset? If so, this should be noted. If not, a brief justification for these values would help reproducibility.
  2. Table II: The asterisk notation for In. 45 9 and In. 45 14 indicates that error metrics are computed for only 76% and 84% of frames, respectively, due to missing frames causing tracking loss. The paper should clarify whether the DL-based systems (which maintained tracking via constant-velocity propagation) are also evaluated on the same subset of frames, or on the full sequence. If the latter, the comparison is not on equal footing.
  3. Fig. 2: The diagram for ORB-SLAM3-OF-U (middle) shows 'Flow and Uncertainty Network' appearing twice. This is slightly confusing—it would help to clarify that the same network is used for both tracking and mapping, or to merge the boxes.
  4. Sec. IV.A: The paper states that 'parameters for each V-SLAM system were specifically tuned for each dataset.' This per-dataset tuning is a potential confound for the generalization claims. A brief discussion of which parameters were tuned and how sensitive the results are to this tuning would strengthen the contribution.
  5. Table I: The average ATE_T for ORB-SLAM3 is listed as '-' because it fails on several sequences. It would be more informative to also report the average over only the sequences it completes, or to use a failure-penalty convention, so readers can assess the partial performance.
  6. Sec. III.B: The description of DPV-SLAM states that 'classical loop closure utilizes image retrieval (e.g., BoW2) and pose graph optimization.' It would help to clarify whether DPV-SLAM's proximity loop closure (which is enabled) also contributes to its global consistency, or whether it is purely a local factor.
  7. References: Several arXiv preprints are cited with access dates (e.g., [3], [21], [24], [56]). Please ensure that published versions are cited where available.

Simulated Author's Rebuttal

5 responses · 0 unresolved

We thank the referee for a careful and constructive review. The referee raises valid concerns about confounding variables in our comparison between ORB-SLAM3-OF-U and recurrent systems, the asymmetry of the MH004 exclusion, and the hybrid matching strategy. We agree to revise the manuscript to soften the recurrence claim, clarify the MH004 exclusion, and explicitly discuss the BoW fallback and mixed weighting scheme. We respectfully defend the core finding that learned data association and uncertainty are the primary drivers of performance, while acknowledging that the recurrence claim is necessarily qualified.

read point-by-point responses

  1. Referee: Sec. V.3 and the Conclusion claim that the recurrent architecture is 'not strictly necessary' to achieve state-of-the-art accuracy. However, the comparison between ORB-SLAM3-OF-U and the recurrent systems does not isolate recurrence as a variable. The systems differ along multiple axes: backend type, loop closure, keyframe management, and frame-to-map vs. frame-to-frame tracking. The authors acknowledge this in Sec. V.3, which undercuts the strong claim.

    Authors: The referee is correct that our comparison does not isolate recurrence as a single variable. The systems differ along multiple axes, and we acknowledge this in Sec. V.3. We will revise the manuscript to make the scope of our claim more precise. Our evidence supports the narrower, defensible statement that a classical feed-forward pipeline with learned data association and uncertainty can match or exceed recurrent architectures on these benchmarks—not that recurrence has been independently isolated as a variable and found unnecessary. The broader claim in the abstract and conclusion will be softened accordingly. We note that the paper's primary contribution is the controlled ablation within the ORB-SLAM3 framework (ORB-SLAM3 → ORB-SLAM3-OF → ORB-SLAM3-OF-U), which does isolate learned data association and uncertainty as variables. The comparison to recurrent systems is supplementary evidence that the resulting system is competitive, not a controlled isolation of recurrence. revision: yes

  2. Referee: The claim that recurrence is unnecessary rests heavily on the UZH-FPV results (OOD), where ORB-SLAM3-OF-U excels. On TartanAir (in-distribution), ORB-SLAM3-OF-U underperforms DPVO and DROID-SLAM. The paper should acknowledge that the evidence for the recurrence claim is strongest in the OOD setting where backend differences are most confounded, and should soften the conclusion to match.

    Authors: This is a fair and incisive observation. We agree that the OOD setting is precisely where the classical backend's outlier-rejection machinery (loop closure, keyframe culling, chi-squared tests) is most likely to compensate for noisy flow predictions, making it difficult to attribute the performance advantage solely to the absence of recurrence. We will add an explicit discussion of this confound in Sec. V.3 and soften the conclusion to state that our results demonstrate the feasibility of feed-forward architectures with learned frontends, rather than claiming recurrence is unnecessary. We will also note the TartanAir in-distribution results, where recurrent systems still hold an advantage, as evidence that recurrence may provide benefits in-distribution that our classical pipeline does not fully replicate. revision: yes

  3. Referee: Sec. III.C.2 states that for keyframe matching, the system uses flow-based matching within a 5-frame temporal window and reverts to BoW-based matching beyond that. This hybrid matching strategy is not a pure replacement of data association with learned flow. The paper should clarify how this fallback affects the interpretation of the results, particularly for sequences where loop closure is triggered.

    Authors: The referee correctly identifies that our system uses a hybrid matching strategy: learned optical flow for keyframe pairs within a 5-frame temporal window, and classical BoW-based matching for longer-baseline matches including those used in loop closure. We will add a clarifying paragraph in Sec. III.C.2 explaining this design choice and its implications. The fallback to BoW matching for loop closure means that loop closure performance is attributable to the classical ORB-SLAM3 pipeline, not to learned data association. This is important for interpreting the UZH-FPV results: the loop closure advantage of ORB-SLAM3-OF-U over DPVO (which has no loop closure) and DPV-SLAM (whose classical loop closure was disabled) is a backend difference, not a frontend difference. We will make this explicit in the discussion. revision: yes

  4. Referee: Table I: MH004 is excluded from the TartanAir evaluation (footnote: 'rendering artifacts'). However, DPV-SLAM reports a result for MH004 (4.98/20.08). If MH004 is excluded only for some systems but not others, the average ATE comparison is not on equal footing. Please clarify whether MH004 is excluded from all systems' averages or only from ORB-SLAM3 variants.

    Authors: MH004 is excluded from the averages of all systems in Table I, not only from the ORB-SLAM3 variants. The DPV-SLAM result for MH004 (4.98/20.08) is reported in the table for completeness, but it is not included in the average. We will clarify the footnote to state explicitly that MH004 is excluded from all averages due to rendering artifacts. We note that including MH004 in the DPV-SLAM average would actually worsen DPV-SLAM's reported average ATE_T (from 1.76 to approximately 2.05), so the current exclusion does not favor our system. We will also verify that the averages for all other systems are computed over the same set of 14 sequences. revision: yes

  5. Referee: Sec. III.C.3 states that 'combining classical ORB-based matches with this learned uncertainty is methodologically inconsistent.' If the BoW fallback for keyframes beyond the 5-frame window uses ORB-based matches, those matches would carry uniform or pyramid-level weights while flow-based matches carry learned uncertainty weights, creating a mixed weighting scheme within the same BA. Please clarify whether this occurs and discuss its potential impact.

    Authors: The referee is correct that this mixed weighting situation does occur. When BoW-based matches are used for keyframe pairs beyond the 5-frame temporal window (including loop closure matches), those matches carry the standard ORB-SLAM3 pyramid-level weights, while flow-based matches within the 5-frame window carry learned uncertainty weights. This creates a mixed weighting scheme within the same bundle adjustment. We will add a discussion of this in Sec. III.C.3. In practice, the impact is limited because the BoW-based matches are primarily used for loop closure, which operates on a pose-graph optimization (similarity transformation + essential graph) rather than the local bundle adjustment where the learned uncertainty weights are applied. The local BA, where learned uncertainty is most impactful, predominantly involves keyframe pairs within the 5-frame window where flow-based matching is used. We will clarify this distinction in the revised manuscript. revision: yes

Circularity Check

0 steps flagged

No circularity found; the paper is an empirical study with no derivation chain to reduce to its inputs.

full rationale

This paper makes an empirical claim — that learned 2D data association and uncertainty, not recurrence, drive DL-based V-SLAM success — supported by controlled experiments on external benchmarks (TartanAir, UZH-FPV) using an externally developed flow network (DROID-SLAM's, by Teed and Deng). There is no first-principles derivation chain, no fitted constants presented as predictions, and no uniqueness theorem invoked. The authors cite their own prior work for the UZH-FPV dataset [8], the ground-truth computation method [54], and the ATE metric tutorial [50], but none of these are load-bearing for the central claim: the results are falsifiable by any researcher running the same systems on the same public datasets. The skeptic's concern about confounded backend differences (ORB-SLAM3's outlier-rejection redundancies vs. DROID-SLAM's differentiable BA) is a validity/correctness concern, not a circularity concern — the paper does not define its conclusion in terms of its inputs, nor does it smuggle an ansatz through self-citation. Score 1 reflects the presence of minor self-citations that are methodological context rather than load-bearing for the central claim.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

The paper is an empirical study with no new entities or theoretical constructs. Free parameters are engineering thresholds. The key axiom is that the experimental design isolates the recurrent architecture variable, which is only partially true given the multiple architectural differences between ORB-SLAM3 and DROID-SLAM.

free parameters (7)
  • Flow confidence threshold for valid match = 0.3
    Used in ORB-SLAM3-OF-U to reject low-confidence correspondences (Sec. III.C.2). Chosen by the authors, not derived.
  • Feature matching search radius = 2 pixels
    Radius for finding closest feature to predicted flow location (Sec. III.C.2). Set by hand.
  • Temporal window for flow-based keyframe matching = 5 frames
    Keyframe pairs within this window use flow matching; beyond it, BoW is used (Sec. III.C.2). Chosen empirically.
  • Flow convergence threshold = 0.0625
    Mean flow revision threshold for ConvGRU iteration termination (Sec. III.C.1). Equivalent to ~0.5 pixels at original resolution.
  • Maximum ConvGRU iterations = 20
    Hard cap on flow network iterations (Sec. III.C.1).
  • BA inlier threshold = 7
    Minimum inliers for tracking success (Sec. III.C.2). Inherited from ORB-SLAM3 defaults.
  • Per-dataset system parameters = unspecified
    The paper states 'parameters for each V-SLAM system were specifically tuned for each dataset' (Sec. IV) but does not report the tuned values.
axioms (5)
  • domain assumption ORB-SLAM3 is a representative and highly optimized classical V-SLAM baseline
    Invoked in Sec. III.A to justify the choice of ORB-SLAM3 as the classical system for controlled study. Reasonable but not the only choice.
  • domain assumption DROID-SLAM, DPVO, and DPV-SLAM define the state-of-the-art in trajectory estimation
    Invoked in Sec. I and Sec. III.A to justify the choice of DL baselines. These are strong systems but the claim excludes newer 3D foundation model approaches.
  • domain assumption TartanAir and UZH-FPV are the most challenging V-SLAM benchmarks
    Invoked in Sec. I and Sec. IV. These are challenging, but the claim of 'most challenging' is subjective and excludes other difficult benchmarks.
  • domain assumption The optical-flow network from DROID-SLAM generalizes sufficiently to UZH-FPV despite being trained on TartanAir
    The network is pre-trained on TartanAir and applied to UZH-FPV without fine-tuning. The paper treats this as acceptable but the domain shift is significant (simulated to real).
  • ad hoc to paper Comparing ORB-SLAM3-OF-U against DROID-SLAM/DPVO isolates the effect of recurrent architecture
    This is the core methodological assumption. The systems differ in multiple ways beyond recurrence (backend type, loop closure, keyframe management), which confounds the isolation.

pith-pipeline@v1.1.0-glm · 22037 in / 3200 out tokens · 346443 ms · 2026-07-08T19:05:29.448702+00:00 · methodology

0 comments
read the original abstract

Visual SLAM is a well-established technology utilized in a wide range of real-world applications. However, its performance still degrades under challenging visual conditions, such as low texture, severe motion blur, and poor illumination. Systems based on deep learning outperform classical geometry-based ones and achieve state-of-the-art results by combining learned 2D data association and uncertainty with differentiable geometric optimization in recurrent architectures. Still, it remains unclear exactly which components are fundamentally responsible for this success. In this paper, we ask: Is the superior performance of deep learning-based systems driven primarily by learned 2D data association, the combination of learned 2D data association and uncertainty, or the recurrent architecture itself? We investigate this question empirically by conducting a controlled study. Our findings reveal that the success of DL-based V-SLAM systems hinges on learned 2D data association and uncertainty rather than their recurrent architecture, underscoring the necessity of learning-based paradigms for the design of these components. Upon acceptance, the code will be released as open source.

Figures

Figures reproduced from arXiv: 2607.06023 by Davide Scaramuzza, Giovanni Cioffi.

Figure 1
Figure 1. Figure 1: Uncovering why deep learning improves visual SLAM. (a) Classical [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Diagram of the classical ORB-SLAM3 architecture (top), [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 2
Figure 2. Figure 2: 3) Learned Uncertainty in Bundle Adjustment: In the stan￾dard ORB-SLAM3 system, the weights of the reprojection errors in the motion-only BA are assigned based on the image pyramid level at which a feature is extracted (1.0 for the original image resolution, lower weight for higher pyramid levels with lower resolutions). ORB-SLAM3 uses eight pyramid levels where the image resolution is reduced by a factor … view at source ↗
Figure 3
Figure 3. Figure 3: Top-down trajectories for eight sequences from the TartanAir dataset. Ground truth is compared against trajectories from ORB-SLAM3-OF-U and the [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Analysis of optical flow magnitude between consecutive frames. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Optical flow prediction confidence analysis for TartanAir sequence [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Top-down trajectories for four sequences from the UZH-FPV dataset. Ground truth is compared against trajectories from ORB-SLAM3-OF-U and the [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Optical flow prediction confidence analysis for UZH-FPV sequence [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references · 56 canonical work pages · 4 internal anchors

  1. [1]

    Present and future of SLAM in extreme environments: The DARPA SubT challenge.IEEE Trans

    Kamak Ebadi, Lukas Bernreiter, Harel Biggie, et al. Present and future of SLAM in extreme environments: The DARPA SubT challenge.IEEE Trans. Robot., 2024

    work page 2024

  2. [2]

    Computer vision for autonomous vehicles: Prob- lems, datasets and state of the art.Foundations and Trends in Computer Graphics and Vision, 2020

    Joel Janai, Fatma G ¨uney, Aseem Behl, and Andreas Geiger. Computer vision for autonomous vehicles: Prob- lems, datasets and state of the art.Foundations and Trends in Computer Graphics and Vision, 2020

    work page 2020

  3. [3]

    Project Aria: A New Tool for Egocentric Multi-Modal AI Research

    Jakob Engel, Kiran Somasundaram, Michael Goesele, et al. Project aria: A new tool for egocentric multi- modal ai research.arXiv preprint arXiv:2308.13561,

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    Sight guide demonstrates robotics-inspired vision assistance at the cybathlon.Science Robotics, 2025

    Patrick Pfreundschuh, Giovanni Cioffi, Cornelius von Einem, et al. Sight guide demonstrates robotics-inspired vision assistance at the cybathlon.Science Robotics, 2025

    work page 2025

  5. [5]

    Ac- quiring submillimeter-accurate multi-task vision datasets for computer-assisted orthopedic surgery.Int

    Emma Most, Jonas Hein, Fr ´ed´eric Giraud, et al. Ac- quiring submillimeter-accurate multi-task vision datasets for computer-assisted orthopedic surgery.Int. Journal of Computer Assisted Radiology and Surgery, 2025

    work page 2025

  6. [6]

    From Localization and Mapping to Spatial Intelligence

    Luca Carlone, Ayoung Kim, Timothy Barfoot, Daniel Cremers, and Frank Dellaert, editors.SLAM Handbook. From Localization and Mapping to Spatial Intelligence. Cambridge University Press, 2026

    work page 2026

  7. [7]

    Tartanair: A dataset to push the limits of visual slam

    Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. Tartanair: A dataset to push the limits of visual slam. InIEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS), 2020

    work page 2020

  8. [8]

    Are we ready for autonomous drone racing? the uzh-fpv drone racing dataset

    Jeffrey Delmerico, Titus Cieslewski, Henri Rebecq, Matthias Faessler, and Davide Scaramuzza. Are we ready for autonomous drone racing? the uzh-fpv drone racing dataset. InIEEE Int. Conf. Robot. Autom. (ICRA), 2019

    work page 2019

  9. [9]

    Dsec: A stereo event camera dataset for driving scenarios.IEEE Robot

    Mathias Gehrig, Willem Aarents, Daniel Gehrig, and Davide Scaramuzza. Dsec: A stereo event camera dataset for driving scenarios.IEEE Robot. Autom. Lett., 2021

    work page 2021

  10. [10]

    The hilti slam challenge dataset.IEEE Robot

    Michael Helmberger, Kristian Morin, Beda Berner, Nitish Kumar, Giovanni Cioffi, and Davide Scaramuzza. The hilti slam challenge dataset.IEEE Robot. Autom. Lett., 2022. 12

    work page 2022

  11. [11]

    Palazzolo, J

    E. Palazzolo, J. Behley, P. Lottes, P. Gigu `ere, and C. Stachniss. ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals. InIEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS), 2019

    work page 2019

  12. [12]

    Deepvo: Towards end-to-end visual odometry with deep recurrent convolutional neural networks

    Sen Wang, Ronald Clark, Hongkai Wen, and Niki Trigoni. Deepvo: Towards end-to-end visual odometry with deep recurrent convolutional neural networks. In IEEE Int. Conf. Robot. Autom. (ICRA), 2017

    work page 2017

  13. [13]

    Tar- tanvo: A generalizable learning-based vo

    Wenshan Wang, Yaoyu Hu, and Sebastian Scherer. Tar- tanvo: A generalizable learning-based vo. InConf. on Robot. Learning (CoRL), 2021

    work page 2021

  14. [14]

    Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras.Advances in Neural Information Processing Systems, 34:16558– 16569, 2021

    Zachary Teed and Jia Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras.Advances in Neural Information Processing Systems, 34:16558– 16569, 2021

    work page 2021

  15. [15]

    Deep Patch Visual Odometry

    Zachary Teed, Lahav Lipson, and Jia Deng. Deep patch visual odometry.arXiv preprint arXiv:2208.04726, 2022. URL https://arxiv.org/abs/2208.04726. Retrieved on Jan. 4th, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  16. [16]

    Deep patch visual slam

    Lahav Lipson, Zachary Teed, and Jia Deng. Deep patch visual slam. InEur. Conf. Comput. Vis. (ECCV), 2024

    work page 2024

  17. [17]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. InEur. Conf. Comput. Vis. (ECCV), 2020

    work page 2020

  18. [18]

    Nice-slam: Neural implicit scalable encoding for slam

    Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin R Oswald, and Marc Pollefeys. Nice-slam: Neural implicit scalable encoding for slam. InIEEE Conf. Comput. Vis. Pattern Recog. (CVPR), 2022

    work page 2022

  19. [19]

    Nicer-slam: Neural implicit scene encoding for rgb slam

    Zihan Zhu, Songyou Peng, Viktor Larsson, Zhaopeng Cui, Martin R Oswald, Andreas Geiger, and Marc Polle- feys. Nicer-slam: Neural implicit scene encoding for rgb slam. InInt. Conf. on 3D Vision (3DV), 2024

    work page 2024

  20. [20]

    Gaussian splatting slam

    Hidenobu Matsuki, Riku Murai, Paul HJ Kelly, and Andrew J Davison. Gaussian splatting slam. InIEEE Conf. Comput. Vis. Pattern Recog. (CVPR), 2024

    work page 2024

  21. [21]

    Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splatting

    Vladimir Yugay, Yue Li, Theo Gevers, and Martin R Oswald. Gaussian-slam: Photo-realistic dense slam with gaussian splatting.arXiv preprint arXiv:2312.10070,

    work page internal anchor Pith review Pith/arXiv arXiv

  22. [22]

    Mast3r-slam: Real-time dense slam with 3d reconstruc- tion priors

    Riku Murai, Eric Dexheimer, and Andrew J Davison. Mast3r-slam: Real-time dense slam with 3d reconstruc- tion priors. InIEEE Conf. Comput. Vis. Pattern Recog. (CVPR), 2025

    work page 2025

  23. [23]

    Vggt-slam: Dense rgb slam optimized on the sl (4) manifold.Conf

    Dominic Maggio, Hyungtae Lim, and Luca Carlone. Vggt-slam: Dense rgb slam optimized on the sl (4) manifold.Conf. Neural Inf. Process. Syst. (NIPS), 2025

    work page 2025

  24. [24]

    Vggt-slam 2.0: Real-time dense feed-forward scene reconstruction.arXiv preprint arXiv:2601.19887, 2026

    Dominic Maggio and Luca Carlone. Vggt-slam 2.0: Real-time dense feed-forward scene reconstruction.arXiv preprint arXiv:2601.19887, 2026. Accessed 2026-04-20

    work page arXiv 2026

  25. [25]

    Mac-vo: Metrics-aware covari- ance for learning-based stereo visual odometry

    Yuheng Qiu, Yutian Chen, Zihao Zhang, Wenshan Wang, and Sebastian Scherer. Mac-vo: Metrics-aware covari- ance for learning-based stereo visual odometry. InIEEE Int. Conf. Robot. Autom. (ICRA), 2025

    work page 2025

  26. [26]

    Resilient odometry via hierarchical adaptation.Science Robotics, 2025

    Shibo Zhao, Sifan Zhou, Yuchen Zhang, Ji Zhang, Chen Wang, Wenshan Wang, and Sebastian Scherer. Resilient odometry via hierarchical adaptation.Science Robotics, 2025

    work page 2025

  27. [27]

    Factor graphs and gtsam: A hands-on introduction.Georgia Institute of Technology, Tech

    Frank Dellaert. Factor graphs and gtsam: A hands-on introduction.Georgia Institute of Technology, Tech. Rep., 2012

    work page 2012

  28. [28]

    A micro Lie theory for state estimation in robotics

    Joan Sol `a, Jeremie Deray, and Dinesh Atchuthan. A micro Lie theory for state estimation in robotics.arXiv preprint arXiv:1812.01537, 2018. Accessed 2026-04-20

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  29. [29]

    A tutorial on graph-based slam.IEEE Intel

    Giorgio Grisetti, Rainer K ¨ummerle, Cyrill Stachniss, and Wolfram Burgard. A tutorial on graph-based slam.IEEE Intel. Transportation Sys. Mag., 2011

    work page 2011

  30. [30]

    Orb-slam: A versatile and accurate monocular slam system.IEEE Trans

    Raul Mur-Artal, Jose Maria Martinez Montiel, and Juan D Tardos. Orb-slam: A versatile and accurate monocular slam system.IEEE Trans. Robot., 2015

    work page 2015

  31. [31]

    Orb-slam2: An open-source slam system for monocular, stereo, and rgb- d cameras.IEEE Trans

    Raul Mur-Artal and Juan D Tard ´os. Orb-slam2: An open-source slam system for monocular, stereo, and rgb- d cameras.IEEE Trans. Robot., 2017

    work page 2017

  32. [32]

    Orb-slam3: An accurate open-source library for visual, visual–inertial, and multimap slam.IEEE Trans

    Carlos Campos, Richard Elvira, Juan J G ´omez Rodr´ıguez, Jos ´e MM Montiel, and Juan D Tard ´os. Orb-slam3: An accurate open-source library for visual, visual–inertial, and multimap slam.IEEE Trans. Robot., 2021

    work page 2021

  33. [33]

    Machine learning for high-speed corner detection

    Edward Rosten and Tom Drummond. Machine learning for high-speed corner detection. InEur. Conf. Comput. Vis. (ECCV), 2006

    work page 2006

  34. [34]

    Orb: An efficient alternative to sift or surf

    Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. Orb: An efficient alternative to sift or surf. In Int. Conf. Comput. Vis. (ICCV), 2011

    work page 2011

  35. [35]

    Direct sparse odometry.IEEE Trans

    Jakob Engel, Vladlen Koltun, and Daniel Cremers. Direct sparse odometry.IEEE Trans. Pattern Anal. Mach. Intell., 2017

    work page 2017

  36. [36]

    Svo: Semidirect visual odometry for monocular and multicam- era systems.IEEE Trans

    Christian Forster, Zichao Zhang, Michael Gassner, Manuel Werlberger, and Davide Scaramuzza. Svo: Semidirect visual odometry for monocular and multicam- era systems.IEEE Trans. Robot., 2016

    work page 2016

  37. [37]

    Monoslam: Real-time single camera slam

    Andrew J Davison, Ian D Reid, Nicholas D Molton, and Olivier Stasse. Monoslam: Real-time single camera slam. IEEE Trans. Pattern Anal. Mach. Intell., 2007

    work page 2007

  38. [38]

    A multi-state constraint kalman filter for vision-aided iner- tial navigation

    Anastasios I Mourikis and Stergios I Roumeliotis. A multi-state constraint kalman filter for vision-aided iner- tial navigation. InIEEE Int. Conf. Robot. Autom. (ICRA), 2007

    work page 2007

  39. [39]

    Keyframe-based visual–inertial odometry using nonlinear optimization

    Stefan Leutenegger, Simon Lynen, Michael Bosse, Roland Siegwart, and Paul Furgale. Keyframe-based visual–inertial odometry using nonlinear optimization. Int. J. Robot. Research, 2015

    work page 2015

  40. [40]

    Vins-mono: A robust and versatile monocular visual-inertial state estimator.IEEE Trans

    Tong Qin, Peiliang Li, and Shaojie Shen. Vins-mono: A robust and versatile monocular visual-inertial state estimator.IEEE Trans. Robot., 2018

    work page 2018

  41. [41]

    Visual slam: why filter?Image and Vision Comput- ing, 2012

    Hauke Strasdat, Jos ´e MM Montiel, and Andrew J Davi- son. Visual slam: why filter?Image and Vision Comput- ing, 2012

    work page 2012

  42. [42]

    Tard ´os, Javier Civera, Margarita Chli, Stefan Leutenegger, Frank Dellaert, and Daniel Cremers

    Jakob Engel, Juan D. Tard ´os, Javier Civera, Margarita Chli, Stefan Leutenegger, Frank Dellaert, and Daniel Cremers. Visual SLAM. In Luca Carlone, Ayoung Kim, Timothy Barfoot, Daniel Cremers, and Frank Dellaert, editors,SLAM Handbook. From Localization and Map- ping to Spatial Intelligence. Cambridge University Press, 2026

    work page 2026

  43. [43]

    D3vo: Deep depth, deep pose and deep un- certainty for monocular visual odometry

    Nan Yang, Lukas von Stumberg, Rui Wang, and Daniel 13 Cremers. D3vo: Deep depth, deep pose and deep un- certainty for monocular visual odometry. InIEEE Conf. Comput. Vis. Pattern Recog. (CVPR), 2020

    work page 2020

  44. [44]

    imap: Implicit mapping and positioning in real- time

    Edgar Sucar, Shikun Liu, Joseph Ortiz, and Andrew J Davison. imap: Implicit mapping and positioning in real- time. InInt. Conf. Comput. Vis. (ICCV), 2021

    work page 2021

  45. [45]

    Nerf: Representing scenes as neural radiance fields for view synthesis.Communications of the ACM, 2021

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis.Communications of the ACM, 2021

    work page 2021

  46. [46]

    Grounding image matching in 3d with mast3r

    Vincent Leroy, Yohann Cabon, and J ´erˆome Revaud. Grounding image matching in 3d with mast3r. InEur. Conf. Comput. Vis. (ECCV), 2024

    work page 2024

  47. [47]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. InIEEE Conf. Comput. Vis. Pattern Recog. (CVPR), 2025

    work page 2025

  48. [48]

    Bags of binary words for fast place recognition in image sequences

    Dorian G ´alvez-L´opez and Juan D Tardos. Bags of binary words for fast place recognition in image sequences. IEEE Trans. Robot., 2012

    work page 2012

  49. [49]

    Least-squares estimation of transfor- mation parameters between two point patterns.IEEE Trans

    Shinji Umeyama. Least-squares estimation of transfor- mation parameters between two point patterns.IEEE Trans. Pattern Anal. Mach. Intell., 2002

    work page 2002

  50. [50]

    A tutorial on quantitative trajectory evaluation for visual(-inertial) odometry

    Zichao Zhang and Davide Scaramuzza. A tutorial on quantitative trajectory evaluation for visual(-inertial) odometry. InIEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS), 2018

    work page 2018

  51. [51]

    Mapanything: Universal feed-forward metric 3d reconstruction

    Nikhil Keetha, Norman M ¨uller, Johannes Sch ¨onberger, Lorenzo Porzi, Yuchen Zhang, Tobias Fischer, Arno Knapitsch, Duncan Zauss, Ethan Weber, Nelson Antunes, et al. Mapanything: Universal feed-forward metric 3d reconstruction. InInt. Conf. on 3D Vision (3DV), 2026

    work page 2026

  52. [52]

    A benchmark for the evaluation of rgb-d slam systems

    J ¨urgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. A benchmark for the evaluation of rgb-d slam systems. InIEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS), 2012

    work page 2012

  53. [53]

    The euroc micro aerial vehicle datasets.Int

    Michael Burri, Janosch Nikolic, Pascal Gohl, Thomas Schneider, Joern Rehder, Sammy Omari, Markus W Achtelik, and Roland Siegwart. The euroc micro aerial vehicle datasets.Int. J. Robot. Research, 2016

    work page 2016

  54. [54]

    Continuous-time vs

    Giovanni Cioffi, Titus Cieslewski, and Davide Scara- muzza. Continuous-time vs. discrete-time vision-based slam: A comparative study.IEEE Robot. Autom. Lett., 2022

    work page 2022

  55. [55]

    Reinforcement learning meets visual odometry

    Nico Messikommer, Giovanni Cioffi, Mathias Gehrig, and Davide Scaramuzza. Reinforcement learning meets visual odometry. InEur. Conf. Comput. Vis. (ECCV), 2024

    work page 2024

  56. [56]

    Image-conditioned adaptive parameter tuning for visual odometry frontends.arXiv preprint arXiv:2603.21785, 2026

    Simone Nascivera, Leonard Bauersfeld, Jeff Delaune, and Davide Scaramuzza. Image-conditioned adaptive parameter tuning for visual odometry frontends.arXiv preprint arXiv:2603.21785, 2026. Accessed 2026-04-15

    work page arXiv 2026