REVIEW 4 major objections 4 minor 38 references
Deep Visual Odometry for Stereo Event Cameras
T0 review · 4 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read A stereo event-camera odometry system that estimates pose in metric scale in real time, including nighttime HDR scenes, by reusing a monocular network's learned patch features for left-right matching.
desk verdict A credible, well-evaluated stereo extension of DEVO that earns its main claims, with a couple of undisclosed hyperparameters that a referee should pin down. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing piece is the static stereo association in Eq. (2): given a patch on the first-level matching feature map of the left frame, the system computes the dot product with patches on the right feature map at all candidate horizontal disparities and takes the argmax. Because the voxel grids are stereo rectified, the search is one-dimensional, and because the feature encoder was already computed for temporal tracking, the extra computation is a few dot products. This disparity is converted to inverse depth, and the patch graph feeds both temporal and stereo residuals into a bundle adjustment solved by two Gauss-Newton iterations with the Schur complement.
What would settle it
Run the released system on a rectified stereo event sequence with close objects whose true disparity exceeds the assumed search range (e.g., a 0.5 m object with a 60 cm baseline), and compare the argmax disparity from Eq. (2) to LiDAR or structure-from-motion depth. If a large fraction of patches choose the wrong disparity while the same features track correctly over time, the static stereo transfer assumption is refuted.
Extended reading notes
Core claim
Starting from DEVO, a monocular deep visual odometry system that tracks sparse event patches through recurrent optical flow, the authors add a static stereo association step. For each tracked patch in the left frame, they search the rectified right feature map along the epipolar line for the disparity that maximizes the dot product of the patch features, then triangulate that disparity into inverse depth. These metric depths are inserted into the existing sliding-window patch graph and refined together with camera poses in a tightly coupled bundle adjustment. The result is a system, Stereo-DEVO, that preserves DEVO's learned generalization while recovering the metric scale that a monocular c
Load-bearing premise
The feature encoder, trained only for temporal optical flow on one event camera, also produces patches whose dot product reliably finds the correct left-right stereo match within the chosen disparity range—if that transfer fails, the depth and the metric scale collapse.
Editorial extensions
If this is right
- Event-only visual odometry can recover metric scale purely from stereo geometry, without IMU fusion or image frames.
- The stereo depth module adds negligible runtime, so a monocular deep VO architecture can be upgraded to a stereo system without a second network or dense depth prediction.
- The system runs online at VGA resolution (10 Hz on a desktop GPU), unlike the offline monocular DEVO baseline.
- A single network trained for optical flow transfers to stereo matching across at least five datasets covering resolutions from 240x180 to 1280x720.
- Nighttime HDR driving scenes, where direct photometric methods fail due to flicker and noise, become tractable for event-based odometry.
Reading between the lines
- The dot-product stereo matcher is an implicit assumption that the optical-flow features are also disparity-discriminative; a natural extension would fine-tune the encoder with a stereo loss or build a learned cost volume, which could extend the method to larger baselines and closer objects.
- The same pattern—reusing a network's existing feature pyramid for a geometric matching task with near-zero extra cost—could apply to other patch-based VO/SLAM systems to recover metric scale from any calibrated multi-camera rig.
- Because the disparity search range is not reported per dataset, the method's stated generality implicitly depends on that range being set wide enough; an adaptive estimate from the camera baseline and a prior depth range would be a testable robustness improvement.
- The real-time claim is platform-dependent (10 Hz on a desktop, about 5 Hz on a laptop, slower on an embedded 25 W device); a lightweight or distilled feature encoder would be a useful extension for embedded deployment.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents Stereo-DEVO, a stereo extension of DEVO for event-based visual odometry. The key idea is to reuse DEVO's patch selector, feature encoder, and recurrent update operator for temporal association, and to add a static stereo association module that matches left patches to right features by maximizing dot-product similarity along a rectified epipolar search range (Eq. 2), yielding disparity and metric depth without an additional network. These static associations are fused with temporal associations in a sliding-window bundle adjustment to estimate metric-scale 6-DoF poses. The system is implemented online in ROS, reported to run in real time at VGA resolution on a desktop, and is evaluated on five public datasets plus two self-collected sequences against six baselines, achieving best or second-best trajectory accuracy on most sequences.
Significance. If the results hold, the paper makes a convincing case that learned temporal event features can be transferred to static stereo matching, providing a lightweight route from monocular, scale-ambiguous deep VO to metric stereo VO. Strengths include the broad empirical evaluation (five datasets, six baselines), the absence of network fine-tuning, the real-time implementation, code release, and the runtime profiling. The metric scale is anchored by the stereo baseline and rectified geometry rather than fitted to the target trajectory, so the reasoning is not circular. The main unresolved risks concern reproducibility due to stochastic patch selection, the undisclosed search-range hyperparameter D, and the absence of direct depth validation for the new stereo association.
major comments (4)
- [Sec. III-B, Eq. (2), Sec. IV] The maximum disparity D in Eq. (2) is never specified per dataset or sequence. Table I shows baselines from 10 to 60 cm and resolutions from 240x180 to 1280x720, so the feasible disparity interval varies substantially. Without the per-dataset values of D and a sensitivity analysis, Eq. (2) is not reproducible and the central metric-scale depth mechanism is under-specified. Please report D for each dataset and validate disparity/depth accuracy against available ground truth, e.g., DSEC LiDAR depth or TUM-VIE depth. Trajectory ATE alone is an indirect test of the depth channel and can hide systematic depth bias that the BA partially absorbs.
- [Sec. III-B, Eq. (2)] The search in Eq. (2) is restricted to 1 <= delta <= D, excluding the zero-disparity hypothesis. For a rectified stereo pair with a finite baseline, distant scene points have disparity below 1 pixel; assigning them a minimum disparity of 1 systematically biases inverse depth and can affect metric scale for far content. This is especially relevant to the large-scale DSEC city sequences. Please justify this choice, include delta = 0 with appropriate matching/ambiguity handling, or at least quantify the effect of excluding it.
- [Sec. IV-B, Tables II and III] The patch selector uses pooled multinomial sampling (Sec. III-B), so the system is stochastic, yet Tables II-IV report single-run numbers. The discrepancy for the DSEC city09d sequence is indicative: our method's ATE is 625.81 cm in Table II but 564.33 cm in Table III. This is either run-to-run variability or a typo; as printed it is unexplained. Please report mean and standard deviation over multiple seeds (or a fixed-seed protocol) and reconcile the values in all tables.
- [Sec. IV-E] The ablation does not directly test the transfer assumption underlying Eq. (2): features trained for monocular temporal optical flow are used for cross-camera static matching. The baselines 'Update', 'ZNCC', and 'Ours' all operate on the same feature representation family, and all metrics are trajectory errors rather than measures of whether the estimated stereo correspondences are correct. A direct disparity or depth error evaluation on at least one dataset is needed to support the claimed static-stereo mechanism and to rule out a systematic bias that the tightly coupled BA may partially absorb.
minor comments (4)
- [Sec. IV-C] The statement 'mean absolute trajectory error of 4.52 m over a 2,510 m trajectory' should clarify that this is the mean of per-segment ATE values, not the ATE of a single concatenated trajectory.
- [Tables II and III] The tables are hard to read because rows and columns are not cleanly separated, e.g., 'box7.20 5.80'. Add visual separators and consistent units. Also fix small typos such as 'stereo stereo' in Sec. III-B, 'employe' in Sec. IV-B, and 'odometrt' in Sec. II.
- [Sec. III-C] The keyframe-removal criterion is described as 'below than a certain threshold' without reporting the threshold value. Please state the value or provide a reference.
- [Sec. IV-F, Table V] For the real-time claim, give the end-to-end latency including both the pre-processing node and the tracking node. The current 'Subtotal' is tracking-only; adding pre-processing gives a more complete picture.
Circularity Check
No significant circularity: metric-scale pose anchoring comes from the external stereo baseline and rectified geometry, not from a fitted quantity or a self-citation chain.
full rationale
The derivation chain is: (1) Eq. (2) selects a disparity by maximizing the dot product of DEVO feature patches along a rectified epipolar line; (2) triangulation converts that disparity to inverse depth using known baseline and camera parameters; (3) Eq. (4) tightly couples these static stereo associations with temporal associations in bundle adjustment. Metric scale therefore enters through the physical stereo baseline and calibration, which are external inputs, not through parameters fitted to the target trajectories. The paper states that all experiments are conducted without modifying or fine-tuning the network models, and the ablation (Tab. IV) compares different depth sources against the same trajectory metric rather than fitting the metric. The only self-reference is the authors' ESVO2 used as a baseline and as the source of the hnu campus sequence; this is a benchmark choice and is not load-bearing for any claimed derivation. DEVO itself is an external prior system, so building on it is not a self-citation chain. The under-reported hyperparameter D and the untested transfer of temporal optical-flow features to cross-camera stereo matching are legitimate correctness and robustness concerns, but they are not instances where the output is equivalent to the input by construction. Accordingly, the paper's central claim is self-contained against external benchmarks, and the circularity score is low.
Assumptions & free parameters
free parameters (2)
- Maximum disparity D =
not reported per dataset
- Keyframe removal optical-flow threshold =
not reported
assumptions (4)
- domain assumption DEVO's pretrained network generalizes to all evaluation datasets without fine-tuning.
- domain assumption The left and right voxel grids are rectified and synchronized.
- ad hoc to paper Learned features from monocular optical flow are discriminative for stereo matching.
- domain assumption LIO-SAM output is a reliable ground truth for DSEC.
Cite this review
Pith. "Pith review of Deep Visual Odometry for Stereo Event Cameras." pith.science (2026). https://pith.science/paper/JB7PFZMG
@misc{pith2026250908235,
author = {Pith},
title = {Pith review of: Deep Visual Odometry for Stereo Event Cameras},
year = {2026},
howpublished = {\url{https://pith.science/paper/JB7PFZMG}},
note = {Machine review of arXiv:2509.08235}
}
read the original abstract
Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, offering the potential to handle state estimation tasks involving motion blur and high dynamic range (HDR) illumination conditions. However, the versatility of event-based visual odometry (VO) relying on handcrafted data association (either direct or indirect methods) is still unreliable, especially in field robot applications under low-light HDR conditions, where the dynamic range can be enormous and the signal-to-noise ratio is spatially-and-temporally varying. Leveraging deep neural networks offers new possibilities for overcoming these challenges. In this paper, we propose a learning-based stereo event visual odometry. Building upon Deep Event Visual Odometry (DEVO), our system (called Stereo-DEVO) introduces a novel and efficient static-stereo association strategy for sparse depth estimation with almost no additional computational burden. By integrating it into a tightly coupled bundle adjustment (BA) optimization scheme, and benefiting from the recurrent network's ability to perform accurate optical flow estimation through voxel-based event representations to establish reliable patch associations, our system achieves high-precision pose estimation in metric scale. In contrast to the offline performance of DEVO, our system can process event data of \zs{Video Graphics Array} (VGA) resolution in real time. Extensive evaluations on multiple public real-world datasets and self-collected data justify our system's versatility, demonstrating superior performance compared to state-of-the-art event-based VO methods. More importantly, our system achieves stable pose estimation even in large-scale nighttime HDR scenarios.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
DSEC: A stereo event camera dataset for driving scenarios,
M. Gehrig, W. Aarents, D. Gehrig, and D. Scaramuzza, “DSEC: A stereo event camera dataset for driving scenarios,”IEEE Robot. Autom. Lett., vol. 6, no. 3, pp. 4947–4954, 2021
2021
-
[2]
LIO-SAM: Tightly-coupled lidar inertial odometry via smoothing and mapping,
T. Shan, B. Englot, D. Meyers, W. Wang, C. Ratti, and D. Rus, “LIO-SAM: Tightly-coupled lidar inertial odometry via smoothing and mapping,” inIEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS), 2020, pp. 5135–5142
work page 2020
-
[3]
Esvo2: Direct visual-inertial odometry with stereo event cameras,
J. Niu, S. Zhong, X. Lu, S. Shen, G. Gallego, and Y . Zhou, “Esvo2: Direct visual-inertial odometry with stereo event cameras,”IEEE Trans- actions on Robotics, 2025
work page 2025
-
[4]
A 128×128 120 dB 15µs latency asynchronous temporal contrast vision sensor,
P. Lichtsteiner, C. Posch, and T. Delbruck, “A 128×128 120 dB 15µs latency asynchronous temporal contrast vision sensor,”IEEE J. Solid- State Circuits, vol. 43, no. 2, pp. 566–576, 2008
2008
-
[5]
Event-based, 6-DOF camera tracking from photometric depth maps,
G. Gallego, J. E. A. Lund, E. Mueggler, H. Rebecq, T. Delbruck, and D. Scaramuzza, “Event-based, 6-DOF camera tracking from photometric depth maps,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 10, pp. 2402–2412, Oct. 2018
2018
-
[6]
Continuous- time visual-inertial odometry for event cameras,
E. Mueggler, G. Gallego, H. Rebecq, and D. Scaramuzza, “Continuous- time visual-inertial odometry for event cameras,”IEEE Trans. Robot., vol. 34, no. 6, pp. 1425–1440, Dec. 2018
work page 2018
-
[7]
Asynchronous blob tracker for event cameras,
Z. Wang, T. Molloy, P. van Goor, and R. Mahony, “Asynchronous blob tracker for event cameras,”IEEE Trans. Robot., 2024
work page 2024
-
[8]
EMVS: Event-based multi-view stereo—3D reconstruction with an event camera in real-time,
H. Rebecq, G. Gallego, E. Mueggler, and D. Scaramuzza, “EMVS: Event-based multi-view stereo—3D reconstruction with an event camera in real-time,”Int. J. Comput. Vis., pp. 1–21, Nov. 2017
work page 2017
Show all 38 references
-
[9]
Ultimate SLAM? combining events, images, and IMU for robust visual SLAM in HDR and high speed scenarios,
A. Rosinol Vidal, H. Rebecq, T. Horstschaefer, and D. Scaramuzza, “Ultimate SLAM? combining events, images, and IMU for robust visual SLAM in HDR and high speed scenarios,”IEEE Robot. Autom. Lett., vol. 3, no. 2, pp. 994–1001, Apr. 2018
2018
-
[10]
Learning monocular dense depth from events,
D. G. Javier Hidalgo-Carrio and D. Scaramuzza, “Learning monocular dense depth from events,” in3D Vision (3DV), Nov. 2020, pp. 534–542
2020
-
[11]
CMax-SLAM: Event-based rotational-motion bundle adjustment and SLAM system using contrast maximization,
S. Guo and G. Gallego, “CMax-SLAM: Event-based rotational-motion bundle adjustment and SLAM system using contrast maximization,” IEEE Trans. Robot., vol. 40, pp. 2442–2461, 2024
2024
-
[12]
Deep event visual odometry,
S. Klenk, M. Motzet, L. Koestler, and D. Cremers, “Deep event visual odometry,” inInternational Conference on 3D Vision, 3DV 2024, Davos, Switzerland, March 18-21, 2024. IEEE, 2024, pp. 739–749
2024
-
[13]
Event-based stereo visual odometry,
Y . Zhou, G. Gallego, and S. Shen, “Event-based stereo visual odometry,” IEEE Transactions on Robotics, vol. 37, no. 5, pp. 1433–1450, 2021
2021
-
[14]
ESVIO: Event-based stereo visual inertial odometry,
P. Chen, W. Guan, and P. Lu, “ESVIO: Event-based stereo visual inertial odometry,”IEEE Robot. Autom. Lett., vol. 8, no. 6, pp. 3661–3668, 2023
2023
-
[15]
ES-PTAM: Event-based stereo parallel tracking and mapping,
S. Ghosh, V . Cavinato, and G. Gallego, “ES-PTAM: Event-based stereo parallel tracking and mapping,” inEur. Conf. Comput. Vis. Workshops (ECCVW), 2024
2024
-
[16]
A combined corner and edge detector,
C. Harris and M. Stephens, “A combined corner and edge detector,” in Proc. Fourth Alvey Vision Conf., vol. 15, 1988, pp. 147–151
1988
-
[17]
Machine learning for high-speed corner detection,
E. Rosten and T. Drummond, “Machine learning for high-speed corner detection,” inEur. Conf. Comput. Vis. (ECCV), 2006, pp. 430–443
2006
-
[18]
Asynchronous corner detection and tracking for event cameras in real time,
I. Alzugaray and M. Chli, “Asynchronous corner detection and tracking for event cameras in real time,”IEEE Robot. Autom. Lett., vol. 3, no. 4, pp. 3177–3184, Oct. 2018
2018
-
[19]
FA-Harris: A fast and asynchronous corner detector for event cameras,
R. Li, D. Shi, Y . Zhang, K. Li, and R. Li, “FA-Harris: A fast and asynchronous corner detector for event cameras,” inIEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS), 2019
2019
-
[20]
A robust o (n) solution to the perspective-n- point problem,
S. Li, C. Xu, and M. Xie, “A robust o (n) solution to the perspective-n- point problem,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 7, pp. 1444–1450, 2012
2012
-
[21]
Feature- based event stereo visual odometry,
A. Hadviger, I. Cvi ˇsi´c, I. Markovi´c, S. Vraˇzi´c, and I. Petrovi ´c, “Feature- based event stereo visual odometry,” inEur. Conf. Mobile Robots (ECMR), 2021, pp. 1–6
2021
-
[22]
A unifying contrast maximization framework for event cameras, with applications to motion, depth, and optical flow estimation,
G. Gallego, H. Rebecq, and D. Scaramuzza, “A unifying contrast maximization framework for event cameras, with applications to motion, depth, and optical flow estimation,” inIEEE Conf. Comput. Vis. Pattern Recog. (CVPR), 2018, pp. 3867–3876
2018
-
[23]
Event-based visual inertial odometry,
A. Z. Zhu, N. Atanasov, and K. Daniilidis, “Event-based visual inertial odometry,” inIEEE Conf. Comput. Vis. Pattern Recog. (CVPR), 2017, pp. 5816–5824
2017
-
[24]
Keyframe-based visual-inertial SLAM using nonlinear optimization,
S. Leutenegger, P. Furgale, V . Rabaud, M. Chli, K. Konolige, and R. Siegwart, “Keyframe-based visual-inertial SLAM using nonlinear optimization,” inRobotics: Science and Systems (RSS), 2013
2013
-
[25]
Real-time 3D reconstruc- tion and 6-DoF tracking with an event camera,
H. Kim, S. Leutenegger, and A. J. Davison, “Real-time 3D reconstruc- tion and 6-DoF tracking with an event camera,” inEur. Conf. Comput. Vis. (ECCV), 2016, pp. 349–364
2016
-
[26]
EVO: A geometric approach to event-based 6-DOF parallel tracking and mapping in real-time,
H. Rebecq, T. Horstsch ¨afer, G. Gallego, and D. Scaramuzza, “EVO: A geometric approach to event-based 6-DOF parallel tracking and mapping in real-time,”IEEE Robot. Autom. Lett., vol. 2, no. 2, pp. 593–600, 2017
2017
-
[27]
Multi-event-camera depth estimation and outlier rejection by refocused events fusion,
S. Ghosh and G. Gallego, “Multi-event-camera depth estimation and outlier rejection by refocused events fusion,”Adv. Intell. Syst., vol. 4, no. 12, p. 2200221, 2022
2022
-
[28]
IMU-aided event-based stereo visual odometry,
J. Niu, S. Zhong, and Y . Zhou, “IMU-aided event-based stereo visual odometry,” inIEEE Int. Conf. Robot. Autom. (ICRA), 2024, pp. 11 977– 11 983
2024
-
[29]
Unsupervised event- based learning of optical flow, depth, and egomotion,
A. Z. Zhu, L. Yuan, K. Chaney, and K. Daniilidis, “Unsupervised event- based learning of optical flow, depth, and egomotion,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 989–997
2019
-
[30]
Unsupervised learning of dense optical flow, depth and egomotion from sparse event data,
C. Ye, A. Mitrokhin, C. Ferm ¨uller, J. A. Yorke, and Y . Aloimonos, “Unsupervised learning of dense optical flow, depth and egomotion from sparse event data,”arXiv preprint arXiv:1809.08625, 2018
2018 arXiv
-
[31]
The multivehicle stereo event camera dataset: An event camera dataset for 3D perception,
A. Z. Zhu, D. Thakur, T. Ozaslan, B. Pfrommer, V . Kumar, and K. Daniilidis, “The multivehicle stereo event camera dataset: An event camera dataset for 3D perception,”IEEE Robot. Autom. Lett., vol. 3, no. 3, pp. 2032–2039, July 2018
-
[32]
Deep patch visual odometry,
Z. Teed, L. Lipson, and J. Deng, “Deep patch visual odometry,” Advances in Neural Information Processing Systems, vol. 36, 2024
2024
-
[33]
Deio: Deep event inertial odometry,
W. Guan, F. Lin, P. Chen, and P. Lu, “Deio: Deep event inertial odometry,”arXiv preprint arXiv:2411.03928, 2024
2024 arXiv
-
[34]
Semi-dense 3D reconstruction with a stereo event camera,
Y . Zhou, G. Gallego, H. Rebecq, L. Kneip, H. Li, and D. Scaramuzza, “Semi-dense 3D reconstruction with a stereo event camera,” inEur. Conf. Comput. Vis. (ECCV), 2018, pp. 242–258
2018
-
[35]
VECtor: A versatile event-centric benchmark for multi-sensor slam,
L. Gao, Y . Liang, J. Yang, S. Wu, C. Wang, J. Chen, and L. Kneip, “VECtor: A versatile event-centric benchmark for multi-sensor slam,” IEEE Robot. Autom. Lett., vol. 7, no. 3, pp. 8217–8224, 2022
2022
-
[36]
Tum-vie: The tum stereo visual-inertial event dataset,
S. Klenk, J. Chui, N. Demmel, and D. Cremers, “Tum-vie: The tum stereo visual-inertial event dataset,” inInternational Conference on Intelligent Robots and Systems (IROS), 2021
2021
-
[37]
A benchmark for the evaluation of RGB-D SLAM systems,
J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers, “A benchmark for the evaluation of RGB-D SLAM systems,” inIEEE/RSJ Int. Conf. Intell. Robot. Syst. (IROS), Oct. 2012
2012
-
[38]
evo: Python package for the evaluation of odometry and slam
M. Grupp, “evo: Python package for the evaluation of odometry and slam.” https://github.com/MichaelGrupp/evo, 2017
2017
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.