REVIEW 2 major objections 6 minor 1 cited by
MCVO: A Generic Visual Odometry for Arbitrarily Arranged Multi-Cameras
T0 review · 2 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Rigid multi-camera rigs can recover metric-scale motion without IMU or overlapping fields of view.
desk verdict Solid multi-camera VO with a genuinely new scale-by-trajectory-consistency idea, but the paper skips the Sim(3) alignment that its scale-estimation derivation depends on. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the per-camera scale factor $s_c$, a positive real number that converts each camera's monocular Structure-from-Motion translations into the shared body frame. The argument is carried by the pairwise trajectory-consistency residual $e(T^b_{i,t}, T^b_{j,t}) = T^b_{i,t} - T^b_{j,t}$, which is linear in the unknown scale pair once camera rotations and extrinsics are fixed; stacking these residuals over camera pairs and frames produces a sparse block matrix and a constrained least-squares problem solved with a standard sparse nonlinear optimizer. Around this core, the system uses a GPU-accelerated learned feature detector with Lucas-Kanade optical flow, a three-priority quad-tree feature selection for spatially uniform tracking, a multi-camera bundle adjustment with inverse-depth landmarks, and a Bag-of-Words loop closure that fuses features from all cameras.
What would settle it
Run only the initialization stage on a sequence with ground-truth metric poses and compare the recovered per-camera scales with the true scales. In a controlled version, take one camera's reconstruction and deliberately rotate and translate it by a random similarity transform before the least-squares solve; if the recovered scales change, the residual is not isolating pure scale and the missing gauge-alignment step matters.
Extended reading notes
Core claim
The discovery is that monocular scale ambiguity can be resolved across cameras by enforcing trajectory consistency, rather than by stereo overlap or an IMU. Each camera runs independent monocular Structure-from-Motion over a short sliding window, giving rotations $R^c_t$ and translations $T^c_t$ up to an unknown scalar per camera. After expressing each camera's trajectory in the shared body frame through the calibrated extrinsics, the body trajectory from camera $i$ and camera $j$ should coincide; the pairwise residual is $e(T^b_{i,t}, T^b_{j,t}) = [T^i_t,\,-T^j_t][s_i;\,s_j] + \theta$, where $\theta$ depends only on rotations and extrinsics. Solving the resulting sparse least-squares problem over all camera pairs and frames yields metric scale at initialization, and the backend bundle adjustment refines the scale continuously by adjusting inverse depths. The paper reports that even a fully non-overlapping three-camera setup achieves lower translation error than stereo baselines on several KITTI-360 sequences.
Load-bearing premise
The scale solve assumes each camera's independent monocular reconstruction already differs from the true shared body trajectory by only a single positive scale factor per camera, with no leftover rotation or translation ambiguity between reconstructions.
Editorial extensions
If this is right
- Metric scale is obtainable for any rigid multi-camera rig with known extrinsics, including configurations with zero field-of-view overlap and mixed pinhole and fisheye cameras.
- A fully non-overlapping three-camera setup on KITTI-360 reports lower translation error than the stereo baselines, so overlap is not a prerequisite for scale accuracy.
- Moving feature extraction to the GPU cuts CPU usage by roughly 47 percent relative to a conventional binary feature frontend on the tested sequence, freeing CPU for backend optimization.
- Fusing features from all cameras into a single Bag-of-Words model strengthens loop detection and pose graph correction relative to single-camera loop closure.
- Because scale is corrected periodically in the backend, scale drift as well as pose drift is bounded over long trajectories.
Reading between the lines
- The pairwise residual treats each camera's reconstruction as differing from the true body trajectory only by a scalar, but monocular Structure-from-Motion carries a full seven-degree-of-freedom similarity ambiguity; unless the arbitrary rotation and translation of each reconstruction are aligned before the least-squares solve, the recovered scale may absorb gauge misalignment. The paper does not s
- If extrinsic calibration or synchronization errors are present, they will leak into the per-camera scale factors; a natural extension is to estimate extrinsics jointly with the scales during initialization.
- The scale estimate depends on the body trajectory being distinguishable across cameras; a perfectly straight, constant-speed motion may leave the scale underdetermined, and a dedicated test on such a trajectory would clarify the failure mode.
- The fixed 50-frame scale-correction interval is a tunable design choice; varying it on a long sequence with slow scale drift would show how much of the accuracy gain comes from correction frequency versus the initial scale estimate.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. MCVO is a multi-camera visual odometry system designed for rigidly mounted cameras with arbitrary relative placement and no requirement for overlapping fields of view. The pipeline consists of a SuperPoint/LK optical-flow frontend with a 3-priority quad-tree feature selection, an initialization stage that reconstructs per-camera monocular SfM trajectories and estimates per-camera scale factors from cross-camera body-trajectory consistency, and a backend that performs bundle adjustment with periodic scale correction and multi-camera loop closure. Experiments on KITTI-360 and MultiCamData compare favorably with stereo ORB-SLAM3, VINS-Fusion, and MultiCamSLAM in ATE and scale drift on several sequences, including a non-overlapping three-camera configuration.
Significance. The paper addresses a practically relevant problem and the system-level pipeline is coherent. The public code release and the systematic comparisons across two multi-camera datasets, including non-overlapping configurations, are concrete strengths. If the scale-estimation derivation is completed by making the gauge-alignment step explicit, the proposed method would be a useful generic baseline for multi-camera VO without IMUs. As it stands, the theoretical justification for metric-scale estimation is incomplete because the derivation drops the 7-DoF similarity ambiguity of monocular SfM without explanation.
major comments (2)
- [III-C, Eqs. (2)-(3)] The derivation of the scale residual assumes that each camera's monocular SfM reconstruction differs from the true trajectory only by a per-camera positive scale factor, but monocular SfM output has a 7-DoF similarity gauge ambiguity (arbitrary rotation, translation, and scale). Equations (2)-(3) use R^c_t and T^c_t from independent SfM runs without describing how the arbitrary rotation and translation of each reconstruction are aligned to a common world frame before forming the residual in Eq. (3). Unless this Sim(3) alignment is performed, the term θ is gauge-dependent, the residual does not isolate s_c, and the least-squares solution of Eq. (5) is not a metric-scale estimate. Please specify the alignment procedure (e.g., Sim(3) alignment of camera trajectories to the principal camera's frame) or prove that the residual is invariant under the gauge degrees of freedom.
- [III-C, Eq. (5)] The optimization in Eq. (5) is called a constrained least-squares problem, but no constraints are listed, and the scale factors are described as "parameterized over Lie groups" without definition. Positive scalars are not a Lie group in the usual sense, and the parameterization and any positivity/regularization constraints should be stated explicitly. Also, the identifiability of s from the least-squares problem is not analyzed: for degenerate motions (e.g., pure rotation or constant-velocity straight motion) the matrix F in Eq. (4) may lose rank, and the paper should state conditions under which the scale solution is unique.
minor comments (6)
- [III-C, Eq. (2)] The extrinsic parameters (r^c, t^c) are used without defining whether they transform camera-to-body or body-to-camera; please state the convention.
- [III-C] The sentence "The scale estimation is formulated as a manifold-aware optimization problem within the ceres solver, with scale factor s parameterized over Lie groups" is unclear because a positive scalar is not a Lie group; please replace with a standard parameterization (e.g., log-scale) or remove the claim.
- [IV-B, Table I] The MultiCamData columns are labeled "Trans. Trans." without distinguishing translation from scale-drift percentage; please use separate labeled columns with units.
- [III-B] The three priority rules for the 3-priority quad-tree are listed, but the algorithm that combines them into the quad-tree selection is not specified; add pseudocode or a precise procedure.
- [IV-C, Table II] Table II reports "UsageCP U" and "UsageGP U" in an unreadable format, and the caption does not define these metrics; please format the table and define CPU/GPU usage.
- [IV-A] The evaluation metrics include "scale drift" but the paper does not define how the scale error percentage in Table I is computed; add a formula or reference to the metric definition.
Circularity Check
No circular derivation found: the scale estimate is solved from cross-camera consistency and checked against independent ground truth; the main rigor gap is an unstated Sim(3) alignment, which is a correctness issue, not circularity.
full rationale
The scale-estimation chain in Section III-C does not reduce to its inputs by construction. The per-camera scale factors s_c are unknowns solved from the least-squares residual in Eqs. (3)-(5), which enforces that the body-frame translations recovered from different cameras agree. The reported ATE/RPE/scale numbers are computed against KITTI-360 ground truth and the MultiCamData marker-based start-end pose discrepancy, i.e., external benchmarks rather than quantities used in the objective, so the evaluation is not a renaming of the fitted values. No load-bearing self-citation appears: reference [9] (He, Yu, Yang) is cited only for GPU front-end processing and for IMU-based scale approaches in the related-work discussion, and the paper's own equations carry the derivation. The one genuine weakness is that Section III-C assumes each camera's monocular SfM output differs from the shared body trajectory by a single scalar scale after applying the extrinsics, whereas monocular SfM carries a 7-DoF similarity gauge; the paper does not state how the rotation and translation gauges are aligned before Eq. (5) is solved. That is an omitted derivation step and a correctness risk, but it does not make the claimed output equivalent to an input: even with alignment supplied, the scale estimate remains an estimated quantity constrained by cross-camera rigidity and validated against independent ground truth. The paper also openly lists the extrinsic-calibration requirement as a limitation, which further indicates that the authors are not claiming scale from nothing. Overall, no circular step meets the bar of Eq. X = Eq. Y by construction or fitted parameter renamed as prediction.
Assumptions & free parameters
free parameters (4)
- Initialization parallax threshold =
30 pixels
- Initialization window size =
10 frames
- Scale correction interval =
every 50 frames
- 3-priority quad-tree feature budget =
not reported
assumptions (5)
- domain assumption All cameras are rigidly bundled with known intrinsic and extrinsic parameters.
- domain assumption Synchronized multi-camera video streams are available.
- ad hoc to paper Independent monocular SfM for each camera recovers poses that differ from the true body trajectory only by a per-camera positive scale factor.
- domain assumption Sufficient parallax and non-degenerate motion occur within the initialization window in every camera.
- standard math Ceres Levenberg-Marquardt optimization converges to a useful solution of the scale least-squares problem.
Cite this review
Pith. "Pith review of MCVO: A Generic Visual Odometry for Arbitrarily Arranged Multi-Cameras." pith.science (2026). https://pith.science/paper/INYCXBPG
@misc{pith2026241203146,
author = {Pith},
title = {Pith review of: MCVO: A Generic Visual Odometry for Arbitrarily Arranged Multi-Cameras},
year = {2026},
howpublished = {\url{https://pith.science/paper/INYCXBPG}},
note = {Machine review of arXiv:2412.03146}
}
read the original abstract
Making multi-camera visual SLAM systems easier to set up and more robust to the environment is attractive for vision robots. Existing monocular and binocular vision SLAM systems have narrow sensing Field-of-View (FoV), resulting in degenerated accuracy and limited robustness in textureless environments. Thus multi-camera SLAM systems are gaining attention because they can provide redundancy with much wider FoV. However, the usual arbitrary placement and orientation of multiple cameras make the pose scale estimation and system updating challenging. To address these problems, we propose a robust visual odometry system for rigidly-bundled arbitrarily-arranged multi-cameras, namely MCVO, which can achieve metric-scale state estimation with high flexibility in the cameras' arrangement. Specifically, we first design a learning-based feature tracking framework to shift the pressure of CPU processing of multiple video streams to GPU. Then we initialize the odometry system with the metric-scale poses under the rigid constraints between moving cameras. Finally, we fuse the features of the multi-cameras in the back-end to achieve robust pose estimation and online scale optimization. Additionally, multi-camera features help improve the loop detection for pose graph optimization. Experiments on KITTI-360 and MultiCamData datasets validate its robustness over arbitrarily arranged cameras. Compared with other stereo and multi-camera visual SLAM systems, our method obtains higher pose accuracy with better generalization ability. Our codes and online demos are available at https://github.com/JunhaoWang615/MCVO
Figures
Figures from the paper (5 more)
Forward citations
Cited by 1 Pith paper
-
Multi-LVI-SAM: A Robust LiDAR-Visual-Inertial Odometry for Multiple Fisheye Cameras
Multi-LVI-SAM fuses multiple fisheye cameras, LiDAR, and IMU using a unified panoramic feature model and an extrinsic compensation step, improving odometry accuracy on public benchmarks.
Reference graph
Works this paper leans on
-
[9]
Towards robust visual- inertial odometry with multiple non-overlapping monocular cameras,
Y . He, H. Yu, W. Yang, and S. Scherer, “Towards robust visual- inertial odometry with multiple non-overlapping monocular cameras,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 9452–9458
work page 2022
-
[1]
VINS-Mono: A robust and versa- tile monocular visual-inertial state estimator,
T. Qin, P. Li, and S. Shen, “VINS-Mono: A robust and versa- tile monocular visual-inertial state estimator,” IEEE Transactions on Robotics, vol. 34, no. 4, pp. 1004–1020, 2018
work page 2018
-
[2]
ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multimap slam,
C. Campos, R. Elvira, J. J. G. Rodr ´ıguez, J. M. Montiel, and J. D. Tard´os, “ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multimap slam,” IEEE Transactions on Robotics , vol. 37, no. 6, pp. 1874–1890, 2021
work page 2021
-
[3]
Multi-camera tracking and mapping for unmanned aerial vehicles in unstructured environments,
A. Harmat, M. Trentini, and I. Sharf, “Multi-camera tracking and mapping for unmanned aerial vehicles in unstructured environments,” Journal of Intelligent and Robotic Systems , vol. 78, p. 291–317, 2015
work page 2015
-
[4]
Towards robust visual odometry with a multi-camera system,
P. Liu, M. Geppert, L. Heng, T. Sattler, A. Geiger, and M. Pollefeys, “Towards robust visual odometry with a multi-camera system,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018, pp. 1154–1161
work page 2018
-
[5]
KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2D and 3D,
Y . Liao, J. Xie, and A. Geiger, “KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2D and 3D,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 3, pp. 3292–3310, 2022
work page 2022
-
[6]
Multicol-slam - a modular real-time multi- camera slam system,
S. Urban and S. Hinz, “Multicol-slam - a modular real-time multi- camera slam system,” in arXiv preprint arXiv:1610.07336 , 2016
arXiv 2016
-
[7]
Design and evaluation of a generic visual slam framework for multi-camera systems,
P. Kaveti, S. N. Vaidyanathan, A. T. Chelvan, and H. Singh, “Design and evaluation of a generic visual slam framework for multi-camera systems,” IEEE Robotics and Automation Letters , 2023
work page 2023
Show all 33 references
-
[8]
A robust multi- stereo visual-inertial odometry pipeline,
J. Jaekel, J. G. Mangelson, S. Scherer, and M. Kaess, “A robust multi- stereo visual-inertial odometry pipeline,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 4623– 4630
2020
-
[10]
Run your visual- inertial odometry on nvidia jetson: Benchmark tests on a micro aerial vehicle,
J. Jeon, S. Jung, E. Lee, D. Choi, and H. Myung, “Run your visual- inertial odometry on nvidia jetson: Benchmark tests on a micro aerial vehicle,” IEEE robotics and automation letters, vol. 6, no. 3, pp. 5332– 5339, 2021
2021
-
[11]
Faster than fast: Gpu- accelerated frontend for high-speed vio,
B. Nagy, P. Foehn, and D. Scaramuzza, “Faster than fast: Gpu- accelerated frontend for high-speed vio,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 4361– 4368
2020
-
[12]
DeepVO: Towards end- to-end visual odometry with deep recurrent convolutional neural net- works,
S. Wang, R. Clark, H. Wen, and N. Trigoni, “DeepVO: Towards end- to-end visual odometry with deep recurrent convolutional neural net- works,” in IEEE International Conference on Robotics and Automation (ICRA), 2017, pp. 2043–2050
2017
-
[13]
Flowdometry: An optical flow and deep learning based approach to visual odometry,
P. Muller and A. Savakis, “Flowdometry: An optical flow and deep learning based approach to visual odometry,” in IEEE Winter Confer- ence on Applications of Computer Vision (WACV), 2017, pp. 624–631
2017
-
[14]
Leveraging deep learning for visual odometry using optical flow,
T. Pandey, D. Pena, J. Byrne, and D. Moloney, “Leveraging deep learning for visual odometry using optical flow,” Sensors, vol. 21, no. 4, p. 1313, 2021
2021
-
[15]
Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,
M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM , vol. 24, no. 6, pp. 381–395, 1981
1981
-
[16]
Descriptor scoring for feature selection in real-time visual slam,
P. Laddha, O. J. Omer, G. S. Kalsi, D. K. Mandal, and S. Subramoney, “Descriptor scoring for feature selection in real-time visual slam,” in IEEE International Conference on Image Processing (ICIP), 2020, pp. 2601–2605
2020
-
[17]
Network uncertainty informed semantic feature selection for visual slam,
P. Ganti and S. L. Waslander, “Network uncertainty informed semantic feature selection for visual slam,” in16th Conference on Computer and Robot Vision (CRV), 2019, pp. 121–128
2019
-
[18]
Attention and anticipation in fast visual- inertial navigation,
L. Carlone and S. Karaman, “Attention and anticipation in fast visual- inertial navigation,” IEEE Transactions on Robotics , vol. 35, no. 1, pp. 1–20, 2018
2018
-
[19]
Good feature matching: Toward accurate, robust vo/vslam with low latency,
Y . Zhao and P. A. Vela, “Good feature matching: Toward accurate, robust vo/vslam with low latency,” IEEE Transactions on Robotics , vol. 36, no. 3, pp. 657–675, 2020
2020
-
[20]
Fusing monocular information in multicamera SLAM,
J. Sola, A. Monin, M. Devy, and T. Vidal-Calleja, “Fusing monocular information in multicamera SLAM,” IEEE Transactions on Robotics , vol. 24, no. 5, pp. 958–968, 2008
2008
-
[21]
Parallel tracking and mapping for small ar workspaces,
G. Klein and D. Murray, “Parallel tracking and mapping for small ar workspaces,” in IEEE and ACM International Symposium on Mixed and Augmented Reality , 2007, pp. 225–234
2007
-
[22]
Multi-camera parallel tracking and mapping with non-overlapping fields of view,
M. J. Tribou, A. Harmat, D. W. Wang, I. Sharf, and S. L. Waslander, “Multi-camera parallel tracking and mapping with non-overlapping fields of view,” International Journal of Robotics Research , vol. 34, p. 1480–1500, 2015
2015
-
[23]
Self-calibration and visual slam with a multi-camera system on a micro aerial vehicle,
L. Heng, G. H. Lee, and M. Pollefeys, “Self-calibration and visual slam with a multi-camera system on a micro aerial vehicle,” Au- tonomous robots, vol. 39, no. 3, pp. 259–277, 2015
2015
-
[24]
Project autovision: Lo- calization and 3d scene perception for an autonomous vehicle with a multi-camera system,
L. Heng, B. Choi, Z. Cui, M. Geppert, S. Hu, B. Kuan, P. Liu, R. Nguyen, Y . C. Yeo, A. Geiger, et al. , “Project autovision: Lo- calization and 3d scene perception for an autonomous vehicle with a multi-camera system,” in IEEE International Conference on Robotics and Automati...
2019
-
[25]
Pose estimation for multi-camera systems,
C. Zhao, B. Fan, J. Hu, L. Tian, Z. Zhang, S. Li, and Q. Pan, “Pose estimation for multi-camera systems,” in IEEE International Conference on Unmanned Systems (ICUS) , 2017, pp. 533–538
2017
-
[26]
Multi-camera collaborative depth prediction via consistent structure estimation,
J. Xu, X. Liu, Y . Bai, J. Jiang, K. Wang, X. Chen, and X. Ji, “Multi-camera collaborative depth prediction via consistent structure estimation,” in 30th ACM International Conference on Multimedia , 2022, pp. 2730–2738
2022
-
[27]
Pano-sfmlearner: Self- supervised multi-task learning of depth and semantics in panoramic videos,
M. Liu, S. Wang, Y . Guo, Y . He, and H. Xue, “Pano-sfmlearner: Self- supervised multi-task learning of depth and semantics in panoramic videos,” IEEE Signal Processing Letters , vol. 28, pp. 832–836, 2021
2021
-
[28]
Panoramic depth estimation via supervised and unsupervised learning in indoor scenes,
K. Zhou, K. Yang, and K. Wang, “Panoramic depth estimation via supervised and unsupervised learning in indoor scenes,” Applied optics, vol. 60, no. 26, pp. 8188–8197, 2021
2021
-
[29]
Megadepth: Learning single-view depth predic- tion from internet photos,
Z. Li and N. Snavely, “Megadepth: Learning single-view depth predic- tion from internet photos,” in IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 2041–2050
2018
-
[30]
Superpoint: Self- supervised interest point detection and description,
D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superpoint: Self- supervised interest point detection and description,” in IEEE Confer- ence on Computer Vision and Pattern Recognition Workshops , 2018, pp. 224–236
2018
-
[31]
Ceres solver—a large scale non-linear optimization library,
S. Agarwal, K. Mierle, et al., “Ceres solver—a large scale non-linear optimization library,” 2019
2019
-
[32]
Bags of binary words for fast place recognition in image sequences,
D. G ´alvez-L´opez and J. D. Tard´os, “Bags of binary words for fast place recognition in image sequences,” IEEE Transactions on Robotics , vol. 28, no. 5, pp. 1188–1197, 2012
2012
-
[33]
A tutorial on quantitative trajectory evaluation for visual(-inertial) odometry,
Z. Zhang and D. Scaramuzza, “A tutorial on quantitative trajectory evaluation for visual(-inertial) odometry,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018, pp. 7244– 7251
2018
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.