pith. sign in

arxiv: 1907.08388 · v1 · pith:TDAEBFB4new · submitted 2019-07-19 · 💻 cs.RO · cs.CV

Robust Real-time RGB-D Visual Odometry in Dynamic Environments via Rigid Motion Model

Sangil Lee , Clark Youngdong Son , H. Jin Kim This is my paper

Pith reviewed 2026-05-24 19:38 UTC · model grok-4.3

classification 💻 cs.RO cs.CV
keywords visual odometryRGB-Ddynamic environmentsscene flowmotion segmentationrigid motion modelcamera pose estimationreal-time tracking
0
0 comments X

The pith

A visual odometry algorithm separates static regions from independently moving rigid objects using grid-based scene flow clustering and a dual-mode motion model for accurate camera pose estimation.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a real-time RGB-D visual odometry system for scenes containing dynamic rigid objects. Spatial segmentation generates motion hypotheses from grid-based scene flow, then clusters them to isolate objects moving independently of the camera. A dual-mode motion model maintains consistent static and dynamic labels across frames in the temporal tracking stage. Camera pose is computed exclusively from the regions classified as static. Tests on a self-collected RGB-D dataset with motion-capture ground truth indicate improved robustness over prior methods when moving objects are present.

Core claim

The algorithm estimates the pose of a camera by taking advantage of the region classified as static parts after spatial motion segmentation that clusters grid-based scene flow hypotheses to separate independently moving objects and temporal motion tracking that applies a dual-mode motion model to keep static/dynamic labels consistent.

What carries the argument

Rigid-motion model updated by scene flow, which performs spatial segmentation by clustering motion hypotheses and temporal tracking with dual-mode labels to select static regions for pose estimation.

If this is right

  • Multiple independently moving rigid objects can be isolated without contaminating the static region used for pose estimation.
  • Temporal consistency from the dual-mode model reduces jitter in the estimated trajectory across frames.
  • The approach runs in real time while handling dynamic content that defeats standard visual odometry pipelines.
  • Performance gains appear when compared against existing visual odometry algorithms on RGB-D sequences with ground-truth motion capture.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same segmentation and labeling pipeline could be inserted into existing SLAM systems to treat moving objects as separate entities rather than outliers.
  • If scene flow computation is replaced by a faster approximation, the overall method might run on embedded hardware with only modest accuracy loss.
  • The clustering step implicitly assumes that object motions remain rigid and distinct; violations would require additional outlier rejection logic.
  • Extending the dual-mode model to include velocity predictions could improve label stability during brief occlusions.

Load-bearing premise

The method assumes that independent rigid motions of objects can be reliably separated by clustering grid-based scene flow hypotheses and that a dual-mode motion model can maintain consistent static/dynamic labels over time.

What would settle it

Running the algorithm on a dataset where objects undergo non-rigid deformation or where scene flow is corrupted by low-texture surfaces would produce visibly incorrect camera trajectories if the clustering or label consistency steps fail.

Figures

Figures reproduced from arXiv: 1907.08388 by Clark Youngdong Son, H. Jin Kim, Sangil Lee.

Figure 1
Figure 1. Figure 1: The 3D trajectory on the uav-flight-circular sequence. In order to differentiate between non-stationary parts and stationary background, we utilize scene flow vectors which are distributed uniformly in the image. Particularly, we choose grid-based scene flow to take advantage of both dense and sparse methods; a dense flow that calculates temporary motions for all pixels provides a high resolution, meanwhil… view at source ↗
Figure 2
Figure 2. Figure 2: A schematic diagram of the proposed algorithm. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The description of the motion segmentation on the [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Variations of the parameters. The relative pose error are denoted as magenta boxplot with 3- [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The validation environment for the uav-flight-circular sequence. From the above boxplot analysis, the parameters were set to: wgrid = 16, hmax = 20, thinlier = 3 × 10−5 , αmax = 5, which are well-tuned values for our indoor dataset. The parameters are kept the same over the dataset for consistency. Rest of the parameters are set as follows: • The number of points and the searching radius in Section III-A a… view at source ↗
Figure 6
Figure 6. Figure 6: Some of the sequence images and the segmentation results that the proposed algorithm provides. Filled circles mean grid cell that has accurate [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
read the original abstract

In the paper, we propose a robust real-time visual odometry in dynamic environments via rigid-motion model updated by scene flow. The proposed algorithm consists of spatial motion segmentation and temporal motion tracking. The spatial segmentation first generates several motion hypotheses by using a grid-based scene flow and clusters the extracted motion hypotheses, separating objects that move independently of one another. Further, we use a dual-mode motion model to consistently distinguish between the static and dynamic parts in the temporal motion tracking stage. Finally, the proposed algorithm estimates the pose of a camera by taking advantage of the region classified as static parts. In order to evaluate the performance of visual odometry under the existence of dynamic rigid objects, we use self-collected dataset containing RGB-D images and motion capture data for ground-truth. We compare our algorithm with state-of-the-art visual odometry algorithms. The validation results suggest that the proposed algorithm can estimate the pose of a camera robustly and accurately in dynamic environments.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes a real-time RGB-D visual odometry algorithm for dynamic environments. It performs spatial motion segmentation by generating motion hypotheses via grid-based scene flow and clustering them to separate independently moving rigid objects. A dual-mode motion model is then used in temporal tracking to maintain consistent static/dynamic labels over time. Camera pose is estimated using only the regions labeled as static. Evaluation uses a self-collected RGB-D dataset with motion-capture ground truth, with comparisons to state-of-the-art visual odometry methods; the abstract states that the approach yields robust and accurate pose estimates in the presence of dynamic rigid objects.

Significance. If the quantitative claims hold with supporting data, the work could offer a practical contribution to visual odometry by explicitly leveraging rigid-motion hypotheses for segmentation and tracking. The grid-based scene-flow clustering plus dual-mode temporal consistency is a coherent strategy for isolating static background. The use of motion-capture ground truth is a methodological strength. However, the absence of any numerical results, ablation studies, or implementation details in the provided description prevents assessment of whether the method meaningfully advances the state of the art or generalizes beyond the collected sequences.

major comments (2)
  1. [Abstract] Abstract: the central claim that the algorithm 'outperforms state-of-the-art visual odometry algorithms' and 'can estimate the pose of a camera robustly and accurately' is unsupported by any quantitative metrics, error statistics, tables, figures, or ablation results. Without these data the robustness assertion cannot be evaluated and is load-bearing for the contribution.
  2. [Spatial motion segmentation] Spatial motion segmentation (as described in the abstract): the pipeline assumes that clustering of grid-based scene-flow hypotheses reliably isolates the static background from independent rigid motions. Scene flow from RGB-D is known to be sensitive to depth noise, fast motion, and partial occlusions; the description provides no explicit outlier rejection, multi-hypothesis fusion, or robustness mechanism inside clusters. If this separation fails, the subsequent static-region pose estimation inherits the error, directly undermining the robustness claim.
minor comments (1)
  1. [Abstract] Abstract: the self-collected dataset is mentioned but no details are given on sequence count, dynamic-object types, camera motion profiles, or environmental conditions, hindering reproducibility and comparison.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on the abstract claims and the details of spatial motion segmentation. We address each major comment below and will revise the manuscript to strengthen the presentation of quantitative support and robustness mechanisms.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that the algorithm 'outperforms state-of-the-art visual odometry algorithms' and 'can estimate the pose of a camera robustly and accurately' is unsupported by any quantitative metrics, error statistics, tables, figures, or ablation results. Without these data the robustness assertion cannot be evaluated and is load-bearing for the contribution.

    Authors: The manuscript reports comparisons against state-of-the-art visual odometry methods on a self-collected RGB-D dataset with motion-capture ground truth, and the evaluation section presents the corresponding results. However, we agree that the abstract alone does not include specific numerical metrics. We will revise the abstract to incorporate key quantitative results (e.g., average trajectory error reductions) and a brief reference to the evaluation protocol so that the claims are directly supported. revision: yes

  2. Referee: [Spatial motion segmentation] Spatial motion segmentation (as described in the abstract): the pipeline assumes that clustering of grid-based scene-flow hypotheses reliably isolates the static background from independent rigid motions. Scene flow from RGB-D is known to be sensitive to depth noise, fast motion, and partial occlusions; the description provides no explicit outlier rejection, multi-hypothesis fusion, or robustness mechanism inside clusters. If this separation fails, the subsequent static-region pose estimation inherits the error, directly undermining the robustness claim.

    Authors: The full manuscript describes the grid-based scene-flow generation and subsequent clustering in the spatial motion segmentation section, with the dual-mode rigid motion model providing temporal consistency. The abstract is necessarily concise and omits these implementation details. We will expand the relevant section to explicitly discuss handling of depth noise and occlusions (including any filtering or clustering thresholds employed) and add a short robustness analysis or reference to failure cases. This will clarify the mechanisms that prevent error propagation to the static-region pose estimation. revision: partial

Circularity Check

0 steps flagged

No circularity: algorithmic pipeline is self-contained procedural description

full rationale

The paper describes a visual odometry algorithm consisting of grid-based scene flow hypothesis generation, clustering for spatial segmentation, dual-mode temporal motion tracking, and static-region pose estimation. No equations, fitted parameters, or predictions are presented that reduce to inputs by construction. No self-citations are invoked as load-bearing uniqueness theorems or ansatzes. The method is evaluated on an external self-collected dataset with motion capture ground truth, making the central claim an independent algorithmic procedure rather than a tautological renaming or fit. This matches the default expectation of no significant circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract supplies no explicit free parameters, axioms, or invented entities; the rigid-motion assumption for dynamic objects is a standard domain assumption in visual odometry rather than a novel postulate.

pith-pipeline@v0.9.0 · 5698 in / 1182 out tokens · 21327 ms · 2026-05-24T19:38:13.552695+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages

  1. [1]

    Visual odometry,

    D. Nist ´er, O. Naroditsky, and J. Bergen, “Visual odometry,” in Com- puter Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on , vol. 1. IEEE, 2004, pp. I–652

    work page 2004

  2. [2]

    Visual odometry [tutorial],

    D. Scaramuzza and F. Fraundorfer, “Visual odometry [tutorial],” IEEE Robotics & Automation Magazine , vol. 18, no. 4, pp. 80–92, 2011

    work page 2011

  3. [3]

    A benchmark for the evaluation of rgb-d slam systems,

    J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers, “A benchmark for the evaluation of rgb-d slam systems,” in Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on. IEEE, 2012, pp. 573–580

    work page 2012

  4. [4]

    Vision meets robotics: The kitti dataset,

    A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, “Vision meets robotics: The kitti dataset,” The International Journal of Robotics Research , vol. 32, no. 11, pp. 1231–1237, 2013

    work page 2013

  5. [5]

    Robust dense visual odometry for rgb-d cameras in a dynamic environment,

    A. Dib and F. Charpillet, “Robust dense visual odometry for rgb-d cameras in a dynamic environment,” in Advanced Robotics (ICAR), 2015 International Conference on . IEEE, 2015, pp. 1–7

    work page 2015

  6. [6]

    Stereo vision-based visual odometry using robust visual feature in dynamic environment,

    S.-J. Jung, J.-B. Song, and S.-C. Kang, “Stereo vision-based visual odometry using robust visual feature in dynamic environment,” The Journal of Korea Robotics Society , vol. 3, no. 4, pp. 263–269, 2008

    work page 2008

  7. [7]

    Monocular simultaneous multi- body motion segmentation and reconstruction from perspective views,

    R. Sabzevari and D. Scaramuzza, “Monocular simultaneous multi- body motion segmentation and reconstruction from perspective views,” in Robotics and Automation (ICRA), 2014 IEEE International Confer- ence on. IEEE, 2014, pp. 23–30

    work page 2014

  8. [8]

    Video segmentation via object flow,

    Y .-H. Tsai, M.-H. Yang, and M. J. Black, “Video segmentation via object flow,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2016, pp. 3899–3908

    work page 2016

  9. [9]

    Rigid motion segmentation using random- ized voting,

    H. Jung, J. Ju, and J. Kim, “Rigid motion segmentation using random- ized voting,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2014, pp. 1210–1217

    work page 2014

  10. [10]

    Effective background model-based rgb-d dense visual odometry in a dynamic environment,

    D.-H. Kim and J.-H. Kim, “Effective background model-based rgb-d dense visual odometry in a dynamic environment,” IEEE Transactions on Robotics, vol. 32, no. 6, pp. 1565–1573, 2016

    work page 2016

  11. [11]

    Real-time visual odometry from dense rgb-d images,

    F. Steinbr ¨ucker, J. Sturm, and D. Cremers, “Real-time visual odometry from dense rgb-d images,” in Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on . IEEE, 2011, pp. 719–722

    work page 2011

  12. [12]

    Dense visual slam for rgb-d cameras,

    C. Kerl, J. Sturm, and D. Cremers, “Dense visual slam for rgb-d cameras,” in Intelligent Robots and Systems (IROS), 2013 IEEE/RSJ International Conference on . IEEE, 2013, pp. 2100–2106

    work page 2013

  13. [13]

    Svo: Semidirect visual odometry for monocular and multicamera systems,

    C. Forster, Z. Zhang, M. Gassner, M. Werlberger, and D. Scaramuzza, “Svo: Semidirect visual odometry for monocular and multicamera systems,” IEEE Transactions on Robotics , vol. 33, no. 2, pp. 249– 265, 2017

    work page 2017

  14. [14]

    Orb-slam2: an open-source slam system for monocular, stereo and rgb-d cameras,

    R. Mur-Artal and J. D. Tardos, “Orb-slam2: an open-source slam system for monocular, stereo and rgb-d cameras,” arXiv preprint arXiv:1610.06475, 2016

    work page arXiv 2016

  15. [15]

    Direct sparse odometry,

    J. Engel, V . Koltun, and D. Cremers, “Direct sparse odometry,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017

    work page 2017

  16. [16]

    Moving on to dynamic envi- ronments: Visual odometry using feature classification,

    B. Kitt, F. Moosmann, and C. Stiller, “Moving on to dynamic envi- ronments: Visual odometry using feature classification,” in Intelligent Robots and Systems (IROS), 2010 IEEE/RSJ International Conference on. IEEE, 2010, pp. 5551–5556

    work page 2010

  17. [17]

    Fast odometry and scene flow from rgb-d cameras based on geometric clus- tering,

    M. Jaimez, C. Kerl, J. Gonzalez-Jimenez, and D. Cremers, “Fast odometry and scene flow from rgb-d cameras based on geometric clus- tering,” in Proc. International Conference on Robotics and Automation (ICRA), 2017

    work page 2017

  18. [18]

    Multi-body motion estimation from monocular vehicle-mounted cameras,

    R. Sabzevari and D. Scaramuzza, “Multi-body motion estimation from monocular vehicle-mounted cameras,” IEEE Transactions on Robotics, vol. 32, no. 3, pp. 638–651, 2016

    work page 2016

  19. [19]

    Adaptive motion segmentation algorithm based on the principal angles configuration,

    L. Zappella, E. Provenzi, X. Llad ´o, and J. Salvi, “Adaptive motion segmentation algorithm based on the principal angles configuration,” Computer Vision–ACCV 2010, pp. 15–26, 2011

    work page 2010

  20. [20]

    Sparse subspace clustering,

    E. Elhamifar and R. Vidal, “Sparse subspace clustering,” in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2009, pp. 2790–2797

    work page 2009

  21. [21]

    Detection of moving objects with non-stationary cameras in 5.8 ms: Bringing motion detection to your mobile device,

    K. Moo Yi, K. Yun, S. Wan Kim, H. Jin Chang, and J. Young Choi, “Detection of moving objects with non-stationary cameras in 5.8 ms: Bringing motion detection to your mobile device,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2013, pp. 27–34

    work page 2013

  22. [22]

    An iterative image registration technique with an application to stereo vision,

    B. D. Lucas, T. Kanade, et al. , “An iterative image registration technique with an application to stereo vision,” 1981

    work page 1981

  23. [23]

    Estimating 3-d rigid body transformations: a comparison of four major algorithms,

    D. W. Eggert, A. Lorusso, and R. B. Fisher, “Estimating 3-d rigid body transformations: a comparison of four major algorithms,” Machine vision and applications , vol. 9, no. 5-6, pp. 272–290, 1997

    work page 1997

  24. [24]

    Mexopencv,

    K. Yamaguchi, “Mexopencv,” Collection and a development kit of matlab mex functions for OpenCV library, available at http://www. cs. stonybrook. edu/˜ kyamagu/mexopencv, 2013

    work page 2013

  25. [25]

    A density-based algorithm for discovering clusters in large spatial databases with noise

    M. Ester, H.-P. Kriegel, J. Sander, X. Xu, et al. , “A density-based algorithm for discovering clusters in large spatial databases with noise.” in Kdd, vol. 96, no. 34, 1996, pp. 226–231

    work page 1996