REVIEW 2 major objections 5 minor 39 references
Bayesian Incremental Inference Update by Re-using Calculations from Belief Space Planning: A New Paradigm
T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Planning-stage factorizations can be reused to perform Bayesian inference updates exactly, at substantially lower cost.
desk verdict Reusing planning factorizations for inference updates is a sound, useful trick with a real KITTI speedup, but the exactness claim only holds for one Gauss-Newton iteration at the same linearization point and the paper owes an error bound for the rest. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the QR-factorized Jacobian of the Gaussian nonlinear least-squares problem: the square-root information matrix $R$ and its RHS vector $d$, often stored as a Bayes tree. Planning at time $k$ produces the augmented linear system formed from the previous factorized Jacobian and the new factors, and the identity $R_{k+1|k+1}=R_{k+1|k}$ says the expensive factorization carries over to inference. Around that identity the paper constructs four RHS-update methods (OTM, OTM-OO, DU, DU-OO) and a data-association update that detaches the involved sub-Bayes-tree, repairs the corresponding sub-factor-graph, and re-eliminates it.
What would settle it
Compare the $R$ matrix obtained by reusing the planning factorization against the $R$ matrix from a standard inference update on a sequence where the actual measurements are deliberately set to shift the linearization point across a re-linearization threshold; statistically significant divergence in the resulting MAP estimates would show that the equality $R_{k+1|k+1}=R_{k+1|k}$ does not survive re-linearization.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is the identity $R_{k+1|k+1}=R_{k+1|k}$: for the first inference iteration at time $k+1$, the factorized Jacobian (square-root information matrix) of the posterior updated from actual measurements equals the factorized Jacobian of the belief propagated during the precursory planning stage, provided the same action is executed, data association is consistent, and both problems linearize at the same point. The only new computation needed is therefore the right-hand-side vector $d_{k+1|k+1}$, and the paper gives four algebraically equivalent methods to produce it: two that rotate the new residual with the rotation matrix already available from planning, and two that use information downdate/update formulas. When planning and inference disagree on data association, an incremental factor-graph/Bayes-tree procedure first removes planning-only factors and adds inference-only factors, after which the RHS update applies. The complete pipeline is compared with standard incremental Bayesian inference on synthetic active-SLAM scenarios and on a real-world driving dataset, reporting the same estimation accuracy and at least a two-fold reduction in inference-update time.
Load-bearing premise
The load-bearing premise is that the planning and inference stages linearize at exactly the same point and one Gauss-Newton update is enough, so if the real measurements move the MAP estimate enough to make the system re-linearize, the reused factorized Jacobian $R$ is stale and the update is only approximate; the paper states the identity for the first inference iteration and gives no bound on the resulting error.
Editorial extensions
If this is right
- The only required change to a belief-space planner is that its output includes the propagated future belief alongside the chosen action; the planner itself does not need to be replaced.
- Because the factorized Jacobian is reused, the inference update avoids paying the same factorization cost twice, which is why loop closures and growing state dimension cost less when the relevant factors were already incorporated during planning.
- The rotation-based method (OTM-OO) is the fastest in the reported experiments, while the downdate-based methods trade some speed for easier availability of the required matrices.
- With the data-association repair step, the paradigm still outperforms the standard baseline in simulation even when only half of the predicted associations are correct, and on real-world data the full pipeline runs in less than half the inference-update time.
Reading between the lines
- The exactness of the reused $R$ is tied to the first inference iteration and identical linearization points; an open question the paper leaves is how much accuracy is lost when real measurements trigger re-linearization, since no error bound is given.
- The same RHS-update machinery could be applied to a curated library of precomputed beliefs rather than only the single planning-stage belief; the unsolved design problem is how to choose the stored belief closest to the needed posterior without spending the saved time on search.
- If planning could predict newly appearing landmarks, the fraction of factors reused would rise above the level seen in the real-data experiment, which suggests the reported speedup is a lower bound that improves with prediction quality.
- The paper's Gaussian/Gauss-Newton carrier object suggests a natural test for non-Gaussian beliefs: the equality would need a different summary statistic (e.g. the Fisher information) and the update-in-place step would likely become approximate.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper considers an MPC plan-act-infer loop and observes that the future belief propagated during belief space planning, b[X_{k+1}|k], already contains most of the factorized calculations needed for the subsequent Bayesian update. Under data association that is consistent between planning and inference, the authors prove that the factorized Jacobians coincide at the first Gauss-Newton iteration (Eq. (35)), so only the RHS vector d needs to be updated. They present four RHS-update methods (OTM, OTM-OO, DU, DU-OO), analyze their computational complexity, and combine this with an iSAM2-style Bayes-tree modification for inconsistent data association (Algorithm 1). The approach is evaluated in synthetic full-SLAM simulations and on 1400 steps of KITTI monocular data, reporting inference-update speedups of up to two orders of magnitude and negligible accuracy differences relative to iSAM2.
Significance. If the exactness claim is accepted for the intended operating regime, this is a useful contribution: it turns a conceptual similarity between BSP and inference into a concrete computational shortcut, and the four RHS-update methods are derived cleanly from standard linear algebra. The empirical evaluation is substantial, includes real-world data, and compares a MATLAB implementation against a C++ iSAM2 baseline, which is a conservative comparison in the authors' favor. I see no circularity: the reuse equations follow from QR factorization rather than from the authors' earlier JIP framework. The principal weakness is that the exactness result is demonstrated only for the first Gauss-Newton iteration at an identical linearization point, while the paper advertises unconditional same-accuracy equivalence; this gap needs to be closed or the claim qualified.
major comments (2)
- [Section 3.4, Eq. (35)] The equality R_{k+1|k+1} = R_{k+1|k} is valid only for the first Gauss-Newton iteration at a common linearization point; the text says 'for the first iteration in the inference stage at time k+1' immediately before Eq. (35). Standard iSAM2 performs fluid relinearization as measurements arrive, and when the optimum shifts, the planning Jacobian is no longer the Jacobian at the current linearization point, so the reused R is stale and a single RHS update does not compute the exact MAP estimate. The paper's unconditional claims of 'same estimation accuracy' (Abstract and end of Section 3.4) and 'algebraically equivalent to estimation via iSAM' (Section 4.2.4) are therefore too strong. The authors should either restrict the claim to the single-iteration/no-relinearization regime, provide a bound on the approximation error when relinearization is needed, or extend the method to update R when the linearization point changes.
- [Section 4.1.1 and Section 4.2.4] The experiments deliberately avoid the regime in which the central equality could fail. Section 4.1.1 states 'no re-linearization is necessary; hence, iSAM comparison is valid,' and the timing includes only the first inference update iteration; the KITTI experiment likewise uses a single horizon step and reports no substantial loop closures or relinearization events. Consequently, the results support the claim that R reuse is exact when a single Gauss-Newton update suffices, but they do not provide evidence for the general claim that RUB inference matches standard Bayesian inference in systems that relinearize. An experiment with active relinearization, or a theoretical error bound, is needed to close this gap.
minor comments (5)
- [Table 1] In Table 1, 'natch' should be 'match' in the description of daug_{t|k}.
- [Figures 5 and 6] The legends of Figures 5 and 6 use 'UD' for the methods that the text calls DU and DU-OO; please make the notation consistent with the body and with Table 1.
- [Throughout] Several typographical errors should be corrected: 'Given's' should be 'Givens' in Section 3.4.5, 'affectively' should be 'effectively' in Section 2.2, 'ssytem' should be 'system' in Section 5, and 'Beysian' should be 'Bayesian' in Figure 10.
- [Section 4.2.4] The phrase 'algebraically equivalent to estimation via iSAM' should be rephrased to state explicitly that the equivalence holds under the identical-linearization-point, first-iteration conditions identified in Section 3.4.
- [Section 4] The paper switches between 'iSAM' and 'iSAM2' when referring to the baseline; please use one name consistently or state explicitly that 'iSAM' denotes the iSAM2-based baseline throughout.
Circularity Check
No circularity: Eq. (35) is an algebraic identity under the paper's stated single-iteration, consistent-DA assumptions, and the RHS-update methods are derived from standard linear algebra and benchmarked against external iSAM2/KITTI results.
full rationale
The central reuse claim rests on R_{k+1|k+1} = R_{k+1|k} (Eq. 35). This follows by construction from two stated conditions: identical data association (Eq. 34) and identical linearization point (Sec. 3.3), and the paper explicitly restricts it to "the first iteration in the inference stage at time k+1" (Sec. 3.4). The four RHS-update methods are then derived from standard QR/information-form identities (Eqs. 39, 48, 53) rather than from the authors' prior claims; no parameter is fitted to the accuracy measure being reported. The speedup is benchmarked against external baselines (iSAM2/GTSAM, KITTI), so the performance claim is not forced by self-citation. Self-citations to Farhi and Indelman (2017, 2019a, 2019b) motivate the joint inference-planning research line and describe follow-up work, but they are not load-bearing for the algebraic equality or the runtime comparison. The main limitation—the equality holds for a single Gauss-Newton iteration, and Sec. 4.1.1 states "no re-linearization is necessary; hence, iSAM comparison is valid"—is a scope restriction on exactness, not a circular reduction. No error bound is given for the approximate multi-iteration regime, but that is a correctness/robustness gap, not circularity. No circular step within the paper's own derivation chain was identified.
Assumptions & free parameters
assumptions (5)
- domain assumption Motion and measurement models are zero-mean Gaussian (Eqs. 6-7).
- domain assumption The executed action equals the planner's optimal first action (Eq. 26).
- domain assumption The same initial linearization point is used for planning and inference (Section 3.3).
- domain assumption Data association is consistent between planning and inference for the four exact methods (Eq. 34).
- domain assumption Planning predicts future measurements by maximum likelihood and cannot predict new landmarks (Section 3.2).
Cite this review
Pith. "Pith review of Bayesian Incremental Inference Update by Re-using Calculations from Belief Space Planning: A New Paradigm." pith.science (2026). https://pith.science/paper/YDZU2O47
@misc{pith2026190802002,
author = {Pith},
title = {Pith review of: Bayesian Incremental Inference Update by Re-using Calculations from Belief Space Planning: A New Paradigm},
year = {2026},
howpublished = {\url{https://pith.science/paper/YDZU2O47}},
note = {Machine review of arXiv:1908.02002}
}
read the original abstract
Inference and decision making under uncertainty are key processes in every autonomous system and numerous robotic problems. In recent years, the similarities between inference and decision making triggered much work, from developing unified computational frameworks to pondering about the duality between the two. In spite of these efforts, inference and control, as well as inference and belief space planning (BSP) are still treated as two separate processes. In this paper we propose a paradigm shift, a novel approach which deviates from conventional Bayesian inference and utilizes the similarities between inference and BSP. We make the key observation that inference can be efficiently updated using predictions made during the decision making stage, even in light of inconsistent data association between the two. We developed a two staged process that implements our novel approach and updates inference using calculations from the precursory planning phase. Using autonomous navigation in an unknown environment along with iSAM2 efficient methodologies as a test case, we benchmarked our novel approach against standard Bayesian inference, both with synthetic and real-world data (KITTI dataset). Results indicate that not only our approach improves running time by at least a factor of two while providing the same estimation accuracy, but it also alleviates the computational burden of state dimensionality and loop closures.
Figures
Figures from the paper (15 more)
Reference graph
Works this paper leans on
- [1]
-
[2]
L. Carlone, A. Censi, and F. Dellaert. Selecting good measurements via l1 relaxation: A convex approach for robust estimation over graphs. In IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), pages 2667--2674. IEEE, 2014
work page 2014
-
[3]
A. Cunningham, V. Indelman, and F. Dellaert. DDF-SAM 2.0: Consistent distributed smoothing and mapping. In IEEE Intl. Conf. on Robotics and Automation (ICRA), Karlsruhe, Germany, May 2013
work page 2013
-
[4]
A. Davison, I. Reid, N. Molton, and O. Stasse. Mono SLAM : Real-time single camera SLAM . IEEE Trans. Pattern Anal. Machine Intell. , 29 0 (6): 0 1052--1067, Jun 2007
work page 2007
- [5]
-
[6]
F. Dellaert and M. Kaess. Square Root SAM : Simultaneous localization and mapping via square root information smoothing. Intl. J. of Robotics Research, 25 0 (12): 0 1181--1203, Dec 2006
work page 2006
-
[7]
R. Eustice, H. Singh, and J. Leonard. Exactly sparse delayed-state filters for view-based SLAM . IEEE Trans. Robotics , 22 0 (6): 0 1100--1114, Dec 2006
work page 2006
-
[8]
E. I. Farhi and V. Indelman. Towards efficient inference update through planning via jip - joint inference and belief space planning. In IEEE Intl. Conf. on Robotics and Automation (ICRA), 2017
work page 2017
Show all 39 references
-
[9]
E. I. Farhi and V. Indelman. ix-bsp: Belief space planning through incremental expectation. In IEEE Intl. Conf. on Robotics and Automation (ICRA), May 2019 a
2019
-
[10]
E. I. Farhi and V. Indelman. Tear down that wall: Calculation reuse across inference and belief space planning. In Toward Online Optimal Control of Dynamic Robots, Workshop in conjunction with IEEE International Conference on Robotics and Automation (ICRA), May 2019 b
2019
-
[11]
Geiger, P
A. Geiger, P. Lenz, C. Stiller, and R. Urtasun. Vision meets robotics: The kitti dataset. International Journal of Robotics Research (IJRR), 2013
2013
-
[12]
Golub and C
G. Golub and C. V. Loan. Matrix Computations. Johns Hopkins University Press, Baltimore, third edition, 1996
1996
-
[13]
R. I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, second edition, 2004
2004
-
[14]
S. S. Haykin et al. Kalman filtering and neural networks. Wiley Online Library, 2001
2001
-
[15]
G. A. Hollinger and G. S. Sukhatme. Sampling-based robotic information gathering algorithms. Intl. J. of Robotics Research, pages 1271--1287, 2014
2014
-
[16]
Indelman, L
V. Indelman, L. Carlone, and F. Dellaert. Planning in the continuous domain: a generalized belief space approach for autonomous navigation in unknown environments. Intl. J. of Robotics Research, 34 0 (7): 0 849--882, 2015
2015
-
[17]
Indelman, E
V. Indelman, E. Nelson, J. Dong, N. Michael, and F. Dellaert. Incremental distributed inference from arbitrary poses and unknown data association: Using collaborating robots to establish a common reference. IEEE Control Systems Magazine (CSM), Special Issue on Distributed Cont...
2016
-
[18]
L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101 0 (1): 0 99--134, 1998
1998
-
[19]
Kaess, A
M. Kaess, A. Ranganathan, and F. Dellaert. iSAM : Incremental smoothing and mapping. IEEE Trans. Robotics , 24 0 (6): 0 1365--1378, Dec 2008
2008
-
[20]
Kaess, V
M. Kaess, V. Ila, R. Roberts, and F. Dellaert. The B ayes tree: Enabling incremental reordering and fluid relinearization for online mapping. Technical Report MIT-CSAIL-TR-2010-021, Computer Science and Artificial Intelligence Laboratory, MIT, Jan 2010
2010
-
[21]
Kaess, H
M. Kaess, H. Johannsson, R. Roberts, V. Ila, J. Leonard, and F. Dellaert. iSAM2 : Incremental smoothing and mapping using the B ayes tree. Intl. J. of Robotics Research, 31: 0 217--236, Feb 2012
2012
-
[22]
Kim and R
A. Kim and R. M. Eustice. Active visual SLAM for robotic area coverage: Theory and experiment. Intl. J. of Robotics Research, 34 0 (4-5): 0 457--475, 2014
2014
-
[23]
Kobilarov, D.-N
M. Kobilarov, D.-N. Ta, and F. Dellaert. Differential dynamic programming for optimal estimation. In IEEE Intl. Conf. on Robotics and Automation (ICRA), pages 863--869. IEEE, 2015
2015
-
[24]
Kschischang, B
F. Kschischang, B. Frey, and H.-A. Loeliger. Factor graphs and the sum-product algorithm. IEEE Trans. Inform. Theory , 47 0 (2): 0 498--519, February 2001
2001
-
[25]
Kurniawati, D
H. Kurniawati, D. Hsu, and W. S. Lee. Sarsop: Efficient point-based pomdp planning by approximating optimally reachable belief spaces. In Robotics: Science and Systems (RSS), volume 2008, 2008
2008
-
[26]
D. G. Lowe. Distinctive image features from scale-invariant keypoints. Intl. J. of Computer Vision, 60 0 (2): 0 91--110, 2004
2004
-
[27]
Olson and P
E. Olson and P. Agarwal. Inference on networks of mixtures for robust robot mapping. Intl. J. of Robotics Research, 32 0 (7): 0 826--840, 2013
2013
-
[28]
Papadimitriou and J
C. Papadimitriou and J. Tsitsiklis. The complexity of markov decision processes. Mathematics of operations research, 12 0 (3): 0 441--450, 1987
1987
-
[29]
Pathak, A
S. Pathak, A. Thomas, and V. Indelman. A unified framework for data association aware robust belief space planning and perception. Intl. J. of Robotics Research, 32 0 (2-3): 0 287--315, 2018
2018
-
[30]
Pineau, G
J. Pineau, G. J. Gordon, and S. Thrun. Anytime point-based approximations for large POMDP s. J. of Artificial Intelligence Research, 27: 0 335--380, 2006
2006
-
[31]
Platt, R
R. Platt, R. Tedrake, L. Kaelbling, and T. Lozano-P\'erez. Belief space planning assuming maximum likelihood observations. In Robotics: Science and Systems (RSS), pages 587--593, Zaragoza, Spain, 2010
2010
-
[32]
Prentice and N
S. Prentice and N. Roy. The belief roadmap: Efficient planning in belief space by factoring the covariance. Intl. J. of Robotics Research, 28 0 (11-12): 0 1448--1465, 2009
2009
-
[33]
Sunderhauf and P
N. Sunderhauf and P. Protzel. Towards a robust back-end for pose graph slam. In IEEE Intl. Conf. on Robotics and Automation (ICRA), pages 1254--1261. IEEE, 2012
2012
-
[34]
D.-N. Ta, M. Kobilarov, and F. Dellaert. A factor graph approach to estimation and model predictive control on unmanned aerial vehicles. In International Conference on Unmanned Aircraft Systems (ICUAS), pages 181--188. IEEE, 2014
2014
-
[35]
Thrun, Y
S. Thrun, Y. Liu, D. Koller, A. Ng, Z. Ghahramani, and H. Durrant-Whyte. Simultaneous localization and mapping with sparse extended information filters. Intl. J. of Robotics Research, 23 0 (7-8): 0 693--716, 2004
2004
-
[36]
E. Todorov. General duality between optimal control and estimation. In IEEE Conference on Decision and Control, pages 4286--4292. IEEE, 2008
2008
-
[37]
Toussaint
M. Toussaint. Robot trajectory optimization using approximate inference. In Intl. Conf. on Machine Learning (ICML), pages 1049--1056. ACM, 2009
2009
-
[38]
Toussaint and A
M. Toussaint and A. Storkey. Probabilistic inference for solving discrete and continuous state markov decision processes. In Intl. Conf. on Machine Learning (ICML), pages 945--952. ACM, 2006
2006
-
[39]
Van Den Berg, S
J. Van Den Berg, S. Patil, and R. Alterovitz. Motion planning under uncertainty using iterative local optimization in belief space. Intl. J. of Robotics Research, 31 0 (11): 0 1263--1278, 2012
2012
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.