REVIEW 3 major objections 5 minor 54 references
Particle-Based Conformal Prediction for Contact-Aware Uncertainty Calibration in Stratified Configuration Spaces
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read CaPTURe proves per-contact-stratum coverage for one-step next-configuration prediction regions built from particle-based dynamics, confined to the feasible configuration space.
desk verdict Correct Mondrian CP theorem and a useful particle-based region construction, but the paper's per-stratum coverage claims outrun what Theorem 5 actually proves. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is Mondrian conformal prediction adapted to a finite stratified configuration space $C_{\mathrm{feas}}=\bigsqcup_m S_m$ with a known stratum indexer $T$: the group assignment $g(X,T(Y))$ sends each transition to a leaf of a regression tree fitted on augmented inputs $(X,T(Y))$ with the particle-score $R_i$ as target, and each leaf gets its own split-conformal threshold $\hat{q}_j$. At inference the region for a candidate future stratum $m$ is built by thresholding the $k$-th-nearest-particle distance inside $S_m$, and the per-stratum regions are unioned. This one construction produces conditional coverage per contact regime, keeps all candidate configurations in feasible strata, and makes the set size adapt to local state- and action-dependent uncertainty.
What would settle it
Run a one-step validation set with a deliberately wrong stratum indexer, for example reducing the wall-inflation radius by 2 mm after calibration so edge and corner labels shift; if any group's empirical coverage falls below $1-\alpha$ by more than the finite-sample quantile slack, or if any predicted region contains an obstacle cell, the exact-$T$ premise fails.
Extended reading notes
Core claim
The central claim is Theorem 5: given exchangeable calibration data, a fixed group map $g(X, T(Y))$ over the prediction-time information and the future stratum index, and any symmetric nonconformity score $r$, the region $$\hat{C}(X_{n+1}) = \{y\in C_{\mathrm{feas}} : r(\hat{f}(X_{n+1}), y) \le \hat{q}_{g(X_{n+1},T(y))}\}$$ satisfies $\Pr(Y_{n+1}\in \hat{C}(X_{n+1}) \mid g(X_{n+1},T(Y_{n+1}))=j)\ge 1-\alpha$ for every group $j$ with positive mass. The nonconformity score is the distance from the true next configuration to its $k$-th nearest particle among $L$ samples from an approximate dynamics model, so the region is distribution-free and can represent multimodal, trans-dimensional post-contact uncertainty. Since the strata are disjoint and cover $C_{\mathrm{feas}}$, unioning the per-stratum regions over candidate $m$ preserves marginal coverage and yields $\hat{C}\cap C_{\mathrm{obs}}=\emptyset$. The paper's experiments confirm this per-stratum coverage empirically at the 90% level in free-space, edge, and corner regimes, where non-stratum baselines undercover free space or overcover contacts. The discovery is that conformal calibration can be made contact-aware by treating the configuration space's stratified structure as part of the group assignment, so coverage and feasibility come from the same construction.
Load-bearing premise
The guarantee rests on calibration data having the same distribution as test data and on the contact-regime labeler being exactly right; the paper concedes in Section 6.3 that geometric uncertainty could put infeasible configurations in the regions.
Editorial extensions
If this is right
- Planners can use prediction regions that are trustworthy per contact regime, not just on average, because each group carries its own finite-sample $1-\alpha$ coverage guarantee.
- Because every region lies inside $C_{\mathrm{feas}}$, uncertainty-aware planning will not be made overly conservative by obstacle-intersecting hypothetical configurations.
- Any approximate dynamics model that can emit particles can be calibrated this way; no Gaussian or other parametric uncertainty representation is required.
- The paper's simulations indicate that achieving per-stratum coverage improves downstream task outcomes, with up to a 30 percentage-point absolute gain in peg-insertion success over the best baseline.
Reading between the lines
- Beyond the paper: the exact-$T$ premise is the main limitation in real systems; extending CaPTURe to calibrate over an uncertain or learned stratum indexer, or to inflate strata near boundaries, would be the natural next step.
- Beyond the paper: since Theorem 5 is single-step, a testable extension is to measure cumulative closed-loop coverage over $H$-step MPC rollouts and to study whether per-step conditional calibration composes along the executed trajectory.
- Beyond the paper: the decision-tree partitioner can be replaced by any exchangeability-preserving partitioner of $(X,T(Y))$; denser data or domain-structured groups should tighten regions without breaking coverage.
- Beyond the paper: the comparison between $k_{\mathrm{NN}}=1$ and $k_{\mathrm{NN}}=L/2$ suggests that smoothing the particle-density score affects planning efficiency; one could test whether larger particle budgets with smoothed scores yield smaller feasible regions at equal coverage.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes CaPTURe, a conformal prediction method for one-step robot configuration prediction in contact-rich settings. It combines probabilistic conformal prediction (particle-based scores) with Mondrian conformal prediction, where the Mondrian groups are leaves of a regression tree fitted on a random split of the calibration data, using features (state, action, future configuration stratum index). The central theoretical results are Theorem 5, which gives finite-sample leaf-conditional coverage, and Corollary 6, which gives marginal coverage by union over output strata; the construction also restricts prediction regions to the feasible C-space. The method is evaluated on two simulated tasks, a marble labyrinth and a tight-tolerance peg insertion, against four baselines, reporting per-stratum empirical coverage, region volume, and planning success rates.
Significance. If the guarantee is read as leaf-conditional, the theoretical core is a correct and clean application of Mondrian conformal prediction with LOCART-style data splitting: the coverage argument does not rely on fitted parameters or on the accuracy of the dynamics model, and the feasible-region construction is principled. The experimental setup is unusually thorough (on the order of 10^5 calibration and validation transitions per map, two contact-rich tasks, multiple baselines), and the reported planning improvements (up to 30% absolute success-rate gain) are practically meaningful. The weakness is that the paper's headline claim of coverage "in and out of contact" is per-stratum, while the theorem only guarantees coverage per DTree leaf; several per-stratum point estimates in Table 5 fall below the 90% target. The paper is therefore a solid contribution to conformal prediction for robot dynamics, but its central narrative currently overstates what is proved.
major comments (3)
- [Section 5.1, Theorem 5 and Corollary 6, versus the Abstract and Tables 1, 3, 5] Theorem 5 establishes coverage conditional on the DTree leaf j = g(X,T(Y)), not on the contact stratum T(Y)=m, and Corollary 6 only aggregates these leaf-conditional guarantees into marginal coverage. The abstract and Section 6 nevertheless claim that CaPTURe "achieves the user-specified coverage requirement both in and out of contact" and use per-stratum coverage as the primary validation target. A leaf can contain several strata, so P(Y in C_hat | g = j) >= 1-alpha does not imply P(Y in C_hat | T(Y)=m) >= 1-alpha. Table 5 contains per-stratum estimates below 90% (Bottom Left Edge 89.5%, Center Free Space 89.5%, Center Right Corner 88.6%), so this is not a merely theoretical gap. Please either modify the method so the Mondrian partition refines the stratum partition (or intersect leaves with strata before computing thresholds) and prove stratum-conditional coverage, or reframe the claims and validation around leaf-conditional coverage and report the corresponding per-leaf numbers.
- [Section 6.1 and 6.2, Tables 1, 3, and 5] The empirical coverage numbers are reported only as point estimates, without confidence intervals or per-stratum (let alone per-leaf) sample counts. Since some entries fall below the nominal 90% (Table 5), the validation cannot distinguish finite-sample variability from genuine undercoverage. Please report binomial confidence intervals for every reported coverage value and give the number of test cases per stratum and per DTree leaf, because the leaf is the level at which Theorem 5 makes a claim.
- [Section 5.1, Lemma 4 and Theorem 5, with Section 5.2] The theorem treats g as a fixed grouping function, but Algorithm 1 defines g as the DTree fitted on D_bar^part_cal; Lemma 4 is stated without proof, and Theorem 5 does not explicitly state that g is fixed before the thresholds are computed from D_bar^cp_cal. Please state Theorem 5 conditionally on the DTree partition (or otherwise make the independence of g from the threshold-calibration subset explicit) and provide the proof of Lemma 4, so that the proof of Theorem 5 is formally complete.
minor comments (5)
- [Section 6] The text reads "kNN = L/27", which contradicts the following sentences and footnote 7 that indicate kNN = L/2 (8 for L=16, 4 for L=8); please correct the typo.
- [Section 5.1] Equation (8) defines g on X times M, but Theorem 5 and surrounding text write g(X,T(Y)); please define the composed map explicitly to avoid notational ambiguity.
- [Table 5 caption] Total test cases per map are given, but not per stratum; please add per-stratum counts so the reader can interpret the point estimates.
- [Section 6.3] The dependence of the feasibility claim on Assumption 2 (a correct, known stratum indexer T) is stated only in the limitations; because the abstract emphasizes feasible regions, this assumption should be flagged prominently in Section 3.
- [Section 4.3] Theorem 3 is cited rather than proved; a one-sentence sketch of the finite-sample quantile argument would make the review section self-contained.
Circularity Check
No significant circularity: CaPTURe's coverage guarantee follows from exchangeability and holdout conformal quantiles; the learned DTree grouping is fitted on a separate split, so the derivation is self-contained.
full rationale
The central derivation is not circular. Theorem 5 defines the prediction region from per-leaf conformal quantiles q_g computed on a holdout split (Dcp_cal) of calibration data; the coverage claim is a direct application of Mondrian/SplitCP under Assumption 1, not a consequence of fitting any parameter to the coverage target. The DTree grouping g is fit on a separate split (Dpart_cal) and fixed before the thresholds are computed, so the learned partition does not consume the conformal calibration data; conditioning on the fitted tree leaves Dcp_cal and the test point exchangeable by the exchangeability of the original calibration set. The self-citations (Marques and Berenson 2026a,b; Marques et al. 2026; Phillips-Grafflin and Berenson 2020) are contextual or methodological rather than load-bearing: the LOCART grouping is attributed to Cabezas et al. and then described in sufficient detail in Section 5.2. The paper also explicitly scopes its guarantees to single steps and to a known, correct stratum indexer T (Assumption 2, Section 6.3), and the proof of Corollary 6 is a standard law-of-total-probability marginalization. The empirical per-stratum coverage tables are not entailed by Theorem 5, which conditions on DTree leaves rather than on strata; that is an interpretation/claim-scope gap, not a circular derivation, since no theorem assumes its own conclusion. No quoted step reduces an output to an input by construction.
Assumptions & free parameters
free parameters (5)
- kNN (nearest-neighbor index in score function) =
L/2 (8 for marble, 4 for peg); ablation uses 1
- DTree hyperparameters (max depth, min samples per leaf, etc.) =
not reported
- 70/30 split of calibration data (Dpart versus Dcp) =
0.7/0.3
- Stratum definition thresholds =
marble: 2 mm edge/corner band; peg: 0.75 mm contact disks/tubes; grid 1 mm, 0.25 mm, 0.5 deg
- SE(2) characteristic length l =
0.0253 m
assumptions (5)
- domain assumption Assumption 1: Dcal is exchangeable with test-time transitions
- domain assumption Assumption 2: C_feas is a known finite stratified space with known stratum indexer T
- domain assumption Approximate dynamics model f_hat can generate L i.i.d. particles from p_hat(Y|X)
- standard math Random split of Dcal into Dpart and Dcp preserves exchangeability for threshold computation
- standard math Theorem 2 of Wang et al. (2023) and Theorem 3 (Mondrian guarantee) are correct
Cite this review
Pith. "Pith review of Particle-Based Conformal Prediction for Contact-Aware Uncertainty Calibration in Stratified Configuration Spaces." pith.science (2026). https://pith.science/paper/LQ3F5BEP
@misc{pith2026260809166,
author = {Pith},
title = {Pith review of: Particle-Based Conformal Prediction for Contact-Aware Uncertainty Calibration in Stratified Configuration Spaces},
year = {2026},
howpublished = {\url{https://pith.science/paper/LQ3F5BEP}},
note = {Machine review of arXiv:2608.09166}
}
read the original abstract
Reliable uncertainty representation is essential for deploying autonomous systems that interact with their environment, as robots must reason about how uncertainty arising from both stochasticity and model mismatch is impacted by contacts with obstacles (e.g., when navigating through a cluttered environment or inserting a part into an assembly). We propose Calibrated Particle-sets for Trans-dimensional Uncertainty Representation (CaPTURe), a geometry-aware, conformal prediction-based algorithm that generates probabilistically valid prediction regions of the unknown future system configuration using particle-based models of arbitrary fidelity. While calibrated uncertainty predictions are essential for safe and efficient planning, analytical or learned motion models are often inaccurate - due to limited data, simplifying assumptions, unmodeled effects, etc. - which can lead to unsafe executions or task failure. Additionally, when a robot contacts an obstacle, the distribution of its future configurations can become multimodal or disjoint, or lie along manifolds of lower intrinsic dimension than the space of possible robot configurations. Our method uses a calibration dataset of system transitions to locally calibrate motion uncertainty estimates, constructing regions guaranteed to contain the future robot configuration at a user-set probability. Our calibration procedure captures how motion uncertainty varies between contact-rich and contactless motions, leading to sufficient coverage in both cases. We evaluate our method on two simulated planning tasks: controlling a marble around a labyrinth and performing tight-tolerance peg-in-hole insertion with a manipulator. Compared to relevant baselines, CaPTURe achieves the user-specified coverage requirement both in and out of contact and achieves up to a 30% absolute improvement in task success rate over the best baseline.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Statistics on stratified spaces , copyright =
Feragen, Aasa and Nye, Tom , year =. Statistics on stratified spaces , copyright =. Riemannian. doi:10.1016/B978-0-12-814725-2.00016-9 , keywords =
-
[2]
Computational Geometry , author =
Computing the geodesic centers of a polygonal domain , volume =. Computational Geometry , author =. 2019 , pages =. doi:10.1016/j.comgeo.2015.10.009 , abstract =
-
[3]
Revista Matemática Complutense , author =
Distance géodésique sur un sous-analytique , volume =. Revista Matemática Complutense , author =. 1997 , pages =. doi:10.5209/rev_REMA.1997.v10.17357 , language =
-
[4]
Harmati, Istv. On fitted stratified and semi-stratified geometric manipulation planning with fingertip relocations , journal =
-
[5]
Control of stratified systems with robotic applications , copyright =
Goodwine, John William , year =. Control of stratified systems with robotic applications , copyright =
-
[6]
Goodwine, B. and Burdick, J. , month = apr, year =. Trajectory generation for kinematic legged robots , volume =. Proceedings of. doi:10.1109/ROBOT.1997.619367 , abstract =
-
[7]
Vendittelli, Marilena and Laumond, Jean-Paul and Mishra, Bud , editor =. Decidability of. Algorithmic. 2015 , doi =
work page 2015
- [8]
Show all 54 references
-
[9]
Autonomous Robots , author =
Decidability in robot manipulation planning , volume =. Autonomous Robots , author =. 2021 , pages =. doi:10.1007/s10514-020-09957-2 , abstract =
2021 doi
-
[10]
and Burdick, J
Goodwine, B. and Burdick, J. W. , title =. IEEE Transactions on Robotics and Automation , volume =
-
[11]
and Rudolph, L
Han, L. and Rudolph, L. and Blumenthal, J. and Valodzin, I. , editor =. Stratified. Algorithmic. 2008 , doi =
2008
-
[12]
Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS) , series =
Wang, Zhendong and Gao, Ruijiang and Yin, Mingzhang and Zhou, Mingyuan and Blei, David , title =. Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS) , series =
-
[13]
and Klingensmith, Matthew and Srinivasa, Siddhartha S
Koval, Michael C. and Klingensmith, Matthew and Srinivasa, Siddhartha S. and Pollard, Nancy S. and Kaess, Michael , month = may, year =. The manifold particle filter for state estimation on high-dimensional implicit manifolds , url =. 2017. doi:10.1109/ICRA.2017.7989543 , abstract =
2017
-
[14]
luis-marques/awesome-stratified-spaces , url =
-
[15]
Machine Learning: ECML 2002 , series =
Papadopoulos, Harris and Proedrou, Kostas and Vovk, Volodya and Gammerman, Alex , title =. Machine Learning: ECML 2002 , series =
2002
-
[16]
Vovk, Vladimir and Lindsay, David and Nouretdinov, Ilia and Gammerman, Alex , title =
-
[17]
Bias reduction through conditional conformal prediction , journal =
L. Bias reduction through conditional conformal prediction , journal =
-
[18]
Proceedings of the Ninth Symposium on Conformal and Probabilistic Prediction and Applications , series =
Bostr\"om, Henrik and Johansson, Ulf , title =. Proceedings of the Ninth Symposium on Conformal and Probabilistic Prediction and Applications , series =
-
[19]
Erdmann, M. A. and Mason, M. T. , title =. IEEE Journal on Robotics and Automation , volume =
-
[20]
Science Robotics , volume =
Rodriguez, Alberto , title =. Science Robotics , volume =
-
[21]
Local conformal calibration of dynamics uncertainty from semantic images , journal =
Marques, Lu. Local conformal calibration of dynamics uncertainty from semantic images , journal =
-
[22]
and Boots, Byron and Theodorou, Evangelos A
Williams, Grady and Wagener, Nolan and Goldfain, Brian and Drews, Paul and Rehg, James M. and Boots, Byron and Theodorou, Evangelos A. , title =. IEEE International Conference on Robotics and Automation (ICRA) , pages =
-
[23]
Quantifying aleatoric and epistemic dynamics uncertainty via local conformal calibration , booktitle =
Marques, Lu. Quantifying aleatoric and epistemic dynamics uncertainty via local conformal calibration , booktitle =
- [24]
- [25]
-
[26]
arXiv preprint arXiv:2005.06095 , year =
Kuchibhotla, Arun Kumar , title =. arXiv preprint arXiv:2005.06095 , year =
2005 arXiv
-
[27]
and Barber, Rina Foygel and Bates, Stephen , title =
Angelopoulos, Anastasios N. and Barber, Rina Foygel and Bates, Stephen , title =. arXiv preprint arXiv:2411.11824 , year =
-
[28]
IEEE Transactions on Automatic Control , author =
Robust. IEEE Transactions on Automatic Control , author =. 2022 , pages =. doi:10.1109/TAC.2021.3059156 , abstract =
2022
-
[29]
2022 , publisher =
Vladimir Vovk and Alexander Gammerman and Glenn Shafer , title =. 2022 , publisher =
2022
-
[30]
and Wasserman, Larry , title =
Lei, Jing and G'Sell, Max and Rinaldo, Alessandro and Tibshirani, Ryan J. and Wasserman, Larry , title =. Journal of the American Statistical Association , volume =
-
[31]
Cabezas, Luben M. C. and Otto, Mateus P. and Izbicki, Rafael and Stern, Rafael B. , title =. Information Sciences , volume =
-
[32]
RSS , year=
Robust safety under stochastic uncertainty with discrete-time control barrier functions , author=. RSS , year=
-
[33]
Rao, A. S. and Goldberg, K. Y. , title =. IEEE Transactions on Robotics and Automation , volume =
-
[34]
and Ayanian, Nora and Lindemann, Lars , title =
Strawn, Kegan J. and Ayanian, Nora and Lindemann, Lars , title =. IEEE Robotics and Automation Letters , volume =
-
[35]
The International Journal of Robotics Research , volume =
Luo, Rachel and Zhao, Shengjia and Kuck, Jonathan and Ivanovic, Boris and Savarese, Silvio and Schmerling, Edward and Pavone, Marco , title =. The International Journal of Robotics Research , volume =
-
[36]
and Mangharam, Rahul and Lindemann, Lars , title =
Yang, Shuo and Pappas, George J. and Mangharam, Rahul and Lindemann, Lars , title =. IEEE Conference on Decision and Control (CDC) , pages =
-
[37]
, title =
Lindemann, Lars and Cleaveland, Matthew and Shim, Gihyun and Pappas, George J. , title =. IEEE Robotics and Automation Letters , volume =
-
[38]
and Cleaveland, Matthew and Pappas, George J
Dixit, Anushri and Lindemann, Lars and Wei, Skylar X. and Cleaveland, Matthew and Pappas, George J. and Burdick, Joel W. , title =. Proceedings of the 5th Annual Learning for Dynamics and Control Conference (L4DC) , series =
-
[39]
Algorithmic Foundations of Robotics XVI, Volume 1 , pages =
Huang, Zhe and Ji, Tianchen and Zhang, Heling and Pouria, Fatemeh Cheraghi and Driggs-Campbell, Katherine and Dong, Roy , title =. Algorithmic Foundations of Robotics XVI, Volume 1 , pages =
-
[40]
Advances in Neural Information Processing Systems (NeurIPS) , volume =
Chua, Kurtland and Calandra, Roberto and McAllister, Rowan and Levine, Sergey , title =. Advances in Neural Information Processing Systems (NeurIPS) , volume =
-
[41]
and Schoellig, Angela P
McKinnon, Christopher D. and Schoellig, Angela P. , title =. European Control Conference (ECC) , pages =
-
[42]
and Feiten, Wendelin and von Wichert, Georg and Burgard, Wolfram , title =
Wirnshofer, Florian and Schmitt, Philipp S. and Feiten, Wendelin and von Wichert, Georg and Burgard, Wolfram , title =. IEEE International Conference on Robotics and Automation (ICRA) , pages =
-
[43]
Algorithmic Foundations of Robotics XII , pages =
Phillips-Grafflin, Calder and Berenson, Dmitry , title =. Algorithmic Foundations of Robotics XII , pages =
-
[44]
Lies we can trust: Quantifying action uncertainty with inaccurate stochastic dynamics through conformalized nonholonomic
Marques, Lu. Lies we can trust: Quantifying action uncertainty with inaccurate stochastic dynamics through conformalized nonholonomic. IEEE Robotics and Automation Letters , volume =
-
[45]
Proceedings of Robotics: Science and Systems (RSS) , year =
Narang, Yashraj and Storey, Kier and Akinola, Iretiayo and Macklin, Miles and Reist, Philipp and Wawrzyniak, Lukasz and Guo, Yunrong and Moravanszky, Adam and State, Gavriel and Lu, Michelle and Handa, Ankur and Fox, Dieter , title =. Proceedings of Robotics: Science and Syste...
-
[46]
IEEE Conference on Decision and Control (CDC) , pages =
Gaber, Johannes and Bi, Thomas and D'Andrea, Raffaello , title =. IEEE Conference on Decision and Control (CDC) , pages =
-
[47]
Loftsgaarden, D. O. and Quesenberry, C. P. , title =. The Annals of Mathematical Statistics , volume =
-
[48]
Spatial planning: A configuration space approach , journal =
Lozano-P. Spatial planning: A configuration space approach , journal =
-
[49]
, title =
LaValle, Steven M. , title =
-
[50]
Kavraki, L. E. and. Probabilistic roadmaps for path planning in high-dimensional configuration spaces , journal =
-
[51]
and Ferguson, Dave and Kuffner, James J
Berenson, Dmitry and Srinivasa, Siddhartha S. and Ferguson, Dave and Kuffner, James J. , title =. IEEE International Conference on Robotics and Automation (ICRA) , pages =
-
[52]
Sampling-based motion planning on sequenced manifolds , booktitle =
Englert, Peter and Rayas Fern. Sampling-based motion planning on sequenced manifolds , booktitle =
-
[53]
Guyon and A
I. Guyon and A. Elisseeff. An Introduction to Variable and Feature Selection. JMLR
-
[54]
Guyon and C
I. Guyon and C. Aliferis and A. Elisseeff , title =
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.