REVIEW 4 major objections 4 minor 3 cited by
Interaction-aware Conformal Prediction for Crowd Navigation
T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read The paper introduces Interaction-aware Conformal Prediction (ICP), which alternates robot motion planning with conformal prediction on human trajectories simulated under the current plan, and proves a finite-sample, distribution-free…
desk verdict A genuinely new method for interaction-aware conformal calibration with a solid simulation evaluation, but the stated safety guarantee is vacuous and the sim-to-real transfer claim is unsupported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism that carries the argument is the closed loop between MPC and conformal prediction. The human simulator takes the latest MPC plan $\bar{x}_{r,k-1}$ and produces synthetic trajectories; the conformal predictor computes per-horizon nonconformity scores (Euclidean prediction errors), sorts them, and picks the quantile $r_{\mathrm{cp}}^\tau = e_{(\lceil(1-\alpha)NM\rceil)}$; MPC then enforces $\|\bar{x}_r^{t+\tau} - \hat{x}_{h,i}^{t+\tau}\|_2 \ge r_r + r_h + r_{\mathrm{cp}}^\tau$. The proof of Theorem 2 rests on the standard conformal lemma that for exchangeable scores the new score lies below the $q$-th order statistic with probability $q/(n+1)$, and the convergence assumption is what makes the execution-time error exchangeable with the calibration scores.
What would settle it
Run ICP in an environment where human motion is generated by a different mechanism than the ORCA simulator (for example, recorded pedestrian data or a learned policy), execute the plan that ICP outputs, and count the fraction of trials over many repetitions in which the robot stays at least $r_r + r_h$ from every human at all predicted steps; if that fraction is reliably below $1 - \alpha N T_{\mathrm{pred}}$, the exchangeability premise fails.
Extended reading notes
Core claim
On its own terms, ICP's central discovery is that the distribution shift caused by the robot's own motion can be folded into the calibration step by making the calibration dataset a function of the current plan. Starting from a plan computed with zero uncertainty radii, the algorithm runs a human simulator (ORCA) that assumes the robot executes that plan, collects the induced trajectory prediction errors, and sets each horizon-step radius to the $\lceil(1-\alpha) N M\rceil$-th order statistic of those errors. Replanning with these radii and repeating produces a fixed point at which the plan's simulated reactions coincide with the reactions used for calibration; under this convergence and MPC feasibility, Theorem 2 gives $\Pr(\|\bar{x}_r^{t+\tau} - x_{h,i}^{t+\tau}\|_2 \geq r_r + r_h,\ \forall \tau\in [T_{\mathrm{pred}}],\ \forall i\in[N]) \geq 1-\alpha N T_{\mathrm{pred}}$. The guarantee is conditional: it holds when the execution-time errors are exchangeable with the simulated calibration errors, a premise the paper does not prove for real humans.
Load-bearing premise
The guarantee rests on the assumption that the way real people move when the robot follows its final plan is statistically interchangeable with the way the simulator moves them under that same plan; if the simulator mispredicts human behavior, the coverage bound does not transfer.
Editorial extensions
If this is right
- Any trajectory predictor can be plugged into ICP without retraining, because conformal prediction treats the predictor as a black box; the paper demonstrates this with a pre-trained Gumbel Social Transformer.
- ICP avoids the lengthy warm-up of online adaptive conformal prediction: because calibration data come from an online simulator conditioned on the current plan, even a single iteration reaches coverage above the lower bound in the reported experiments.
- The failure probability scales as $\alpha N T_{\mathrm{pred}}$, which makes the bound degrade with crowd size and prediction horizon; keeping a target safety level therefore requires shrinking $\alpha$ as the scene gets denser.
- The approach runs fast enough for real-time use: the paper reports execution within the 0.25 s (single-step) or 1.25 s (prediction-step) schemes on a commercial GPU.
- The number of iterations has little effect in the experiments (1, 3, and 10 iterations perform comparably), indicating that the plan-simulation loop converges quickly in practice.
Reading between the lines
- Editorial inference: the same loop could be run with any forward model of human motion in place of ORCA; the guarantee would then certify that model's own reaction assumptions, giving a practical way to test a simulator against real-world data before deployment.
- Editorial inference: the union bound over $N$ and $T_{\mathrm{pred}}$ suggests a direct extension to heterogeneous failure budgets; the paper mentions adaptive $\alpha$ only as future work for infeasible MPC, not as a way to tighten the bound in dense crowds.
- Editorial inference: oscillations or growth of the calibrated radii across iterations could be used as a run-time alarm for sim-to-real mismatch, since a stable fixed point is exactly what Theorem 2's convergence assumption demands.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Interaction-aware Conformal Prediction (ICP), an iterative planning-and-calibration loop for crowd navigation. At each control cycle, ICP generates a nominal MPC plan, simulates human responses to that plan with an ORCA-based human simulator, computes conformal prediction radii from the simulated calibration data, and replans with these radii as safety margins. The central formal claim, Theorem 2, states that, under a convergence assumption, the executed plan keeps the robot at least r_r + r_h away from every human over the whole prediction horizon with probability at least 1 - alpha * N * Tpred. The empirical section compares ICP with OffCP, ACP, and ORCA in simulated crowd navigation across different crowd densities, reporting coverage rates, navigation time, intrusion ratio, and success rates.
Significance. If the formal guarantee were correct in the stated form, the paper would make a useful contribution: it addresses the dependence of human motion uncertainty on the robot's own plan, which is a real limitation of offline conformal prediction for interactive navigation. The iterative closed-loop calibration idea is clear and the empirical evaluation is fairly extensive, with code released. The paper also makes an honest attempt to discuss infeasible MPC and the sim-to-real gap. However, the advertised safety guarantee currently has several load-bearing technical problems: the conformal quantile index is wrong, the union-bound guarantee is vacuous for the tested parameters, and the theorem's transfer from the simulated calibration distribution to execution is not established. The empirical coverage rates are consistent with the algorithm's behavior inside the ORCA simulator, but they do not by themselves validate a real-world safety claim.
major comments (4)
- [Sec. 4.2, Eq. (5)-(6)] The quantile index in Eq. (5) is inconsistent with Theorem 1. With n = N*M calibration scores and one test score, Theorem 1 requires q = ceil((n+1)(1-alpha)), but Eq. (5) uses q = ceil((1-alpha) n). For finite n this gives q/(n+1) < 1-alpha in general, so the claimed coverage probability in Eq. (6) is not guaranteed. For example, with N=10, M=8, alpha=0.05, the stated index gives coverage about 76/81 = 0.938, not 0.95. This error propagates into the per-event step of Theorem 2's proof and into Eq. (9).
- [Theorem 2 and Sec. 5.1] The bound in Theorem 2 is vacuous for the configurations actually tested. With alpha=0.05, N=10 humans, and Tpred=5, the claimed lower bound is 1 - 0.05*10*5 = -1.5. The proof's union bound only gives a nontrivial statement when alpha*N*Tpred < 1, which is not the case in Table 1. The text in Sec. 5.1 states a lower bound of 0.75 from Eq. (9), but that bound applies to a single human over the horizon, not to all N humans simultaneously; for the all-human event the bound is negative. The theorem should either use alpha/(N*Tpred) as the per-event failure probability or construct a simultaneous conformal set for the whole prediction window.
- [Algorithm 1 and Theorem 2] Theorem 2 assumes that Algorithm 1 converges, but the implementation runs a fixed number K of iterations (K=1, 3, 10 in Table 1) with no convergence check. In particular, for K=1, the calibration data are collected under the nominal zero-radius plan xbar_{r,0}, whereas the executed plan is xbar_{r,1}; the theorem's convergence assumption is not verified for any reported configuration. As a result, the formal safety guarantee does not apply to the configurations whose performance is the main empirical evidence. The authors should either provide a convergence criterion and check it, or prove a guarantee that holds after a fixed number of iterations.
- [Sec. 4.2, Sec. 5.1, Sec. 6] The finite-sample guarantee in Theorem 2 transfers to real-world execution only if the ORCA-generated calibration scores are exchangeable with the human trajectory prediction errors observed when the final plan is executed. This is a distributional condition, not an iteration-count condition. The paper does not establish it; the evaluation in Sec. 5.1 uses ORCA for both calibration and test, so the high coverage rates largely confirm self-consistency within the simulator. The Limitations paragraph says the sim-to-real gap would 'require more iterations for better performance,' but additional ICP iterations cannot repair a distribution mismatch between the simulator and the real crowd. The theorem and the abstract's real-world claims should be qualified to state explicitly that the guarantee is conditional on this exchangeability, or real-world validation should be provided.
minor comments (4)
- [Eq. (8)] The coverage indicator in Eq. (8) uses the strict inequality ||xhat - x||_2 < r_cp, whereas the conformal set in Theorem 1 and Eq. (6) uses <=. These should be made consistent to avoid a mismatch between the theoretical coverage and the reported metric.
- [Sec. 5.2, Fig. 3 caption] The sentence 'the top row of Fig. 3 indicates the coverage rate of ACP-W is in the SSE scheme is better than in the PSE scheme' has a grammatical error and should be rewritten for clarity.
- [Sec. 5.2, paragraph on runtime] The claim that ICP can run in real time is not supported by a table or quantitative runtime numbers; adding the measured planning time per iteration would make the practicality claim verifiable.
- [Sec. 6] The limitations paragraph correctly notes that cached plans beyond Tpred steps void the safety guarantee, but it does not mention that the Theorem 2 bound is already vacuous for the tested N and Tpred; this should be acknowledged when stating the contribution.
Circularity Check
No circular derivation: the conformal guarantee is conditional on explicit exchangeability and convergence assumptions; the ORCA-based evaluation is in-distribution but that is a limitation, not a circular step.
full rationale
The derivation chain is the standard split-conformal argument: Theorem 1 is the textbook exchangeability quantile bound, Eq. (5) defines each radius as a calibration quantile, Eq. (6) applies exchangeability to a fresh error, and Theorem 2 adds a union bound under the explicit assumption that Algorithm 1 has converged. No step defines its output as its input: the conformal radius is not the coverage frequency, and the coverage statement is a probabilistic consequence of exchangeability, not an algebraic identity. The paper does rely on ORCA as both the calibration-time human simulator (Section 4.2) and the test-time human motion generator (Section 5.1), and the Limitations section explicitly states: 'ORCA is used both in the human simulator of ICP algorithm and for generating human motion in test scenarios.' This makes the reported coverage rates in-distribution checks of the conformal argument, not external evidence for real-world transfer; however, Theorem 2 is explicitly conditional on exchangeability and convergence, so this is a limitation of external validity, not circularity. Self-citations to prior work for ORCA transferability and for the GST predictor are not load-bearing for the theorem's derivation. (A separate correctness concern exists: Eq. (6) uses quantile index ceil((1-alpha)NM) rather than the ceil((NM+1)(1-alpha)) that Theorem 1 requires; that is a mathematical flaw, not a circularity.)
Assumptions & free parameters
free parameters (3)
- MPC cost weights (omega_g, omega_v, omega_reg) =
1, 5, 0.5
- Calibration size (CS, number of simulation episodes) =
8 for default ICP1, tested 2-64
- Number of ICP iterations K =
3 for default, tested 1, 3, 10
assumptions (4)
- domain assumption Exchangeability of calibration errors in E^tau
- domain assumption ORCA simulator faithfully models real human motion
- domain assumption Algorithm 1 converges and MPC remains feasible
- standard math Standard conformal prediction result (Theorem 1)
Cite this review
Pith. "Pith review of Interaction-aware Conformal Prediction for Crowd Navigation." pith.science (2026). https://pith.science/paper/7ZCHB5J6
@misc{pith2026250206221,
author = {Pith},
title = {Pith review of: Interaction-aware Conformal Prediction for Crowd Navigation},
year = {2026},
howpublished = {\url{https://pith.science/paper/7ZCHB5J6}},
note = {Machine review of arXiv:2502.06221}
}
read the original abstract
During crowd navigation, robot motion plan needs to consider human motion uncertainty, and the human motion uncertainty is dependent on the robot motion plan. We introduce Interaction-aware Conformal Prediction (ICP) to alternate uncertainty-aware robot motion planning and decision-dependent human motion uncertainty quantification. ICP is composed of a trajectory predictor to predict human trajectories, a model predictive controller to plan robot motion with confidence interval radii added for probabilistic safety, a human simulator to collect human trajectory calibration dataset conditioned on the planned robot motion, and a conformal prediction module to quantify trajectory prediction error on the decision-dependent calibration dataset. Crowd navigation simulation experiments show that ICP strikes a good balance of performance among navigation efficiency, social awareness, and uncertainty quantification compared to previous works. ICP generalizes well to navigation tasks under various crowd densities. The fast runtime and efficient memory usage make ICP practical for real-world applications. Code is available at https://github.com/tedhuang96/icp.
Figures
Figures from the paper (1 more)
Forward citations
Cited by 3 Pith papers
-
CoCoNav: Conformal Control for Safe Robot Navigation in Crowds
CoCoNav combines horizon-specific conformal PI calibration with a relax-then-verify MPC planner to provide runtime-certified crowd navigation.
-
Navigating the Proximity-Safety Balance: Constraint Decomposition for Human Following in Pedestrian Crowds
A constrained reinforcement learning framework with separate proximity and safety cost limits, plus conformal prediction uncertainty margins, improves human-following robots in dense crowds.
-
A Lightweight Crowd Model for Robot Social Navigation
A single-layer ConvRNN crowd predictor matches or beats a deep baseline's accuracy on mall pedestrian data while running 3.6x faster and giving more reliable navigation costs.
Reference graph
Works this paper leans on
-
[1]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., Savarese, S.: Social lstm: Human trajectory prediction in crowded spaces. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 961–971 (2016)
work page 2016
-
[2]
Angelopoulos, A.N., Bates, S.: A gentle introduction to conformal prediction and distribution-free uncertainty quantification (2022)
work page 2022
-
[3]
In: Robust- ness in identification and control, pp
Bemporad, A., Morari, M.: Robust model predictive control: A survey. In: Robust- ness in identification and control, pp. 207–226. Springer (2007) Interaction-aware Conformal Prediction for Crowd Navigation 17
work page 2007
-
[4]
In: 2019 in- ternational conference on robotics and automation (ICRA)
Chen, C., Liu, Y., Kreiss, S., Alahi, A.: Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforcement learning. In: 2019 in- ternational conference on robotics and automation (ICRA). pp. 6015–6022. IEEE (2019)
work page 2019
-
[5]
IEEE Transactions on Systems, Man, and Cybernetics: Systems 52(4), 2289–2301 (2021)
Chen, Y., Zhao, F., Lou, Y.: Interactive model predictive control for robot nav- igation in dense crowds. IEEE Transactions on Systems, Man, and Cybernetics: Systems 52(4), 2289–2301 (2021)
work page 2021
-
[6]
Dietterich, T.G., Hostetler, J.: Conformal prediction intervals for markov decision process trajectories (2022)
work page 2022
-
[7]
In: Learning for Dynamics and Control Conference
Dixit, A., Lindemann, L., Wei, S.X., Cleaveland, M., Pappas, G.J., Burdick, J.W.: Adaptive conformal prediction for motion planning among dynamic agents. In: Learning for Dynamics and Control Conference. pp. 300–314. PMLR (2023)
work page 2023
-
[8]
IEEE Transactions on Robotics28(1), 101–115 (2011)
Du Toit, N.E., Burdick, J.W.: Robot motion planning in dynamic, uncertain envi- ronments. IEEE Transactions on Robotics28(1), 101–115 (2011)
work page 2011
Show all 33 references
-
[9]
Automatica25(3), 335–348 (1989)
Garcia, C.E., Prett, D.M., Morari, M.: Model predictive control: Theory and prac- tice—a survey. Automatica25(3), 335–348 (1989)
1989
-
[10]
IEEE Transactions on Systems Science and Cyber- netics 4(2), 100–107 (1968)
Hart, P.E., Nilsson, N.J., Raphael, B.: A formal basis for the heuristic determina- tion of minimum cost paths. IEEE Transactions on Systems Science and Cyber- netics 4(2), 100–107 (1968)
1968
-
[11]
Physical re- view E 51(5), 4282 (1995)
Helbing, D., Molnar, P.: Social force model for pedestrian dynamics. Physical re- view E 51(5), 4282 (1995)
1995
-
[12]
arXiv preprint arXiv:2309.14595 (2023)
Huang, Z., Chen, H., Pohovey, J., Driggs-Campbell, K.: Neural informed rrt*: Learning-based path planning with point cloud state representations under admis- sible ellipsoidal constraints. arXiv preprint arXiv:2309.14595 (2023)
2023 arXiv
-
[13]
IEEE Robotics and Automation Letters7(2), 1198–1205 (2022).https://doi.org/10.1109/LRA
Huang, Z., Li, R., Shin, K., Driggs-Campbell, K.: Learning sparse interaction graphs of partially detected pedestrians for trajectory prediction. IEEE Robotics and Automation Letters7(2), 1198–1205 (2022).https://doi.org/10.1109/LRA. 2021.3138547
2022
-
[14]
Incremona, G.P., Ferrara, A., Magni, L.: Mpc for robot manipulators with integral slidingmodesgeneration.IEEE/ASMETransactionsonMechatronics 22(3),1299– 1307 (2017)
2017
-
[15]
In: 2022 IEEE 61st Conference on De- cision and Control (CDC)
Ji, T., Geng, J., Driggs-Campbell, K.: Robust output feedback mpc with reduced conservatism under ellipsoidal uncertainty. In: 2022 IEEE 61st Conference on De- cision and Control (CDC). pp. 1782–1789. IEEE (2022)
2022
-
[16]
In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Kamel, M., Alonso-Mora, J., Siegwart, R., Nieto, J.: Robust collision avoidance for multiple micro aerial vehicles using nonlinear model predictive control. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). pp. 236–243. IEEE (2017)
2017
-
[17]
Karaman, S., Frazzoli, E.: Sampling-based algorithms for optimal motion planning 30(7), 846–894 (2011)
2011
-
[18]
International Journal of Fore- casting 37(2), 777–799 (2021)
Kath, C., Ziel, F.: Conformal prediction interval estimation and applications to day-ahead and intraday power markets. International Journal of Fore- casting 37(2), 777–799 (2021). https://doi.org/https://doi.org/10.1016/j. ijforecast.2020.09.006, https://www.sciencedirect.com...
2021 doi
-
[19]
Proceedings of Machine Learning Research vol vvv1, 16 (2024)
Lin, A., Bansal, S.: Verification of neural reachable tubes via scenario optimization and conformal prediction. Proceedings of Machine Learning Research vol vvv1, 16 (2024)
2024
-
[20]
IEEE Robotics and Automation Letters 8(8), 5116–5123 (2023).https://doi.org/10.1109/LRA.2023.3292071 18 Z
Lindemann, L., Cleaveland, M., Shim, G., Pappas, G.J.: Safe planning in dynamic environments using conformal prediction. IEEE Robotics and Automation Letters 8(8), 5116–5123 (2023).https://doi.org/10.1109/LRA.2023.3292071 18 Z. Huang et al
2023
-
[21]
In: 2023 IEEE International Conference on Robotics and Automation (ICRA)
Liu, S., Chang, P., Huang, Z., Chakraborty, N., Hong, K., Liang, W., McPher- son, D.L., Geng, J., Driggs-Campbell, K.: Intention aware robot crowd navigation with attention-based interaction graph. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). pp. 1...
2023
-
[22]
In: 2021 IEEE international conference on robotics and automation (ICRA)
Liu,S.,Chang,P.,Liang,W.,Chakraborty,N.,Driggs-Campbell,K.:Decentralized structural-rnn for robot crowd navigation with deep reinforcement learning. In: 2021 IEEE international conference on robotics and automation (ICRA). pp. 3517–
2021
-
[23]
In: 2023 62nd IEEE Conference on Decision and Control (CDC)
Muthali, A., Shen, H., Deglurkar, S., Lim, M.H., Roelofs, R., Faust, A., Tomlin, C.: Multi-agent reachability calibration with conformal prediction. In: 2023 62nd IEEE Conference on Decision and Control (CDC). pp. 6596–6603 (2023).https: //doi.org/10.1109/CDC49753.2023.10383723
2023
-
[24]
Nature communications13(1), 7761 (2022)
Olsson, H., Kartasalo, K., Mulliqi, N., Capuccini, M., Ruusuvuori, P., Sama- ratunga, H., Delahunt, B., Lindskog, C., Janssen, E.A., Blilie, A., et al.: Estimating diagnostic uncertainty in artificial intelligence assisted pathology using conformal prediction. Nature communica...
2022
-
[25]
In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems
Park, J.J., Johnson, C., Kuipers, B.: Robot navigation with model predictive equi- librium point control. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. pp. 4945–4952. IEEE (2012)
2012
-
[26]
Control engineering practice11(7), 733–764 (2003)
Qin, S.J., Badgwell, T.A.: A survey of industrial model predictive control technol- ogy. Control engineering practice11(7), 733–764 (2003)
2003
-
[27]
arXiv preprint arXiv:2107.02792 (2021)
Sivakumar, A.N., Modi, S., Gasparino, M.V., Ellis, C., Velasquez, A.E.B., Chowd- hary,G.,Gupta,S.:Learnedvisualnavigationforunder-canopyagriculturalrobots. arXiv preprint arXiv:2107.02792 (2021)
2021 arXiv
-
[28]
In: Neural Information Processing Systems (2023), https://api.semanticscholar
Sun, J., Jiang, Y., Qiu, J., Nobel, P., Kochenderfer, M.J., Schwager, M.: Confor- mal prediction for uncertainty-aware planning with diffusion dynamics model. In: Neural Information Processing Systems (2023), https://api.semanticscholar. org/CorpusID:268095750
2023
-
[29]
In: Robotics Research: The 14th International Symposium ISRR
Van Den Berg, J., Guy, S.J., Lin, M., Manocha, D.: Reciprocal n-body collision avoidance. In: Robotics Research: The 14th International Symposium ISRR. pp. 3–19. Springer (2011)
2011
-
[30]
Journal of Healthcare Informatics Research6(3), 241–252 (2022)
Vazquez, J., Facelli, J.C.: Conformal prediction in clinical medical sciences. Journal of Healthcare Informatics Research6(3), 241–252 (2022)
2022
-
[31]
Springer Science & Business Media (2005)
Vovk, V., Gammerman, A., Shafer, G.: Algorithmic learning in a random world. Springer Science & Business Media (2005)
2005
-
[32]
In: Gammerman, A., Vovk, V., Luo, Z., Smirnov, E., Cherubin, G
Wisniewski, W., Lindsay, D., Lindsay, S.: Application of conformal prediction in- terval estimations to market makers’ net positions. In: Gammerman, A., Vovk, V., Luo, Z., Smirnov, E., Cherubin, G. (eds.) Proceedings of the Ninth Symposium on Conformal and Probabilistic Predic...
2020
-
[33]
Zhang, H., Ratliff, L.J., Dong, R.: Distribution-free guarantees for systems with decision-dependent noise (2024)
2024
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.