{"id":"71598df0-1831-44c3-a4a7-84477454acfe","arxiv_id":"2502.06221","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"ICP alternates robot motion planning with conformal prediction on human trajectories simulated under the current plan, improving navigation time and uncertainty coverage in simulated crowds.","lead":"Interaction-aware Conformal Prediction (ICP) repeatedly replans a robot's path and recalibrates uncertainty bounds on human motion by simulating how people react to each new plan. In simulated crowd navigation, it achieves lower navigation time and higher coverage of human trajectories than prior conformal-prediction baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central safety guarantee only holds if the ORCA simulator is exchangeable with deployment-time human motion; the paper acknowledges a sim-to-real gap but not that it voids Theorem 2's transfer.","rationale":"The paper's central contribution is a finite-sample safety guarantee that accounts for robot-human interaction. The theorem's proof is a direct application of conformal prediction; the only route from calibration data to execution-time coverage is the assumption that the nonconformity scores in D_K and the score of the executed trajectory are exchangeable. This is not a modeling detail but the very condition under which the guarantee exists. The paper never states that ORCA must be an accurate model of real human behavior for the theorem to apply; instead, the Limitations section frames the sim-to-real gap as an iteration-count issue. That frame is misleading: more iterations re-calibrate against the same ORCA distribution and cannot correct a distribution mismatch. Because the experiments use ORCA for both calibration and test, the reported CR values cannot validate the real-world claim. I agree with the reader's weakest_assumption; the off-by-one quantile and vacuous union bound are real but secondary, since they are fixable and do not undermine the algorithm's utility in simulation. The exchangeability gap is structural: without either a real-world evaluation or a stated scope restriction to simulation, the headline safety guarantee is unsupported for the intended deployment domain.","tokens_in":13054,"tokens_out":8745,"duration_ms":86836,"concrete_test":"Run the ICP pipeline with calibration data generated by ORCA as in the paper, but replace the execution-time human model with an independent crowd simulator (e.g., Social Force) or a real pedestrian dataset with the same initial/goal distributions, and report per-step/per-horizon coverage rates. If the empirical CR falls below the nominal 1-alpha (or the per-horizon bound), the exchangeability assumption fails and Theorem 2 does not transfer to real deployment.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing condition is that the ORCA-based human simulator used for calibration produces scores exchangeable with the scores observed when the final plan is executed. Theorem 2 inherits this from Theorem 1: without exchangeability between D_K and the execution-time trajectory errors, the rank argument in Eq. (6) gives no finite-sample coverage. The paper's own Limitations paragraph (\"sim-to-real gap... require more iterations\") mischaracterizes this: increasing the number of iterations cannot repair a distribution mismatch, because conformal prediction is distribution-free only within an exchangeable calibration/test pair. The simulation evaluation cannot detect this, since the same ORCA model generates both calibration and test data (Section 5.1), so the guarantee is self-referential. As stated, the theorem applies only to the simulated world, not to real crowds.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Interaction-aware Conformal Prediction (ICP), an iterative planning-and-calibration loop for crowd navigation. At each control cycle, ICP generates a nominal MPC plan, simulates human responses to that plan with an ORCA-based human simulator, computes conformal prediction radii from the simulated calibration data, and replans with these radii as safety margins. The central formal claim, Theorem 2, states that, under a convergence assumption, the executed plan keeps the robot at least r_r + r_h away from every human over the whole prediction horizon with probability at least 1 - alpha * N * Tpred. The empirical section compares ICP with OffCP, ACP, and ORCA in simulated crowd navigation across different crowd densities, reporting coverage rates, navigation time, intrusion ratio, and success rates.","tokens_in":13017,"tokens_out":6079,"duration_ms":60828,"significance":"If the formal guarantee were correct in the stated form, the paper would make a useful contribution: it addresses the dependence of human motion uncertainty on the robot's own plan, which is a real limitation of offline conformal prediction for interactive navigation. The iterative closed-loop calibration idea is clear and the empirical evaluation is fairly extensive, with code released. The paper also makes an honest attempt to discuss infeasible MPC and the sim-to-real gap. However, the advertised safety guarantee currently has several load-bearing technical problems: the conformal quantile index is wrong, the union-bound guarantee is vacuous for the tested parameters, and the theorem's transfer from the simulated calibration distribution to execution is not established. The empirical coverage rates are consistent with the algorithm's behavior inside the ORCA simulator, but they do not by themselves validate a real-world safety claim.","major_comments":[{"comment":"The quantile index in Eq. (5) is inconsistent with Theorem 1. With n = N*M calibration scores and one test score, Theorem 1 requires q = ceil((n+1)(1-alpha)), but Eq. (5) uses q = ceil((1-alpha) n). For finite n this gives q/(n+1) < 1-alpha in general, so the claimed coverage probability in Eq. (6) is not guaranteed. For example, with N=10, M=8, alpha=0.05, the stated index gives coverage about 76/81 = 0.938, not 0.95. This error propagates into the per-event step of Theorem 2's proof and into Eq. (9).","section":"Sec. 4.2, Eq. (5)-(6)"},{"comment":"The bound in Theorem 2 is vacuous for the configurations actually tested. With alpha=0.05, N=10 humans, and Tpred=5, the claimed lower bound is 1 - 0.05*10*5 = -1.5. The proof's union bound only gives a nontrivial statement when alpha*N*Tpred < 1, which is not the case in Table 1. The text in Sec. 5.1 states a lower bound of 0.75 from Eq. (9), but that bound applies to a single human over the horizon, not to all N humans simultaneously; for the all-human event the bound is negative. The theorem should either use alpha/(N*Tpred) as the per-event failure probability or construct a simultaneous conformal set for the whole prediction window.","section":"Theorem 2 and Sec. 5.1"},{"comment":"Theorem 2 assumes that Algorithm 1 converges, but the implementation runs a fixed number K of iterations (K=1, 3, 10 in Table 1) with no convergence check. In particular, for K=1, the calibration data are collected under the nominal zero-radius plan xbar_{r,0}, whereas the executed plan is xbar_{r,1}; the theorem's convergence assumption is not verified for any reported configuration. As a result, the formal safety guarantee does not apply to the configurations whose performance is the main empirical evidence. The authors should either provide a convergence criterion and check it, or prove a guarantee that holds after a fixed number of iterations.","section":"Algorithm 1 and Theorem 2"},{"comment":"The finite-sample guarantee in Theorem 2 transfers to real-world execution only if the ORCA-generated calibration scores are exchangeable with the human trajectory prediction errors observed when the final plan is executed. This is a distributional condition, not an iteration-count condition. The paper does not establish it; the evaluation in Sec. 5.1 uses ORCA for both calibration and test, so the high coverage rates largely confirm self-consistency within the simulator. The Limitations paragraph says the sim-to-real gap would 'require more iterations for better performance,' but additional ICP iterations cannot repair a distribution mismatch between the simulator and the real crowd. The theorem and the abstract's real-world claims should be qualified to state explicitly that the guarantee is conditional on this exchangeability, or real-world validation should be provided.","section":"Sec. 4.2, Sec. 5.1, Sec. 6"}],"minor_comments":[{"comment":"The coverage indicator in Eq. (8) uses the strict inequality ||xhat - x||_2 < r_cp, whereas the conformal set in Theorem 1 and Eq. (6) uses <=. These should be made consistent to avoid a mismatch between the theoretical coverage and the reported metric.","section":"Eq. (8)"},{"comment":"The sentence 'the top row of Fig. 3 indicates the coverage rate of ACP-W is in the SSE scheme is better than in the PSE scheme' has a grammatical error and should be rewritten for clarity.","section":"Sec. 5.2, Fig. 3 caption"},{"comment":"The claim that ICP can run in real time is not supported by a table or quantitative runtime numbers; adding the measured planning time per iteration would make the practicality claim verifiable.","section":"Sec. 5.2, paragraph on runtime"},{"comment":"The limitations paragraph correctly notes that cached plans beyond Tpred steps void the safety guarantee, but it does not mention that the Theorem 2 bound is already vacuous for the tested N and Tpred; this should be acknowledged when stating the contribution.","section":"Sec. 6"}],"recommendation":"major_revision","confidential_remarks":"The paper's core algorithmic idea is promising and the experiments are reasonably thorough for a simulation study, but the formal safety guarantee is currently overstated in three independent ways: the quantile index error, the vacuous union bound, and the unverified transfer from the ORCA simulator to real human motion. These are fixable in revision by correcting the conformal quantile, revising the guarantee to a nontrivial bound or using simultaneous conformal prediction, and sharply delimiting the claim to the simulation setting or adding real-world data. I therefore recommend major revision rather than rejection, provided the authors are willing to restate the guarantees and scope honestly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely new idea for interaction-aware conformal calibration in crowd navigation, and the experiments back up the practical story. The iterative loop—simulate human responses to your latest robot plan, recalibrate conformal radii from those simulated trajectories, replan—directly addresses the distribution-shift problem that OffCP and ACP leave open. The writing is clear, the ablations are informative (calibration size, iteration count, execution scheme), and the code is public. The empirical core, evaluated within the simulator, is solid.\n\nThe formal claims need work. Theorem 2's union-bound guarantee is 1 − α N T_pred, which is −1.5 for N=10, α=0.05, T_pred=5: vacuous in the tested regime. The per-human horizon bound 1 − α T_pred used in the experiments is legitimate, but it doesn't support the theorem as written. Eq. (5)-(6) also has an off-by-one quantile index: with n = N×M calibration scores, you need ⌈(n+1)(1−α)⌉, not ⌈n(1−α)⌉, for the stated 1−α coverage. That is a small fix, but it matters for the proof.\n\nThe bigger caveat is about what the guarantee actually covers. Exchangeability holds relative to the ORCA-based simulator that produces the calibration set. So Theorem 2 is a simulation-world guarantee unless the simulator is the same distribution as deployment-time human motion. The paper does not state that as an assumption, and it sells the method as practical for real-world application without any transfer test. The Limitations section acknowledges a sim-to-real gap but says it would 'require more iterations' to improve. That is not right: conformal prediction is distribution-free only within an exchangeable calibration/test pair; more iterations of the same simulator cannot repair a distribution mismatch. The authors need to state the exchangeability assumption explicitly, correct the quantile formula, replace the vacuous bound with a meaningful one, and either delimit the real-world claims or validate them.\n\nNet: a solid, clearly written method paper with a novel algorithmic core and a reproducible evaluation. It deserves peer review—conditional on the authors fixing the guarantee statements and the transfer claim.","headline":"A genuinely new method for interaction-aware conformal calibration with a solid simulation evaluation, but the stated safety guarantee is vacuous and the sim-to-real transfer claim is unsupported.","tokens_in":13758,"tokens_out":3649,"would_cite":true,"duration_ms":31413,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces Interaction-aware Conformal Prediction (ICP), which alternates robot motion planning with conformal prediction on human trajectories simulated under the current plan, and proves a finite-sample, distribution-free…","keywords":["conformal prediction","crowd navigation","human-robot interaction","uncertainty quantification","model predictive control","probabilistic safety guarantee","trajectory prediction","decision-dependent uncertainty"],"falsifier":"Run ICP in an environment where human motion is generated by a different mechanism than the ORCA simulator (for example, recorded pedestrian data or a learned policy), execute the plan that ICP outputs, and count the fraction of trials over many repetitions in which the robot stays at least $r_r + r_h$ from every human at all predicted steps; if that fraction is reliably below $1 - \\alpha N T_{\\mathrm{pred}}$, the exchangeability premise fails.","tokens_in":12668,"feed_emoji":"🤖","tokens_out":12016,"duration_ms":93738,"temperature":0.7,"pith_summary":"Interaction-aware Conformal Prediction (ICP) claims to give a mobile robot a finite-sample, distribution-free safety guarantee in crowd navigation even though human motion uncertainty is not fixed but depends on the robot's own plan. The method alternates between planning with model-predictive control, simulating how humans would react to that plan, and recomputing conformal prediction radii from the simulations before planning again. Theorem 2 states that once this loop converges and the planning problem is feasible, the executed trajectory stays at least $r_r + r_h$ away from every human over the whole prediction horizon with probability at least $1 - \\alpha N T_{\\mathrm{pred}}$. Experiments in ORCA-based simulations report higher coverage than offline- and adaptive-conformal baselines at comparable navigation times, across crowds of 5 to 20 people. The paper argues this makes interaction-aware safety guarantees practical without requiring a long warm-up of real human-robot interaction data.","feed_headline":"Simulating human reactions gives crowd robots a safety guarantee","feed_subtitle":"Calibrating intervals on simulated reactions to each new plan keeps the guarantee valid without real-world data.","key_machinery":"The mechanism that carries the argument is the closed loop between MPC and conformal prediction. The human simulator takes the latest MPC plan $\\bar{x}_{r,k-1}$ and produces synthetic trajectories; the conformal predictor computes per-horizon nonconformity scores (Euclidean prediction errors), sorts them, and picks the quantile $r_{\\mathrm{cp}}^\\tau = e_{(\\lceil(1-\\alpha)NM\\rceil)}$; MPC then enforces $\\|\\bar{x}_r^{t+\\tau} - \\hat{x}_{h,i}^{t+\\tau}\\|_2 \\ge r_r + r_h + r_{\\mathrm{cp}}^\\tau$. The proof of Theorem 2 rests on the standard conformal lemma that for exchangeable scores the new score lies below the $q$-th order statistic with probability $q/(n+1)$, and the convergence assumption is what makes the execution-time error exchangeable with the calibration scores.","core_discovery":"On its own terms, ICP's central discovery is that the distribution shift caused by the robot's own motion can be folded into the calibration step by making the calibration dataset a function of the current plan. Starting from a plan computed with zero uncertainty radii, the algorithm runs a human simulator (ORCA) that assumes the robot executes that plan, collects the induced trajectory prediction errors, and sets each horizon-step radius to the $\\lceil(1-\\alpha) N M\\rceil$-th order statistic of those errors. Replanning with these radii and repeating produces a fixed point at which the plan's simulated reactions coincide with the reactions used for calibration; under this convergence and MPC feasibility, Theorem 2 gives $\\Pr(\\|\\bar{x}_r^{t+\\tau} - x_{h,i}^{t+\\tau}\\|_2 \\geq r_r + r_h,\\ \\forall \\tau\\in [T_{\\mathrm{pred}}],\\ \\forall i\\in[N]) \\geq 1-\\alpha N T_{\\mathrm{pred}}$. The guarantee is conditional: it holds when the execution-time errors are exchangeable with the simulated calibration errors, a premise the paper does not prove for real humans.","pith_inferences":["Editorial inference: the same loop could be run with any forward model of human motion in place of ORCA; the guarantee would then certify that model's own reaction assumptions, giving a practical way to test a simulator against real-world data before deployment.","Editorial inference: the union bound over $N$ and $T_{\\mathrm{pred}}$ suggests a direct extension to heterogeneous failure budgets; the paper mentions adaptive $\\alpha$ only as future work for infeasible MPC, not as a way to tighten the bound in dense crowds.","Editorial inference: oscillations or growth of the calibrated radii across iterations could be used as a run-time alarm for sim-to-real mismatch, since a stable fixed point is exactly what Theorem 2's convergence assumption demands."],"forward_implications":["Any trajectory predictor can be plugged into ICP without retraining, because conformal prediction treats the predictor as a black box; the paper demonstrates this with a pre-trained Gumbel Social Transformer.","ICP avoids the lengthy warm-up of online adaptive conformal prediction: because calibration data come from an online simulator conditioned on the current plan, even a single iteration reaches coverage above the lower bound in the reported experiments.","The failure probability scales as $\\alpha N T_{\\mathrm{pred}}$, which makes the bound degrade with crowd size and prediction horizon; keeping a target safety level therefore requires shrinking $\\alpha$ as the scene gets denser.","The approach runs fast enough for real-time use: the paper reports execution within the 0.25 s (single-step) or 1.25 s (prediction-step) schemes on a commercial GPU.","The number of iterations has little effect in the experiments (1, 3, and 10 iterations perform comparably), indicating that the plan-simulation loop converges quickly in practice."],"supporting_citations":[{"why":"Foundational conformal prediction framework and the exchangeability result (Theorem 1) on which Theorem 2's guarantee is built.","marker":"[31]"},{"why":"ORCA simulator used to generate the decision-dependent calibration dataset by simulating human reactions to the current robot plan.","marker":"[29]"},{"why":"Offline conformal prediction baseline that ICP extends; its fixed-radius approach motivated the need for decision-dependent calibration.","marker":"[20]"},{"why":"Adaptive conformal prediction baseline with asymptotic guarantees that ICP compares against and seeks to surpass with finite-sample bounds.","marker":"[7]"},{"why":"Gumbel Social Transformer trajectory predictor used inside ICP to compute the nonconformity scores and predicted human positions.","marker":"[13]"}],"fun_headline_variants":["Crowd robots calibrate on simulated reactions to their own plan","Plan-dependent conformal prediction for safer crowd navigation","Simulated human feedback sets safety intervals for robot plans","Robot motion drives uncertainty bounds in crowd navigation","Interaction-aware safety: robot's plan tunes its error intervals"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The guarantee rests on the assumption that the way real people move when the robot follows its final plan is statistically interchangeable with the way the simulator moves them under that same plan; if the simulator mispredicts human behavior, the coverage bound does not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Crowd robots calibrate on simulated reactions to their own plan","Plan-dependent conformal prediction for safer crowd navigation","Simulated human feedback sets safety intervals for robot plans","Robot motion drives uncertainty bounds in crowd navigation","Interaction-aware safety: robot's plan tunes its error intervals"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000357,"raw_usage":{"total_tokens":1929,"prompt_tokens":932,"completion_tokens":997,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":548,"completion_tokens_details":{"reasoning_tokens":920}},"tokens_in":548,"tokens_out":997,"duration_ms":9824,"temperature":1.0,"reasoning_tokens":920,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T16:22:26.399053+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run ICP in an environment where human motion is generated by a different mechanism than the ORCA simulator (for example, recorded pedestrian data or a learned policy), execute the plan that ICP outputs, and count the fraction of trials over many repetitions in which the robot stays at least $r_r + r_h$ from every human at all predicted steps; if that fraction is reliably below $1 - \\alpha N T_{\\mathrm{pred}}$, the exchangeability premise fails.","supporting_citations":[{"cited_title":"Springer Science & Business Media (2005)","cited_arxiv_id":null,"evidence_quote":"Foundational conformal prediction framework and the exchangeability result (Theorem 1) on which Theorem 2's guarantee is built."},{"cited_title":"In: Robotics Research: The 14th International Symposium ISRR","cited_arxiv_id":null,"evidence_quote":"ORCA simulator used to generate the decision-dependent calibration dataset by simulating human reactions to the current robot plan."},{"cited_title":"IEEE Robotics and Automation Letters 8(8), 5116–5123 (2023).https://doi.org/10.1109/LRA.2023.3292071 18 Z","cited_arxiv_id":null,"evidence_quote":"Offline conformal prediction baseline that ICP extends; its fixed-radius approach motivated the need for decision-dependent calibration."},{"cited_title":"In: Learning for Dynamics and Control Conference","cited_arxiv_id":null,"evidence_quote":"Adaptive conformal prediction baseline with asymptotic guarantees that ICP compares against and seeks to surpass with finite-sample bounds."},{"cited_title":"IEEE Robotics and Automation Letters7(2), 1198–1205 (2022).https://doi.org/10.1109/LRA","cited_arxiv_id":null,"evidence_quote":"Gumbel Social Transformer trajectory predictor used inside ICP to compute the nonconformity scores and predicted human positions."}],"review_version":1}