Pith. sign in

REVIEW 3 major objections 5 minor 18 references

Deploying SICNav in the Field: Safe and Interactive Crowd Navigation using MPC and Bilevel Optimization

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SICNav, a crowd-navigation planner that couples robot and human motion in one bilevel optimization, completed 6.73 km of autonomous field navigation in roughly two hours under real noisy sensing.

desk verdict Deployment report of a published planner; the useful contribution is the field integration and data, but the headline distance claim needs a clearer definition of 'autonomous' before the numbers can be trusted. read the letter →

arxiv 2506.08851 v1 pith:U6LTQVU6 submitted 2025-06-10 cs.RO

Sepehr Samavi , Garvish Bhutani , Florian Shkurti , Angela P. Schoellig This is my paper
classification cs.RO
keywords crowdnavigationbileveloptimizationmodelpredictivecontrolORCAfielddeploymentlidarperceptionhumantrackingautonomous
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports the first field deployment of SICNav, a crowd-navigation approach that folds human motion prediction and robot planning into a single bilevel nonlinear MPC problem. The claim is that the method can drive a small ground robot through previously unseen indoor and outdoor pedestrian spaces using only lidar, IMU, and wheel odometry, without motion capture. Over 1h51m of autonomous operation the robot traveled 6.73 km, with mean optimization solve times of 0.10 to 0.11 s, comfortably below the 0.25 s control period. If this holds, interactive crowd navigation is not limited to lab settings with clean state estimates; it can run in real time on noisy, real-world perception.

What carries the argument

The load-bearing component is the SICNav bilevel MPC: a nonlinear program whose outer level chooses the robot's trajectory while the inner level solves, for each human, an ORCA reciprocal-collision-avoidance problem; the inner problems are replaced by their Karush-Kuhn-Tucker optimality conditions so the whole system is optimized jointly. Around this, the field system adds camera-free human perception by running YOLOv9 segmentation on lidar intensity and reflectivity images, tracking detections with Kalman filters, and filtering tracked humans out of the static obstacle map so the optimizer can treat them interactively rather than as walls.

What would settle it

Run the deployed system in a corridor where three or more people approach from different directions with crossing paths, and record all robot-human distances and manual takeovers; a collision, a repeated freeze, or a takeover caused by the two-humans-only model would show that the field guarantee does not hold outside the modeled regime. A simpler quantitative check is to monitor MPC solve times: any recorded solve time exceeding the 0.25 s control period would falsify the real-time claim.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a planner which treats human reactions as part of the optimization, replacing each human prediction subproblem with its optimality conditions and solving the combined problem, remains computationally tractable and behaviorally sensible when embedded in a full autonomy stack with SLAM, lidar-based human detection and tracking, and costmap-based static obstacles. The deployed system models the two humans with the lowest time-to-collision as interactive agents using ORCA and treats everyone else as static obstacles; this keeps the nonlinear program small enough to solve at 10 Hz. The authors report 6.73 km traversed and call the results preliminary. They present this as evidence that collision-free-by-construction interactive planning can run outside a controlled laboratory.

Load-bearing premise

The safety and interactivity guarantees collapse if human motion is not well described by ORCA with intent guessed from the latest observed velocity, or if a real crowd has more than two humans approaching simultaneously, since additional humans are treated as static obstacles.

Editorial extensions

If this is right

  • SICNav can act as a drop-in replacement for the local planner in a standard ROS navigation stack, since it consumes the same costmap and velocity-command interfaces.
  • The measured solve times of about 0.1 s are sufficient for 10 Hz replanning with a 2 s horizon, matching the control loops of small ground robots.
  • Filtering the two most imminent humans out of the costmap and modeling them as interactive prevents the optimization from becoming intractable while keeping behavior responsive.
  • In the authors' simulation noise study, SICNav's advantage over a constant-velocity non-interactive MPC narrows once the standard deviation of state-estimation noise grows past 0.03, indicating that perception quality sets a floor on interactive planning gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural stress test is to raise the number of interactively modeled humans from two to three or four as solver hardware improves; the paper's two-human cap is a computational simplification, not a modeling necessity, so the same bilevel structure has a clear scaling path.
  • Because the perception pipeline runs on lidar intensity images rather than a camera, it may remain functional in low-light or rain, where RGB cameras degrade; this is an untested implication of the camera-free design.
  • If the authors perturb localization, detection, and tracking separately and add perception delays in simulation, they may find that one component, not the planner, sets the ceiling on field performance; their current aggregate-noise simulation cannot isolate this yet.
  • Manual-takeover frequency could itself be used as a deployment metric: comparing takeover rates across methods in identical environments would quantify how much interactive modeling helps under realistic noise.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper reports an initial field deployment of SICNav, a bilevel MPC planner that couples robot trajectory planning with ORCA-based predictions of human motion, on a Clearpath Jackal robot. The system integrates Cartographer 2D SLAM, YOLOv9/aUToTrack-based human detection and tracking, costmap-based obstacle representation, and hybrid A* global planning. The authors report 6.73 km of autonomous navigation over 1h51m in previously unseen indoor, sidewalk, and campus environments, along with wall-clock statistics on manual takeovers, detected human counts, and SICNav solve times. They also present a preliminary simulation study comparing SICNav, SICNav-Diffusion, and a non-interactive CVMM MPC under additive state-estimation noise. The paper is explicitly framed as preliminary, with a stated plan for more extensive evaluation and comparison in a final version.

Significance. If the reported field statistics are verifiable, this is a useful systems-integration milestone: it demonstrates that a bilevel interactive MPC crowd-navigation algorithm can run in real time on a mobile robot in previously unseen environments with noisy localization and perception, which goes beyond the earlier in-lab VICON-based experiments. The paper does not claim new theoretical results; its value is the deployment architecture, the adaptation of aUToTrack to lidar intensity/range images, and the preliminary field data. The simulation robustness comparison is a reasonable first step, but the current evidence is too thin to substantiate the headline autonomy claims, and the safety/interactivity claims are stated without acknowledging the model assumptions under which they hold.

major comments (3)
  1. [Sec. III-B, Table I] The headline claim that the robot 'navigated autonomously for nearly one hour and 51 minutes, traversing 6.73 km' is not verifiable as stated because the paper never defines how Duration and Distance are computed with respect to the 49 manual takeovers. If these quantities include wall-clock time and distance while the operator is in control or during takeover recovery, then the autonomous distance is inflated. The authors must state explicitly that Duration and Distance are computed only while the SICNav controller is in command, define an autonomy-active flag, and provide per-segment logs that separate autonomous, takeover, and recovery intervals. Without this, the central deployment claim cannot be interpreted.
  2. [Sec. III-B, Fig. 7] The real-time performance claim is not supported by the reported statistics. The paper states that SICNav re-plans at 10 Hz, which corresponds to a 0.1 s deadline, yet the mean solve times are 0.100-0.114 s and are compared only to the 0.25 s MPC discretization period. The histograms also show a tail of larger solve times. The authors should report the maximum and 99th percentile solve times, the number of deadline misses, and whether the replanning loop is synchronous, asynchronous, or pipelined with the previous solution. As written, 'acceptable solve times' is a claim about the mean, not about whether the controller actually met its real-time schedule.
  3. [Sec. I and Sec. II-E.3] The statements that SICNav is 'collision-free by construction' and ensures safety are model-level claims, not field-validated guarantees. In the deployment, these claims rely on the assumptions that ORCA accurately models human reactions, that the intended velocity is well estimated by projecting the latest observed velocity forward with a fixed horizon, and that modeling only the two humans with lowest time-to-collision with ORCA while treating all other humans as static is sufficient in real crowds. The paper should explicitly scope the safety claim to these model assumptions and should report safety-relevant events from the field (near-misses, collisions, interventions taken because of unexpected human behavior) so that readers can assess whether the assumptions actually held in the tested environments.
minor comments (5)
  1. [Sec. III-C.1, Fig. 8] The text says noise is sampled from a zero-mean Gaussian with standard deviation sigma in {0, 0.2}, but the discussion refers to performance changing as sigma increases 'beyond 0.03' and the plots show many intermediate levels; please specify the full set of noise levels and how the 200 scenarios are distributed across them.
  2. [Sec. II-C] The percentile rejection thresholds (5th and 65th) used to filter lidar points from detection contours are introduced without justification or sensitivity analysis; please state how these values were selected and whether the tracking results are sensitive to them.
  3. [Sec. II-E.2] The intended human velocity is estimated 'by projecting the agent's latest observed velocity forward with a fixed time horizon,' but the horizon length is never specified; please give its value and comment on its effect on the results.
  4. [Fig. 3 caption] There is a typo in the caption ('Point cCloud' should be 'Point Cloud'), and the detection model name is written inconsistently as 'YOLOv9' and 'Yolov9' in the text.
  5. [Abstract and Sec. IV] The abstract and conclusion say 'nearly 7 km' and 'nearly two hours,' while Table I provides exact values (6.73 km, 1:51:41); please either cite exact numbers consistently or clarify that the rounded values are approximate.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SICNav and SICNav-Diffusion are cited as prior work, not re-derived; field statistics and simulation comparisons are empirical and not fitted to the reported claim.

full rationale

This is a field-deployment systems report, not a derivation of a new theoretical result. The safety/interactivity properties of SICNav are inherited from the previously published bi-level MPC formulation (Eq. 2) and are cited from [9]; the present paper does not use those citations to replace a missing derivation, and the prior in-lab experiments are independent empirical evidence. The simulation study (Fig. 8) compares SICNav against SICNav-Diffusion [17] and a CVMM-based non-interactive MPC under added Gaussian noise, with no parameter fitted to the reported outcome. The paper explicitly defers the head-to-head field comparison: Sec. III-C.1 says "in the final version of this paper, we aim to compare the performance of SICNav with other local planning methods," and Sec. III-C.2 flags the step/non-drivable-area detection limitation that caused manual takeovers. Table I's duration/distance reporting leaves open whether takeover segments are excluded from the "autonomous" totals, but that is a measurement-accounting or verifiability concern, not a circular dependency: no equation reduces to its inputs by construction and no fitted quantity is renamed as a prediction. Therefore no circular step is exhibited.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rest on standard robotics components (Cartographer, YOLOv9, aUToTrack, acados) and on the ORCA-based interaction model from prior work. No new physical entities are introduced. The main free parameters are hand-chosen thresholds and the limited number of interactively modeled humans; the MPC time step and horizon are standard design choices that also affect the results.

free parameters (3)
  • Range percentile rejection thresholds = 5th and 65th percentiles
    Hand-chosen in Sec. II-C to reject ground or foreign points inside YOLOv9 contours; directly affects the 3D human centroid estimates fed to the planner.
  • Number of ORCA-modeled humans = 2
    Set for real-time feasibility in Sec. II-E.3; humans beyond the two lowest time-to-collision are treated as static obstacles, changing the interaction model and potentially the safety margin.
  • Intended-velocity projection horizon = not specified in paper
    Sec. II-E.2 estimates the human intended goal by projecting the latest observed velocity forward with a fixed time horizon; the horizon value is not reported, leaving a tunable parameter that influences all human predictions.
assumptions (4)
  • domain assumption ORCA is a valid model of real pedestrian motion and reactions to the robot
    Used in Sec. II-E.2 to define the lower-level human prediction; if humans do not behave as ORCA agents, the joint optimization does not capture true interactions.
  • standard math KKT reformulation of the ORCA lower-level problem is exact
    Sec. II-E.3 replaces each lower-level problem with its KKT conditions; this requires convexity and constraint qualification, which hold for the ORCA QCQP under standard assumptions.
  • domain assumption YOLOv9 detections on lidar intensity/reflectivity images reliably identify humans
    Sec. II-C relies on YOLOv9 segmentation and percentile-based range filtering to produce human centroids; missed or false detections propagate directly into the planner's state estimates.
  • domain assumption Treating non-ORCA humans as static obstacles is safe
    Sec. II-E.3 states that only two humans are modeled interactively and all others appear as static obstacles; this may underestimate the motion of pedestrians outside the ORCA set.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deploying SICNav in the Field: Safe and Interactive Crowd Navigation using MPC and Bilevel Optimization." pith.science (2026). https://pith.science/paper/U6LTQVU6

@misc{pith2026250608851,
  author       = {Pith},
  title        = {Pith review of: Deploying SICNav in the Field: Safe and Interactive Crowd Navigation using MPC and Bilevel Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U6LTQVU6}},
  note         = {Machine review of arXiv:2506.08851}
}
read the original abstract

Safe and efficient navigation in crowded environments remains a critical challenge for robots that provide a variety of service tasks such as food delivery or autonomous wheelchair mobility. Classical robot crowd navigation methods decouple human motion prediction from robot motion planning, which neglects the closed-loop interactions between humans and robots. This lack of a model for human reactions to the robot plan (e.g. moving out of the way) can cause the robot to get stuck. Our proposed Safe and Interactive Crowd Navigation (SICNav) method is a bilevel Model Predictive Control (MPC) framework that combines prediction and planning into one optimization problem, explicitly modeling interactions among agents. In this paper, we present a systems overview of the crowd navigation platform we use to deploy SICNav in previously unseen indoor and outdoor environments. We provide a preliminary analysis of the system's operation over the course of nearly 7 km of autonomous navigation over two hours in both indoor and outdoor environments.

Figures

Figures reproduced from arXiv: 2506.08851 by the authors.

Figure 1
Figure 1. Photo of the robot operating in a crowd environment [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Photos of the front (left) and rear (right) view of the [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Lidar signal image (a) and point cloud (b) generated [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: (a) Robot operation areas highlighted on the Apple Maps satellite image of the Technical University of Munich campus in [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Top-down view of the Rviz visualization and goal input [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: Total number of humans detected in the scene in the different operation areas. We plot histograms for the (a) Indoor areas [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: SICNav solve times in the (a) Indoor areas [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]
Figure 8
Figure 8. Figure 8: Robot crowd navigation performance using different control methods in the face of increasing state estimation noise in simulation. [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

18 extracted references · 13 canonical work pages

  1. [9]

    SICNav: Safe and Interactive Crowd Navigation Using Model Predictive Control and Bilevel Optimization,

    S. Samavi, J. R. Han, F. Shkurti, and A. P. Schoellig, “SICNav: Safe and Interactive Crowd Navigation Using Model Predictive Control and Bilevel Optimization,”IEEE Transactions on Robotics, vol. 41, pp. 801–818, 2024. [Online]. Available: http://sepehr.fyi/projects/sicnav

    work page 2024

  2. [1]

    Constrained model predictive control: Stability and optimality,

    D. Mayne, J. Rawlings, C. Rao, and P. Scokaert, “Constrained model predictive control: Stability and optimality,”Automatica, vol. 36, no. 6, pp. 789–814, June 2000. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S0005109899002149

    work page 2000

  3. [2]

    Robot Motion Planning in Dynamic, Uncertain Environments,

    N. E. Du Toit and J. W. Burdick, “Robot Motion Planning in Dynamic, Uncertain Environments,”IEEE Transactions on Robotics, vol. 28, no. 1, pp. 101–115, Feb. 2012. [Online]. Available: doi.org/10.1109/TRO.2011.2166435

    work page arXiv 2012

  4. [3]

    Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data,

    T. Salzmann, B. Ivanovic, P. Chakravarty, and M. Pavone, “Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data,” in2020 European Conference on Computer Vision (ECCV), 2020. [Online]. Available: http://arxiv.org/abs/2001. 03093

    work page 2020

  5. [4]

    From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting,

    K. Mangalam, Y . An, H. Girase, and J. Malik, “From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting,” in2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, QC, Canada: IEEE, Oct. 2021, pp. 15 213–15 222. [Online]. Available: https://ieeexplore.ieee.org/document/9709992/

    work page arXiv 2021

  6. [5]

    Human Trajectory Prediction via Neural Social Physics

    J. Yue, D. Manocha, and H. Wang, “Human Trajectory Prediction via Neural Social Physics,” inProceedings of the European Conference on Computer Vision (ECCV). arXiv, July 2022, arXiv:2207.10435 [cs]. [Online]. Available: http://arxiv.org/abs/2207.10435

    work page Pith review arXiv 2022

  7. [6]

    Move Beyond Trajectories: Distribution Space Coupling for Crowd Navigation,

    M. Sun, F. Baldini, P. Trautman, and T. Murphey, “Move Beyond Trajectories: Distribution Space Coupling for Crowd Navigation,” inRobotics: Science and Systems XVII. Robotics: Science and Systems Foundation, July 2021. [Online]. Available: doi.org/10.15607/RSS.2021.XVII.053

    work page doi:10.15607/rss.2021.xvii.053 2021

  8. [7]

    Relational Graph Learning for Crowd Navigation,

    C. Chen, S. Hu, P. Nikdel, G. Mori, and M. Savva, “Relational Graph Learning for Crowd Navigation,” in2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Las Vegas, NV , USA: IEEE, Oct. 2020, pp. 10 007–10 013. [Online]. Available: https://ieeexplore.ieee.org/document/9340705/

    work page arXiv 2020

Show all 18 references
  1. [8]

    Collision avoidance in pedestrian-rich environments with deep reinforcement learning,

    M. Everett, Y . F. Chen, and J. P. How, “Collision avoidance in pedestrian-rich environments with deep reinforcement learning,”IEEE Access, vol. 9, pp. 10 357–10 377, 2021

    2021

  2. [10]

    Reciprocal n-Body Collision Avoidance,

    J. van den Berg, S. J. Guy, M. Lin, and D. Manocha, “Reciprocal n-Body Collision Avoidance,” inRobotics Research, B. Siciliano, O. Khatib, F. Groen, C. Pradalier, R. Siegwart, and G. Hirzinger, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, vol. 70, pp. 3–19, serie...

    2011 doi

  3. [11]

    Real-time loop closure in 2d lidar slam,

    W. Hess, D. Kohler, H. Rapp, and D. Andor, “Real-time loop closure in 2d lidar slam,” inIEEE International Conference on Robotics and Automation (ICRA), 2016, pp. 1271–1278

    2016

  4. [12]

    Yolov9: Learning what you want to learn using programmable gradient information,

    C.-Y . Wang and H.-Y . M. Liao, “Yolov9: Learning what you want to learn using programmable gradient information,” 2024

    2024

  5. [13]

    aUToTrack : A Lightweight Object Detection and Tracking System for the SAE AutoDrive Challenge,

    K. Burnett, S. Samavi, S. L. Waslander, T. D. Barfoot, and A. P. Schoellig, “aUToTrack : A Lightweight Object Detection and Tracking System for the SAE AutoDrive Challenge,” inConference on Computer and Robot Vision (CRV), 2019. [Online]. Available: https://ieeexplore.ieee.org...

    2019

  6. [14]

    Patchwork++: Fast and robust ground segmentation solving partial under-segmentation using 3d point cloud,

    S. Lee, H. Lim, and H. Myung, “Patchwork++: Fast and robust ground segmentation solving partial under-segmentation using 3d point cloud,” in2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 13 276–13 283

    2022

  7. [15]

    costmap converter ros package,

    C. Rosmann, “costmap converter ros package,” 2015. [Online]. Available: https://wiki.ros.org/costmap converter

    2015

  8. [16]

    acados – a modular open-source framework for fast embedded optimal control,

    R. Verschueren, G. Frison, D. Kouzoupis, J. Frey, N. van Duijkeren, A. Zanelli, B. Novoselnik, T. Albin, R. Quirynen, and M. Diehl, “acados – a modular open-source framework for fast embedded optimal control,”Mathematical Programming Computation, Oct 2021. [Online]. Available:...

    2021 doi

  9. [17]

    Sicnav-diffusion: Safe and interactive crowd navigation with diffusion trajectory predictions,

    S. Samavi, A. Lem, F. Sato, S. Chen, Q. Gu, K. Yano, A. P. Schoellig, and F. Shkurti, “Sicnav-diffusion: Safe and interactive crowd navigation with diffusion trajectory predictions,”arXiv preprint arXiv:2503.08858, 2025

    2025 arXiv

  10. [18]

    Spherical transformer for lidar-based 3d recognition,

    X. Lai, Y . Chen, F. Lu, J. Liu, and J. Jia, “Spherical transformer for lidar-based 3d recognition,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 17 545–17 555. Presented at the 2025 IEEE ICRA Workshop on Field Robotics

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.