Pith. sign in

REVIEW 4 major objections 6 minor 44 references

A hybrid social navigator runs a fast RL policy by default and invokes a vision-language model only when people enter personal space, raising success and cutting collisions on photo-realistic benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-07-14 07:48 UTC pith:6GAFLUG7

load-bearing objection Solid hybrid engineering for social nav: Falcon RL by default, LoRA VLM only on PSC breach; headline gains are real but partly inflated by freezing the world during ~2.4 s VLM calls. the 4 major comments →

arxiv 2607.10991 v1 pith:6GAFLUG7 submitted 2026-07-13 cs.RO cs.AIcs.CV

Think When It Matters: Conditional VLM Reasoning for Social Navigation with RL Policies

Ali Ahmadi , Hamed Rahimi , Adrien Jacquet Cretides , Marie Samson , Mahdi Khoramshahi , Mohamed Chetouani This is my paper
classification cs.RO cs.AIcs.CV
keywords social robot navigationhybrid RL-VLMpersonal space compliancevision-language modelsreal-time navigationhuman-aware motion planningconditional reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Social robots need both split-second reactions and the ability to read awkward human situations, but pure reinforcement-learning controllers are rigid and pure vision-language models are too slow for continuous motion. This paper claims that the right answer is conditional: keep a lightweight RL policy in charge for ordinary low-density travel, and hand control to a post-trained vision-language model only when a rolling personal-space score shows that someone has entered a sensitive zone. On two social navigation benchmarks the hybrid system lifts task success while reducing human collisions, without giving up path efficiency. Real-robot deployment is offered as evidence that the same switch can run under ordinary robot middleware. The practical stake is simple: robots can stay real-time most of the time and still think carefully when comfort and safety matter most.

Core claim

The central claim is that selectively conditioning an RL navigation policy on a LoRA-adapted vision-language model, triggered only when a short-window personal-space compliance score falls below threshold, measurably improves success rate and reduces human collisions on Social-MP3D and Social-HM3D relative to strong RL and hybrid baselines, while remaining competitive on path length and personal-space metrics.

What carries the argument

HUMA's Personal Space Compliance (PSC) switch: a rolling average of binary distance signals that routes observations either to a Falcon-style depth-and-goal RL policy or to a fused depth-JET-plus-prompt VLM that emits short discrete action sequences.

Load-bearing premise

The reported gains partly assume that the simulated world freezes or slows while the vision-language model spends multiple seconds deciding, an assumption the paper itself marks as unrealistic for full-speed humans.

What would settle it

Re-run the Social-MP3D and Social-HM3D evaluations with continuous full-speed human motion during every multi-second VLM call (no freeze or pause) and check whether the absolute gains in success rate and human-collision rate over the pure RL baseline still hold.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X LinkedIn Reddit HN

If this is right

  • Routine corridor travel can stay on cheap reactive controllers; expensive semantic models need only fire near people.
  • Personal-space distance becomes a practical runtime gate for foundation-model compute rather than a mere evaluation metric.
  • Colorized depth with edge overlays can serve as a compact visual token for social VLMs instead of raw RGB.
  • Real-time social navigation stacks can be assembled as ROS-style nodes that swap action sources under a compliance threshold.
  • Further gains should come from predicting future proximity breaches rather than reacting after they occur.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same proximity-gated pattern could apply to other slow foundation models (speech, long-horizon planners) that robots only need in rare social moments.
  • Adaptive thresholds that tighten in dense crowds or open in empty halls would reduce unnecessary VLM calls beyond the fixed 0.8 m rule.
  • Shorter VLM action horizons with higher call frequency would address the observed failure mode of padding plans with repeated 'forward' steps.
  • Closing the sim-to-real visual gap for depth-JET tokens is likely the next bottleneck once timing freeze is removed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces HUMA, a hybrid social-navigation controller that runs a Falcon-style RL policy by default and switches to a LoRA-adapted Qwen3-VL-2B only when a rolling personal-space compliance score (PSC_W) falls below a threshold. The VLM is given a fused depth-JET/Canny/goal image plus a text prompt and emits short discrete action sequences. On Social-HM3D and Social-MP3D the method reports SR 62.07% / 70.35% and H-Coll 34.48% / 20.82%, improving success by roughly 3 and 20 absolute points over the strongest listed baselines while remaining competitive on SPL and PSC. Ablations cover VLM size and training data, visual input modality, proximity threshold, and environment evolution during VLM inference; a qualitative ROS2 deployment on the Mirokai robot is also shown.

Significance. Selective, proximity-triggered VLM reasoning is a timely and practical response to the latency–semantics trade-off in social navigation and is of clear interest to the HRI and mobile-robotics communities. Concrete strengths include systematic ablations (Tables 2–5, Figure 3), PEFT/LoRA adaptation of a compact 2B VLM, an explicit switching metric built on Falcon’s PSC, and a closed-loop ROS2 real-robot integration. If the reported gains survive continuous human motion and multi-seed evaluation, HUMA would be a useful template for hybrid foundation-model controllers. As written, the strongest evidence is the Social-MP3D Freeze result; the Social-HM3D margin is small and sensitive to the freeze assumption the authors themselves flag as unrealistic.

major comments (4)
  1. Table 1 (primary claim) is measured under the Freeze strategy of Table 5, in which the simulator is paused for the full multi-second VLM call (~2425 ms/call, Table 3). Under the authors’ own Realistic schedule, Social-HM3D SR falls from 62.07% to 56.90% and H-Coll rises from 34.48% to 40.23%, essentially erasing the reported ~3-point edge over Falcon and NavThinker. No Realistic numbers are given for Social-MP3D, so it is unknown whether the larger ~20-point SR / ~20-point H-Coll gains survive continuous human motion. Because the hybrid’s value proposition is precisely selective insertion of a slow VLM into a real-time loop, Freeze systematically overstates the social benefit that would be observed on a physical robot. Primary tables and abstract claims should be re-reported under Realistic (or an equivalent continuous-time protocol), with all baselines under the same timing model, or th
  2. No multi-seed statistics, confidence intervals, or episode-level variance are reported for Table 1 (or the ablations). The Social-HM3D SR gain is only ~2.6 points over NavThinker and ~5.8 over Falcon; without error bars it is impossible to judge whether this is a reliable improvement or run-to-run noise. At minimum, multi-seed means and standard deviations (or bootstrap CIs) for SR, SPL, PSC, and H-Coll on both benchmarks are needed before the small HM3D margin can support the abstract’s “3%” claim.
  3. Section 3.2 and Table 4 leave the switching rule under-specified relative to the reported numbers. The text defines a binary PSC signal at 1.0 m and a rolling PSC_W compared to an unnamed threshold θ, yet Table 4 ablates a distance parameter r ∈ {0.70,…,0.90} m and reports #Calls that jump by nearly an order of magnitude. It is unclear whether r replaces the 1.0 m radius, whether θ is fixed or co-varied, and what W is. Because the entire hybrid benefit is gated by this switch, the exact (r, W, θ) used for Table 1 must be stated, and the same switch must be applied when re-evaluating under Realistic timing.
  4. The VLM emits multi-step plans (e.g., 8-action sequences in Figure 7) that are then executed while the RL policy is off. Under non-frozen dynamics this is open-loop control for a non-trivial horizon; the Limitations section notes residual “forward” padding as a failure mode. The paper should quantify how often the plan is invalidated by intervening human motion (or replan frequency under Realistic), and whether shorter plans with higher call rates improve H-Coll/SR—otherwise the hybrid’s safety claims rest on an unmeasured open-loop assumption.
minor comments (6)
  1. Abstract and §5 claim “significantly reducing personal space violations,” but Table 1 PSC is only competitive (e.g., Social-MP3D: Ours 92.96 vs NavThinker 93.68 / ORCA 94.03). The clear win is H-Coll; rephrase to match the table.
  2. Figure 3 and Table 2 evaluate VLM action accuracy / SBERT / ROUGE on SNEI–MUSON, which is useful, but the link from those QA metrics to closed-loop SR/H-Coll is not quantified; a short correlation or “VLM-only vs hybrid” closed-loop row would help.
  3. §3.2 reward and loss notation mixes R^t_socialnav, R^t_scp, and L_aux without stating the numerical β weights used at training time; listing them (or pointing to Falcon’s defaults) would aid reproducibility.
  4. Real-world §6 / Figures 4–5 are qualitative only (no SR, H-Coll, or latency on Mirokai). A short quantitative trial or at least measured end-to-end latency and switch rate would better support “practical viability.”
  5. Typographical / consistency: “Mirokaï / Mirokai / Miroka ¨ı” variants; “post-trained” vs “fine-tuned”; arXiv-style “†Results taken from the original paper” should state whether baselines were re-run under identical episode seeds.
  6. Related Work could more clearly position HUMA against concurrent hybrid planners (NavThinker is compared in Table 1 but the methodological difference—PSC switch vs action-conditioned world model—is only briefly stated).

Circularity Check

0 steps flagged

No circularity: empirical hybrid RL+VLM architecture with external benchmarks and ablations; no derivation reduces to its inputs by construction.

full rationale

HUMA is an engineering proposal that re-uses an external RL policy (Falcon) and its PSC definition as a base, then adds a LoRA-tuned VLM invoked by a rolling compliance threshold. All headline claims (Table 1 SR/H-Coll gains on Social-HM3D/MP3D) are measured success and collision rates against independent baselines (A*, ORCA, Habitat, Falcon, NavThinker) on held-out photo-realistic scenes; they are not algebraic identities, fitted constants re-labeled as predictions, or uniqueness theorems imported from the same authors. Ablations (VLM size, training set, visual augmentation, proximity threshold r, freeze/pause/realistic timing) and the Mirokaï deployment are likewise empirical comparisons, not self-definitional loops. Mild architectural reuse of Falcon is ordinary engineering practice and does not force the reported numbers. Evaluation caveats (freeze during multi-second VLM calls) affect external validity but do not create circularity in the derivation chain, which contains none.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

The central empirical claim rests on Falcon’s RL policy and social reward, a hand-chosen personal-space trigger, a small LoRA-adapted VLM, and a simulation protocol that often freezes the world during multi-second inference. No new physical entity is postulated; the free parameters are engineering knobs selected by ablation.

free parameters (4)
  • PSC proximity trigger r
    Binary switch distance; ablated over {0.70…0.90} m and best SR taken at 0.80 m (Table 4), so the headline configuration is selected on evaluation performance.
  • PSC rolling window W and compliance threshold θ
    Localized compliance PSC_W averages the last W binary 1.0 m checks; W and θ are design choices that gate VLM calls and are not derived from first principles.
  • Social reward weights (β_prox, β_traj, β_coll, 2.0 m envelope, H-step trajectory horizon)
    Inherited from Falcon’s composite social penalty; magnitudes and the 2.0 m exponential proximity envelope shape the RL base policy that HUMA conditions.
  • LoRA rank/α/dropout and SFT dataset choice
    r=16, α=32, dropout 0.05 and exclusive SNEI training are chosen for action accuracy; they define the VLM policy that produces the reported gains.
axioms (4)
  • domain assumption A Falcon-style DD-PPO depth+goal RL policy is an adequate default controller for low-density navigation.
    Section 3.2 adopts Falcon’s architecture and social reward as the always-on policy; all hybrid gains are measured relative to that base.
  • domain assumption Personal-space compliance within ~1 m (plus robot/human radii) is a sufficient binary signal for when high-level social reasoning is needed.
    Switching mechanism in §3.2; if important social failures occur outside this zone, the VLM is never invoked.
  • domain assumption A LoRA-tuned 2B VLM given JET depth, Canny edges, and a short goal prompt can emit socially compliant discrete action sequences.
    Equation (1) and visual pre-processing in §3.2; action accuracy ablations (Table 2–3) support but do not prove generalization.
  • ad hoc to paper Simulator may pause or slow environment evolution during VLM inference without invalidating the comparison.
    Table 5 Freeze vs Realistic; authors note the freeze assumption is unrealistic and costs ~5 points SR.
invented entities (2)
  • HUMA hybrid architecture (PSC switch + RL default + conditional VLM planner) no independent evidence
    purpose: Name and structure the arbitration between reactive RL and post-trained VLM for social navigation.
    System-level construct; independent evidence is the reported benchmark and robot demo, not an external physical prediction.
  • Localized rolling PSC_W compliance metric as policy switch no independent evidence
    purpose: Turn Falcon’s PSC idea into a real-time binary gate that avoids long-horizon smoothing of recent violations.
    Defined in §3.2 from a window of 1.0 m distance indicators; usefulness is internal to the paper’s ablations.

reviewed 2026-07-14 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Think When It Matters: Conditional VLM Reasoning for Social Navigation with RL Policies." pith.science (2026). https://pith.science/paper/6GAFLUG7

@misc{pith2026260710991,
  author       = {Pith},
  title        = {Pith review of: Think When It Matters: Conditional VLM Reasoning for Social Navigation with RL Policies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6GAFLUG7}},
  note         = {Machine review of arXiv:2607.10991}
}
Share X LinkedIn Reddit HN
read the original abstract

As mobile robots become more integrated into everyday human environments, social robot navigation is becoming essential for ensuring human comfort, safety, and trust. While reinforcement learning (RL) navigation policies provide the fast inference and reactive behavior necessary for real-time deployment, they still lack flexible semantic reasoning capabilities and often fail to generalize to complex social scenarios. Recent approaches have increasingly turned to vision-language models (VLMs) in place of RL policies to improve semantic and social reasoning in robot navigation. Nevertheless, their high computational cost and slow inference remain major barriers to real-time deployment. To overcome these limitations, we introduce HUMA (Hybrid Understanding for Multi-modal social Navigation), a hybrid architecture that dynamically balances the computational efficiency of RL policies with the deep semantic understanding of VLMs. Our approach uses a reactive RL policy to handle low-density, routine navigation tasks, while conditioning it on a post-trained high-level VLM when a human enters sensitive situations, such as the robot's proximity zone. We evaluate HUMA on the Social-MP3D and Social-HM3D benchmarks, where it achieves task success improvements of 20% and 3%, respectively, while significantly reducing personal space violations and human collisions against state-of-the-art baselines. Extensive ablation studies validate each architectural component, and real-world deployment on the Miroka\"i mobile robot further demonstrates the practical viability of our approach.

Figures

Figures reproduced from arXiv: 2607.10991 by Adrien Jacquet Cretides, Ali Ahmadi, Hamed Rahimi, Mahdi Khoramshahi, Marie Samson, Mohamed Chetouani.

Figure 1
Figure 1. Figure 1: Overview. Existing approaches for social navigation face a fundamental trade-off: RL [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: HUMA is a hybrid human-aware navigation framework that dynamically balances com [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: VLM selection ablation. Our fine-tuned (ft) and zero-shot (zs) 2B models achieve accuracy [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Real-World Deployment of HUMA on the Mirokai robot. The chronological sequence [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Real-world deployment architecture of HUMA. Agent positions from the OptiTrack mo [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: VLM reasoning in a proximity￾triggered social navigation scenario. (a) Raw RGB, (b) JET colormap depth, and (c) fused VLM input with Canny edge overlay. With PSC = 0.800 and a human at 1.58 m, the PSC switch activates the VLM, which generates a forward action sequence while describing the social con￾text in natural language. Detailed example pro￾vided in [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: An overview Example. Two consecutive VLM invocations triggered by PSC threshold [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 4 canonical work pages

  1. [1]

    P. T. Singamaneni, P. Bachiller-Burgos, L. J. Manso, A. Garrell, A. Sanfeliu, A. Spalanzani, and R. Alami. A survey on socially aware robot navigation: Taxonomy and future challenges. The International Journal of Robotics Research, 43(10):1533–1572, 2024

    2024

  2. [2]

    Triebel, K

    R. Triebel, K. Arras, R. Alami, L. Beyer, S. Breuers, R. Chatila, M. Chetouani, D. Cremers, V . Evers, M. Fiore, H. Hung, O. A. I. Ram´ırez, M. Joosse, H. Khambhaita, T. Kucner, B. Leibe, A. J. Lilienthal, T. Linder, M. Lohse, M. Magnusson, B. Okal, L. Palmieri, U. Rafi, M. van Rooij, and L. Zhang.SPENCER: A Socially Aware Service Robot for Passenger Guid...

    work page doi:10.1007/978-3-319-27702-8 2016

  3. [3]

    Kruse, A

    T. Kruse, A. K. Pandey, R. Alami, and A. Kirsch. Human-aware robot navigation: A survey. Robotics and Autonomous Systems, 61(12):1726–1743, 2013. ISSN 0921-8890. doi:https:// doi.org/10.1016/j.robot.2013.05.007. URLhttps://www.sciencedirect.com/science/ article/pii/S0921889013001048

    doi:10.1016/j.robot.2013.05.007 2013

  4. [4]

    Aghzal, X

    M. Aghzal, X. Yue, E. Plaku, and Z. Yao. Evaluating vision-language models as evaluators in path planning, 2025. URLhttps://arxiv.org/abs/2411.18711

    Pith/arXiv arXiv 2025

  5. [5]

    L. C. Nogueira, P. Thomaz Aquino, and F. Tonidandel. Legibility on social robot navigation: A survey. In2025 Brazilian Conference on Robotics (CROS), volume 1, pages 1–6, 2025. doi:10.1109/CROS66186.2025.11066134

    work page doi:10.1109/cros66186.2025.11066134 2025

  6. [6]

    A. R, C. C, R. R, and P.-G. D. Social robot navigation: a review and benchmarking of learning- based methods.Front. Robot. AI 12:1658643, 2025. doi:10.3389/frobt.2025.1658643

    work page doi:10.3389/frobt.2025.1658643 2025

  7. [7]

    O. A. I. Ram ´ırez, H. Khambhaita, R. Chatila, M. Chetouani, and R. Alami. Robots learning how and where to approach people. In2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pages 347–353, 2016. doi:10.1109/ROMAN. 2016.7745154

    doi:10.1109/roman 2016

  8. [8]

    H. Pan, S. Huang, J. Yang, J. Mi, K. Li, X. You, P. Liang, J. Yang, Y . Liu, J. Zhang, M. Wang, J. Yang, X. Zhang, L. Zhao, M. Chen, J. Zhou, and X. Wei. Robot navigation via foundation language models: A review.ACM Comput. Surv., 58(11), Apr. 2026. ISSN 0360-0300. doi: 10.1145/3802539. URLhttps://doi.org/10.1145/3802539

    doi:10.1145/3802539 2026

  9. [9]

    Huang, J

    Y . Huang, J. Sansom, Z. Ma, F. Gervits, and J. Chai. Drivlme: Enhancing llm-based au- tonomous driving agents with embodied and social experiences. In2024 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS), pages 3153–3160. IEEE, 2024

    2024

  10. [10]

    Z. Gong, T. Hu, R. Qiu, and J. Liang. From cognition to precognition: A future-aware frame- work for social navigation, 2025. URLhttps://arxiv.org/abs/2409.13244

    Pith/arXiv arXiv 2025

  11. [11]

    M. Korkmaz. Human-aware dynamic path planning. In2021 International Conference on INnovations in Intelligent SysTems and Applications (INISTA), pages 1–5. IEEE, 2021

    2021

  12. [12]

    S. B. Banisetty, S. Forer, L. Yliniemi, M. Nicolescu, and D. Feil-Seifer. Socially aware navi- gation: A non-linear multi-objective optimization approach.ACM Transactions on Interactive Intelligent Systems (TiiS), 11(2):1–26, 2021

    2021

  13. [13]

    P. T. Singamaneni, A. Favier, and R. Alami. Human-aware navigation planner for diverse human-robot interaction contexts. In2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5817–5824. IEEE, 2021. 12

    2021

  14. [14]

    Kollmitz, K

    M. Kollmitz, K. Hsiao, J. Gaa, and W. Burgard. Time dependent planning on a layered social cost map for human-aware robot navigation. In2015 European Conference on Mobile Robots, ECMR 2015 - Proceedings, United States, Nov. 2015. Institute of Electrical and Electronics Engineers Inc. doi:10.1109/ecmr.2015.7324184. European Conference on Mobile Robots, ECM...

    work page doi:10.1109/ecmr.2015.7324184 2015

  15. [15]

    Talebpour, D

    Z. Talebpour, D. Viswanathan, R. Ventura, G. Englebienne, and A. Martinoli. Incorporating perception uncertainty in human-aware navigation: A comparative study. In2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pages 570–577. Ieee, 2016

    2016

  16. [16]

    Vega-Magro, L

    A. Vega-Magro, L. Manso, P. Bustos, P. N´u˜nez, and D. G. Macharet. Socially acceptable robot navigation over groups of people. In2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pages 1182–1187. IEEE, 2017

    2017

  17. [17]

    Luber, L

    M. Luber, L. Spinello, J. Silva, and K. O. Arras. Socially-aware robot navigation: A learning approach. In2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 902–907. IEEE, 2012

    2012

  18. [18]

    P ´erez-Higueras, F

    N. P ´erez-Higueras, F. Caballero, and L. Merino. Learning human-aware path planning with fully convolutional networks. In2018 IEEE international conference on robotics and automa- tion (ICRA), pages 5897–5902. IEEE, 2018

    2018

  19. [19]

    Karnan, A

    H. Karnan, A. Nair, X. Xiao, G. Warnell, S. Pirk, A. Toshev, J. Hart, J. Biswas, and P. Stone. Socially compliant navigation dataset (scand): A large-scale dataset of demonstrations for social navigation.IEEE Robotics and Automation Letters, 7(4):11807–11814, 2022

    2022

  20. [20]

    Brito, M

    B. Brito, M. Everett, J. P. How, and J. Alonso-Mora. Where to go next: Learning a subgoal rec- ommendation policy for navigation in dynamic environments.IEEE Robotics and Automation Letters, 6(3):4616–4623, 2021

    2021

  21. [21]

    Y . Luo, P. Cai, A. Bera, D. Hsu, W. S. Lee, and D. Manocha. Porca: Modeling and planning for autonomous driving among many pedestrians.IEEE Robotics and Automation Letters, 3 (4):3418–3425, 2018

    2018

  22. [22]

    Charalampous, I

    K. Charalampous, I. Kostavelis, and A. Gasteratos. Robot navigation in large-scale social maps: An action recognition approach.Expert Systems with Applications, 66:261–273, 2016

    2016

  23. [23]

    Peddi, C

    R. Peddi, C. Di Franco, S. Gao, and N. Bezzo. A data-driven framework for proactive intention- aware motion planning of a robot in a human environment. In2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5738–5744. IEEE, 2020

    2020

  24. [24]

    P. T. Singamaneni, A. Favier, and R. Alami. Watch out! there may be a human. addressing in- visible humans in social navigation. In2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 11344–11351. IEEE, 2022

    2022

  25. [25]

    Vasquez, B

    D. Vasquez, B. Okal, and K. O. Arras. Inverse reinforcement learning algorithms and features for robot navigation in crowds: an experimental comparison. In2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1341–1346. IEEE, 2014

    2014

  26. [26]

    Valiente, B

    R. Valiente, B. Toghi, R. Pedarsani, and Y . P. Fallah. Robustness and adaptability of reinforce- ment learning-based cooperative autonomous driving in mixed-autonomy traffic.IEEE Open Journal of Intelligent Transportation Systems, 3:397–410, 2022

    2022

  27. [27]

    L. Liu, D. Dugas, G. Cesari, R. Siegwart, and R. Dub ´e. Robot navigation in crowded envi- ronments using deep reinforcement learning. In2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5671–5677. IEEE, 2020. 13

    2020

  28. [28]

    X. Sun, Y . Zhang, X. Tang, A. S. Bedi, and A. Bera. Trustnavgpt: Modeling uncertainty to improve trustworthiness of audio-guided llm-based robot navigation. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8794–8801. IEEE, 2024

    2024

  29. [29]

    J. Chen, Y . Li, P. Jiang, J. Du, Z. Chen, C. Tie, J. Deng, and L. Shao. Lisn: Language-instructed social navigation with vlm-based controller modulating.arXiv preprint arXiv:2512.09920, 2025

    arXiv 2025

  30. [30]

    Narasimhan, A

    S. Narasimhan, A. H. Tan, D. Choi, and G. Nejat. Olivia-nav: An online lifelong vision language approach for mobile robot social navigation. In2025 IEEE international conference on robotics and automation (ICRA), pages 9130–9137. IEEE, 2025

    2025

  31. [31]

    A. J. Sathyamoorthy, K. Weerakoon, M. Elnoor, A. Zore, B. Ichter, F. Xia, J. Tan, W. Yu, and D. Manocha. Convoi: Context-aware navigation using vision language models in outdoor and indoor environments. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 13837–13844. IEEE, 2024

    2024

  32. [32]

    Zhang, X

    L. Zhang, X. Hao, X. Bu, Y . Tang, H. Li, J. Lu, X.-s. Wei, J. Ma, Y . Liu, J. Zhang, et al. Walk with me: Long-horizon social navigation for human-centric outdoor assistance.arXiv preprint arXiv:2604.26839, 2026

    Pith/arXiv arXiv 2026

  33. [33]

    D. Song, J. Liang, A. Payandeh, A. H. Raj, X. Xiao, and D. Manocha. Vlm-social-nav: Socially aware robot navigation through scoring using vision-language models.IEEE Robotics and Automation Letters, 10(1):508–515, 2024

    2024

  34. [34]

    Z. Fang, A. Xiao, D. Hsu, and G. H. Lee. From obstacles to etiquette: Robot social navigation with vlm-informed path selection.IEEE Robotics and Automation Letters, 2026

    2026

  35. [35]

    S. K. Ramakrishnan, A. Gokaslan, E. Wijmans, O. Maksymets, A. Clegg, J. Turner, E. Un- dersander, W. Galuba, A. Westbury, A. X. Chang, et al. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai.arXiv preprint arXiv:2109.08238, 2021

    Pith/arXiv arXiv 2021

  36. [36]

    Chang, A

    A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner, M. Savva, S. Song, A. Zeng, and Y . Zhang. Matterport3d: Learning from rgb-d data in indoor environments.arXiv preprint arXiv:1709.06158, 2017

    Pith/arXiv arXiv 2017

  37. [37]

    Payandeh, D

    A. Payandeh, D. Song, M. Nazeri, J. Liang, P. Mukherjee, A. H. Raj, Y . Kong, D. Manocha, and X. Xiao. Social-llava: Enhancing robot navigation through human-language reasoning in social spaces.arXiv preprint arXiv:2501.09024, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    Z. Liu, X. Zhang, Z. Wang, T. Kawabata, X. Xiao, and L. Xiao. Muson: A reasoning-oriented multimodal dataset for socially compliant navigation in urban environments.arXiv preprint arXiv:2512.22867, 2025

    Pith/arXiv arXiv 2025

  39. [39]

    Zhang, E

    L. Zhang, E. Xiao, X. Hao, H. Fu, Z. Gong, L. Chen, X. Liang, R. Xu, H. Ye, and W. Ding. Socialnav-map: Dynamic mapping with human trajectory prediction for zero-shot social navi- gation.arXiv preprint arXiv:2511.12232, 2025

    arXiv 2025

  40. [40]

    P. E. Hart, N. J. Nilsson, and B. Raphael. A formal basis for the heuristic determination of minimum cost paths.IEEE Transactions on Systems Science and Cybernetics, 4(2):100–107,

  41. [41]

    doi:10.1109/TSSC.1968.300136

    doi:10.1109/tssc.1968.300136 1968

  42. [42]

    van den Berg, S

    J. van den Berg, S. J. Guy, M. Lin, and D. Manocha. Reciprocal n-body collision avoidance. In C. Pradalier, R. Siegwart, and G. Hirzinger, editors,Robotics Research, pages 3–19, Berlin, Heidelberg, 2011. Springer Berlin Heidelberg. ISBN 978-3-642-19457-3. 14

    2011

  43. [43]

    X. Puig, E. Undersander, A. Szot, M. Dallaire Cote, T.-Y . Yang, R. Partsey, R. Desai, A. Clegg, M. Hlavac, S. Y . Min, et al. Habitat 3.0: A co-habitat for humans, avatars, and robots. In International Conference on Learning Representations, volume 2024, pages 15306–15336, 2024

    2024

  44. [44]

    T. Hu, Z. Gong, L. Kong, X. Mei, Y . Ding, Q. Zeng, A. Liang, R. Li, Y . Zhong, and J. Liang. Navthinker: Action-conditioned world models for coupled prediction and planning in social navigation, 2026. URLhttps://arxiv.org/abs/2603.15359. 15

    arXiv 2026

This paper was first reviewed by grok-4.5 on July 14, 2026.