Pith. sign in

REVIEW 4 major objections 4 minor 46 references

A vision-language model's social judgments about people and places can be turned into a navigation costmap, and with semantic verification and a safety reflex zone this yields safe, efficient, socially compliant robot navigation in unstruct

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 19:25 UTC pith:APOEAMSZ

load-bearing objection Architecture is sensible and the paper is honest, but the social-compliance claim rests on a VLM scoring step that is never specified or validated. the 4 major comments →

arxiv 2607.16956 v1 pith:APOEAMSZ submitted 2026-07-18 cs.RO

G2-Nav: Grounded and Guarded Vision-Language Costmaps for Robot Social Navigation

Yuwen Liao , Yihang Lan , Yizhuo Yang , Ruimeng Liu , Xinhang Xu , Shenghai Yuan , Lihua Xie This is my paper
classification cs.RO
keywords social navigationvision-language modelcostmapopen-set perceptionsafety reflextraversabilityrobot autonomysemantic verification
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

G2-Nav tries to close the gap between a vision-language model's broad social understanding and the reliability needed for an autonomous robot moving through people. Instead of letting the VLM issue trajectories or pick from sampled paths, the paper converts its judgments about traversable ground, relevant agents, and social scores into a costmap that a classical planner can optimize. The authors argue that this keeps the model's semantic richness while giving the robot a mathematically grounded, inspectable interface, and they add two guards: VLM verification of upstream tracking, and a high-frequency reflex zone that catches unregistered nearby obstacles. Real-world tests on a recorded dataset and a crowded campus corridor report lower safety-violation time and better trajectory alignment with human operators than end-to-end and instruction-following baselines. If the approach holds, it offers a cheap way to make VLM commonsense about social norms actually drive a physical robot.

Core claim

The central object is the vision-language costmap C = λ1·C_goal + λ2·C_obs + λ3·C_obj + C_trav, where the social-agent term C_obj = Σ_j s_j·N(x_j, Σ(v_j)) places an elliptical Gaussian of height equal to the VLM-assigned social score s_j at each relevant agent's tracked position and velocity-encoded covariance. The VLM also supplies a binary traversability mask M (via SAM) used in C_trav, and the paper adds two safeguards: upstream verification, in which the VLM checks whether a tracked object's depth and heading match the image stream and triggers re-registration when they do not, and a dynamic reflex zone along the predicted trajectory that injects high cost for any unregistered LiDAR poin

What carries the argument

The vision-language costmap: a weighted sum of goal attraction, occupancy-based obstacle cost, a social-agent term formed by elliptical Gaussians whose heights are VLM-assigned social scores, and a traversability mask cost. It is the interface that turns the VLM's open-set perception and semantic scoring into a smooth, bounded cost space a gradient-descent planner can optimize. The other two load-bearing pieces are the upstream verification loop (VLM checks depth/heading consistency of tracked objects and triggers re-registration) and the reflex zone (a geometric region along the predicted trajectory that flags unregistered LiDAR points as immediate threats).

Load-bearing premise

The load-bearing premise is that the VLM's per-object social score s_j, elicited by an unspecified prompt and rubric, is a meaningful quantitative cost; Sec. 3.3 never defines the score's scale, prompt, or shows agreement with human judgments, so if these scores are miscalibrated or noisy the social-agent term C_obj becomes arbitrary and the social-compliance claims rest on an unmeasured quantity.

What would settle it

Have human annotators rate the same scenes with the same objects as the VLM using a fixed rubric, and measure rank correlation with the social scores used in C_obj; near-zero or negative correlation on a held-out set of SCAND trials would show the social cost is not a faithful grounding of human norms. Alternatively, a simple probe: perturb all VLM scores by a fixed offset and check whether the planned trajectories change; if they do not, the social term is inert; if they change drastically, the system is hypersensitive to an unvalidated number.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A VLM can contribute to full autonomy without being the final planner: its outputs become costs, so its failures are bounded by the costmap's safety terms rather than becoming direct dangerous actions.
  • Social compliance can be achieved with classic controllers (gradient descent + pure pursuit) once the costmap encodes social context, meaning no task-specific training or fine-tuning of the VLM is required.
  • Asynchronous VLM inference no longer blocks the control loop: the costmap updates whenever a VLM response arrives, while the reflex zone covers the latency gap.
  • Trajectories from the system align more closely with human expert demonstrations than a VLM trajectory-selection baseline using the same VLM, suggesting costmap grounding preserves the model's semantic understanding while improving behavioral naturalness.
  • The safety reflex zone reduces safety-violation time in interactive deployments even though the human-operated recorded dataset rarely requires it, indicating the guard matters for autonomous operation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the VLM's social scores were calibrated against human ratings (for instance, on the same recorded-driving frames), the social-agent term could double as an interpretable audit log: a supervisor could read which object the robot considered dangerous and why, which is harder with end-to-end policies.
  • The same costmap grounding recipe could extend beyond social etiquette to other abstract semantics—traffic rules, construction zones, or animal encounters—wherever a VLM can produce a score and a mask, making the framework a general 'semantic costmap' template rather than a social-navigation-specific fix.
  • A natural stress test would be to deploy the reflex zone alone (VLM costs removed) in a corridor with fast-moving pedestrians: if safety-violation time stays near zero while social compliance drops, the paper's safety claims are robust to VLM score miscalibration, but its social-compliance claims are not.
  • The paper's own limitation about edge-device latency suggests a concrete improvement path: distilling the VLM or replacing the multi-step open-set pipeline with a single compact network could cut the 4-second query time on the robot, making the async costmap update nearly continuous.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes G2-Nav, an autonomous social navigation framework that uses a pretrained vision-language model (VLM) to produce traversability masks and per-agent social scores, which are then grounded into a costmap for trajectory generation. A high-frequency reflex check is added to mitigate latency and unregistered obstacles. The method is evaluated on 83 trials of the recorded SCAND dataset via trajectory-similarity metrics, and in real-world interactive deployments on a Unitree Go2-W against DWA, Social Force, PeopleAsPlanner, CityWalker, and path-etiquette. The central claim is that VLM abstract social reasoning can be made reliable and interpretable for real-time robot control by grounding it in a costmap.

Significance. If substantiated, this is a useful contribution to social navigation: it avoids end-to-end VLM trajectory generation and preserves interpretability through a costmap representation. The paper's strengths include real-world hardware experiments, open-set object detection, a safety reflex layer, and ablation studies. The idea of using VLM scores as a bounded social cost is timely and could inspire follow-up work. However, the central social-scoring mechanism is under-specified and unvalidated, and key evaluation choices (hyperparameter tuning on the test set, small interactive sample with no error bars) weaken the empirical claims. The contribution is therefore promising but not yet conclusively demonstrated.

major comments (4)
  1. [Sec. 3.3, Eq. (2)] The social cost term C_obj = Σ_j s_j · N(x_j, Σ(v_j)) is the core mechanism through which G2-Nav claims social compliance. Yet Sec. 3.3 never specifies the VLM prompt, the scoring rubric, the numerical scale of s_j, or the default score assigned to newly detected objects. Nor is there any experiment measuring s_j directly or comparing VLM scores with human/expert judgments. The 'w/o scoring' ablation in Table 1 shows only that removing the scores changes L2/MAOE, which is also true for arbitrary or systematically biased scores; sensitivity alone does not establish validity. The interactive metrics (SVT, Tavg, Davg) measure geometry and efficiency, not whether the score encoding is socially correct. As written, many scoring functions consistent with the paper's qualitative description would produce different robot behaviors while still matching aggregate numbers. This is load-bearing for
  2. [Sec. 4.1, Implementation Details; Table 1] The costmap weights λ1:3 are chosen by grid search on 'the longest trial in SCAND' and the method is then evaluated on the SCAND recorded setting (83 trials). The paper does not state whether the tuning trial is excluded from the 83-trial average; as written, the reported L2/MAOE scores include data used for hyperparameter selection. This is a form of leakage for the main recorded-setting result. The authors should report results with the tuning trial excluded or with a proper validation split, report the chosen λ values, and provide a sensitivity analysis. This is important because the quantitative margin over baselines is the primary evidence for the social-compliance claim.
  3. [Sec. 4.3, Table 2] The interactive results are based on only 6 trials per method, with no error bars, variance, or statistical significance tests. In particular, G2-Nav's SVT is 6.9s, which is worse than Social Force's 4.4s; the text explains why SF's behavior is socially undesirable, but the safety metric itself is still worse. The claim of 'strong safety awareness' is not statistically supported. The authors should report per-trial values, confidence intervals, and ideally the number of reflex-brake events or near-collision episodes, and discuss the SVT gap explicitly.
  4. [Sec. 4.2, Table 1] The safeguard is presented as a key component, but the ablation shows 'w/o safeguard' yields L2 0.48 versus 0.55 with the safeguard, while MAOE is 15.90 versus 15.64. The quantitative evidence that the safeguard improves safety is absent; the paper relies on a single qualitative example (Fig. 4d). Since safety is one of the three stated contributions, the authors should quantify safeguard interventions during interactive runs—for example, the frequency of reflex responses, the reduction in near-collision events, and the latency the reflex zone compensates for.
minor comments (4)
  1. [Tables 1 and 2] Use consistent formatting for metric names (Tavg, Davg) and include standard deviations or confidence intervals. Also note in the table captions which numbers are averages over which set of trials.
  2. [Appendix B, Eq. (3)] The piecewise definition of the reflex zone is typeset poorly; the square-root and θ definitions are not visually clear. Please reformat for readability.
  3. [Figures 4 and 5] The VLM response snippets and costmap overlays are too small to be read in the current figures. Enlarge them or quote the relevant text in the captions.
  4. [Sec. 4.1] The model names 'QWen3.5-2B' and 'GPT5.4-Nano' appear to be future/placeholder versions; make sure the exact versions and prompts used are documented in the supplementary material, and that the VLM responses are reproducible.

Circularity Check

1 steps flagged

Mild data leak: costmap weights are tuned on a SCAND trial and the method is then scored on the full SCAND set; no equation-level circular derivation found.

specific steps
  1. fitted input called prediction [Sec. 4.1 (Implementation Details) + Table 1 caption]
    "We find the set of costmap weights λ1:3 through grid search using the longest trial in SCAND, and use the same values in both experiment settings. ... Results are averaged from 83 trials."

    The three weights λ1:3 directly determine the costmap used for planning (Eq. 1), and they are selected by grid search on one SCAND trial. The paper then reports Table 1's L2/MAOE as an average over 83 SCAND trials, with no statement that the tuning trial was held out from that aggregate or from the baselines. Therefore the claimed predictive performance on the recorded setting is partially in-sample for the trial used to select the weights. This is a leak rather than a full circular reduction: the VLM social scores are not fitted to the target metrics, the interactive setting is an independent (though same-weights) deployment, and only three scalar weights are tuned; still, the evaluation is not fully external to the tuning data.

full rationale

The derivation chain runs from sensor data through open-set perception, VLM social scoring, Eqs. (1)-(2) costmap formation, and gradient-descent planning. I found no equation or definition in which an output quantity is defined in terms of the quantity it is claimed to predict. The social scores s_j and traversable mask M are VLM outputs; they are not fitted to L2/MAOE/SVT, and the costmap is not solved backwards from the evaluation metrics. The social-leader mechanism comes from a shared-author prior work (Ref [38]), but it is presented as an input design choice ('we also ask the VLM to assign a special negative score'), not as an external uniqueness theorem or as the sole justification of the central claim. The concrete circularity-adjacent issue is experimental: λ1:3 are tuned on the longest SCAND trial and then the method is scored on the full 83-trial SCAND set, so the headline recorded-setting numbers are partly in-sample. I weight this as a mild leak (3), not a forced reduction: the costmap terms are not defined by the metrics, the interactive trials provide a separate deployment check, and the openness about grid-search tuning lets a reader see the leak. The under-specification of the VLM scoring rubric (Sec. 3.3) is a validity/calibration concern, not a circularity, so it does not raise the score further.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

Everything the central claim rests on: the VLM's semantic judgments (scores, traversability, verification) are taken at face value; the costmap weights are tuned on a trial from the evaluation set; the reflex zone parameters are hand-set. The mathematical machinery (Gaussian costs, gradient descent) is standard.

free parameters (3)
  • costmap weights λ1, λ2, λ3 = not reported (grid search on longest SCAND trial)
    Tuned on SCAND and reused in both settings.
  • reflex zone parameters T, W, ε = T=1.5s, W=0.8m, ε=0.05
    Set by hand in Appendix B.
  • default social score for newly detected objects = unspecified
    Objects get a default score before VLM analysis; value not stated (Section 3.3).
axioms (4)
  • domain assumption VLM zero-shot social scores correlate with human social norms
    The central social cost C_obj uses s_j produced by the VLM without ground-truth validation.
  • domain assumption Negative-score social leaders provide useful implicit navigation guidance
    Borrowed from PAP [38]; the paper adopts it without re-validation.
  • domain assumption Unregistered LiDAR points in the reflex zone are immediate threats
    Safety layer ignores known static objects and treats all else as dangerous; could cause false stops.
  • standard math Classical potential-field/costmap optimization yields feasible trajectories
    Assumes gradient descent on the costmap gives local minima that are collision-free within the planning horizon.

pith-pipeline@v1.3.0-alltime-deepseek · 9832 in / 11493 out tokens · 100411 ms · 2026-08-01T19:25:20.855901+00:00 · methodology

0 comments
read the original abstract

Social navigation requires the robot to reason and respond in complex real-world environments. While recent works attempt to incorporate human-level intelligence into robot planning using large Vision-Language Models (VLMs), end-to-end frameworks often create an unpredictable black-box, and existing instruction-following methods are not designed for full autonomy. To bridge this gap, we present G2-Nav, a novel framework that grounds abstract social reasoning and guards safe real-world deployment. Instead of asking the VLM for direct planning decisions, G2-Nav translates its semantic reasoning into a vision-language costmap with reliability and interpretability. The VLM evaluates traversable regions and social agents from open-set perception, mapping social context into the costmap. To improve real-world robustness, the VLM performs semantic verification on upstream tracking, and we introduce a high-frequency safety check to guard against system latency prior to trajectory generation. We demonstrate through real-world experiments that G2-Nav delivers safe, efficient, and socially compliant autonomous navigation in unstructured environments. Code will be made publicly available.

Figures

Figures reproduced from arXiv: 2607.16956 by Lihua Xie, Ruimeng Liu, Shenghai Yuan, Xinhang Xu, Yihang Lan, Yizhuo Yang, Yuwen Liao.

Figure 1
Figure 1. Figure 1: Grounding social context rea￾soning into vision-language costmap. Understanding social context is crucial for autonomous robots navigating in human-centric environments. Be￾yond basic obstacle avoidance, early works rely on hand￾crafted behavioural rules [1] and assumptions [2] that struggle to capture dynamic human interactions and un￾spoken social norms. While subsequent works have pro￾posed various data… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the G2-Nav framework. In this example, the VLM suggests that person [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Example scenarios from recorded (a) and interactive (b) settings. 4.1 Experiment Settings Scenarios. SCAND originally consists of 138 trials of robot onboard data. We remove trials that are too short, missing topics, and those that involve stair-climbing, which yields 83 trials and a total duration of 5.6 hours. We conduct the real-world experiment in a 150m-long crowded campus corridor connecting several … view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative results under recorded setting. On the left, we project ground-truth trajectories [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative results under interactive setting. We project the robot trajectories from the next [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Example of the upstream verification step. The VLM response snippet suggests that [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 3 linked inside Pith

  1. [1]

    Helbing and P

    D. Helbing and P. Molnar. Social force model for pedestrian dynamics.Physical Review E, 51 (5):4282, 1995

    1995

  2. [2]

    Van Den Berg, S

    J. Van Den Berg, S. J. Guy, M. Lin, and D. Manocha. Reciprocal n-body collision avoidance. InRobotics Research: The 14th International Symposium ISRR, pages 3–19. Springer, 2011

    2011

  3. [3]

    Korbmacher and A

    R. Korbmacher and A. Tordeux. Review of pedestrian trajectory prediction methods: Compar- ing deep learning and knowledge-based approaches.IEEE Transactions on Intelligent Trans- portation Systems, 23(12):24126–24144, 2022

    2022

  4. [4]

    D. M. Nguyen, M. Nazeri, A. Payandeh, A. Datar, and X. Xiao. Toward human-like so- cial robot navigation: A large-scale, multi-modal, social human navigation dataset. In2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7442–

  5. [5]

    Kawaharazuka, J

    K. Kawaharazuka, J. Oh, J. Yamada, I. Posner, and Y . Zhu. Vision-language-action models for robotics: A review towards real-world applications.IEEE Access, 2025

    2025

  6. [6]

    Huang, T

    X. Huang, T. Chen, M. Bryson, T. Chaffey, H. Chen, K.-K. R. Choo, I. R. Manchester, et al. Trust in llm-controlled robotics: a survey of security threats, defenses and challenges.arXiv preprint arXiv:2601.02377, 2025

    arXiv 2025

  7. [7]

    Q. Li, B. Yin, W. Huang, R. Liu, B. Zou, R. Yu, J. Ye, W. Yu, and X. Wang. Vision- language-action safety: Threats, challenges, evaluations, and mechanisms.arXiv preprint arXiv:2604.23775, 2026

    Pith/arXiv arXiv 2026

  8. [8]

    Rimon.Exact robot navigation using artificial potential functions

    E. Rimon.Exact robot navigation using artificial potential functions. Yale University, 1990

    1990

  9. [9]

    S. S. Ge and Y . J. Cui. Dynamic motion planning for mobile robots using potential field method.Autonomous robots, 13(3):207–222, 2002

    2002

  10. [10]

    Papadakis, A

    P. Papadakis, A. Spalanzani, and C. Laugier. Social mapping of human-populated environ- ments by implicit function learning. In2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1701–1706. IEEE, 2013

    2013

  11. [11]

    Weerakoon, M

    K. Weerakoon, M. Elnoor, G. Seneviratne, V . Rajagopal, S. H. Arul, J. Liang, M. K. M. Jaffar, and D. Manocha. Behav: Behavioral rule guided autonomy using vlms for robot navigation in outdoor scenes. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 7044–7051. IEEE, 2025

    2025

  12. [12]

    M. Yuan, L. Wang, and S. L. Waslander. Opennav: Open-world navigation with multimodal large language models. In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 18948–18955. IEEE, 2025

    2025

  13. [13]

    Huang, O

    C. Huang, O. Mees, A. Zeng, and W. Burgard. Visual language maps for robot navigation. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 10608– 10615. IEEE, 2023

    2023

  14. [14]

    S. Liu, H. Xia, F. C. Pouria, K. Hong, N. Chakraborty, Z. Hu, J. Biswas, and K. Driggs- Campbell. Height: Heterogeneous interaction graph transformer for robot navigation in crowded and constrained environments.IEEE Transactions on Automation Science and Engi- neering, 23:1211–1230, 2025

    2025

  15. [15]

    D. Shah, A. Sridhar, N. Dashora, K. Stachowicz, K. Black, N. Hirose, and S. Levine. ViNT: A foundation model for visual navigation. In7th Annual Conference on Robot Learning, 2023. URLhttps://arxiv.org/abs/2306.14846. 9

    Pith/arXiv arXiv 2023

  16. [16]

    Sridhar, D

    A. Sridhar, D. Shah, C. Glossop, and S. Levine. Nomad: Goal masked diffusion policies for navigation and exploration. In2024 IEEE International Conference on Robotics and Automa- tion (ICRA), pages 63–70. IEEE, 2024

    2024

  17. [17]

    A. Bar, G. Zhou, D. Tran, T. Darrell, and Y . LeCun. Navigation world models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 15791–15801, 2025

    2025

  18. [18]

    X. Liu, J. Li, Y . Jiang, N. Sujay, Z. Yang, J. Zhang, J. Abanes, J. Zhang, and C. Feng. City- walker: Learning embodied urban navigation from web-scale videos. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 6875–6885, 2025

    2025

  19. [19]

    G. Zhou, Y . Hong, and Q. Wu. Navgpt: Explicit reasoning in vision-and-language navigation with large language models. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 7641–7649, 2024

    2024

  20. [20]

    Z. Chen, Y . Guo, Z. Chu, M. Luo, Y . Shen, M. Sun, J. Hu, S. Xie, K. Yang, P. Shi, et al. So- cialnav: Training human-inspired foundation model for socially-aware embodied navigation. arXiv preprint arXiv:2511.21135, 2025

    arXiv 2025

  21. [21]

    D. Song, J. Liang, A. Payandeh, A. H. Raj, X. Xiao, and D. Manocha. Vlm-social-nav: Socially aware robot navigation through scoring using vision-language models.IEEE Robotics and Automation Letters, 10(1):508–515, 2024

    2024

  22. [22]

    A. J. Sathyamoorthy, K. Weerakoon, M. Elnoor, A. Zore, B. Ichter, F. Xia, J. Tan, W. Yu, and D. Manocha. Convoi: Context-aware navigation using vision language models in outdoor and indoor environments. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 13837–13844. IEEE, 2024

    2024

  23. [23]

    Nasiriany, F

    S. Nasiriany, F. Xia, W. Yu, T. Xiao, J. Liang, I. Dasgupta, A. Xie, D. Driess, A. Wahid, Z. Xu, et al. Pivot: iterative visual prompting elicits actionable knowledge for vlms. InProceedings of the 41st International Conference on Machine Learning, pages 37321–37341, 2024

    2024

  24. [24]

    Z. Fang, A. Xiao, D. Hsu, and G. H. Lee. From obstacles to etiquette: Robot social navigation with vlm-informed path selection.IEEE Robotics and Automation Letters, 2026

    2026

  25. [25]

    D. Song, J. Liang, X. Xiao, and D. Manocha. Vl-tgs: Trajectory generation and selection using vision language models in mapless outdoor environments.IEEE Robotics and Automation Letters, 2025

    2025

  26. [26]

    Potnis, F

    A. Potnis, F. Affonso, S. Gummadi, N. K. Uppalapati, and G. Chowdhary. Catnav: Cached vision-language traversability for efficient zero-shot robot navigation.arXiv preprint arXiv:2603.22800, 2026

    arXiv 2026

  27. [27]

    J. Chen, Y . Li, P. Jiang, J. Du, Z. Chen, C. Tie, J. Deng, and L. Shao. Lisn: Language-instructed social navigation with vlm-based controller modulating.arXiv preprint arXiv:2512.09920, 2025

    arXiv 2025

  28. [28]

    Zhang, E

    L. Zhang, E. Xiao, X. Hao, H. Fu, Z. Gong, L. Chen, X. Liang, R. Xu, H. Ye, and W. Ding. Socialnav-map: Dynamic mapping with human trajectory prediction for zero-shot social navi- gation.arXiv preprint arXiv:2511.12232, 2025

    arXiv 2025

  29. [29]

    Ravichandran, D

    Z. Ravichandran, D. Snyder, A. Robey, H. Hassani, V . Kumar, and G. J. Pappas. Contextual safety reasoning and grounding for open-world robots.arXiv preprint arXiv:2602.19983, 2026

    arXiv 2026

  30. [30]

    Elnoor, K

    M. Elnoor, K. Weerakoon, G. Seneviratne, J. Liang, V . Rajagopal, and D. Manocha. Vi-lad: Vision-language attention distillation for socially-aware robot navigation in dynamic environ- ments.arXiv preprint arXiv:2503.09820, 2025. 10

    arXiv 2025

  31. [31]

    D. Shah, A. Sridhar, A. Bhorkar, N. Hirose, and S. Levine. Gnm: A general navigation model to drive any robot. In2023 IEEE International Conference on Robotics and Automation (ICRA), pages 7226–7233. IEEE, 2023

    2023

  32. [32]

    Zhang, X

    Y . Zhang, X. Huang, J. Ma, Z. Li, Z. Luo, Y . Xie, Y . Qin, T. Luo, Y . Li, S. Liu, et al. Recognize anything: A strong image tagging model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1724–1732, 2024

    2024

  33. [33]

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European conference on computer vision, pages 38–55. Springer, 2024

    2024

  34. [34]

    Ester, H.-P

    M. Ester, H.-P. Kriegel, J. Sander, X. Xu, et al. A density-based algorithm for discovering clusters in large spatial databases with noise. Inkdd, volume 96, pages 226–231, 1996

    1996

  35. [35]

    Zhou and O

    Y . Zhou and O. Tuzel. V oxelnet: End-to-end learning for point cloud based 3d object detection. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 4490–4499, 2018

    2018

  36. [36]

    L. Yang, B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10371–10381, 2024

    2024

  37. [37]

    Kirillov, E

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, et al. Segment anything. InProceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026, 2023

    2023

  38. [38]

    Y . Liao, X. Xu, R. Bai, Y . Yang, M. Cao, S. Yuan, and L. Xie. Following is all you need: Robot crowd navigation using people as planners.IEEE Robotics and Automation Letters, 2025

    2025

  39. [39]

    Karnan, A

    H. Karnan, A. Nair, X. Xiao, G. Warnell, S. Pirk, A. Toshev, J. Hart, J. Biswas, and P. Stone. Socially compliant navigation dataset (scand): A large-scale dataset of demonstrations for social navigation.IEEE Robotics and Automation Letters, 7(4):11807–11814, 2022

    2022

  40. [40]

    Qwen3.5: Towards native multimodal agents, February 2026

    Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URLhttps: //qwen.ai/blog?id=qwen3.5

    2026

  41. [41]

    L. Wang, X. Zhang, C. Li, D. He, Y . Pan, and J. Yi. Super-lio: A robust and efficient lidar- inertial odometry system with a compact mapping strategy.IEEE Robotics and Automation Letters, 11(3):2666–2673, 2026

    2026

  42. [42]

    Francis, C

    A. Francis, C. P ´erez-d’Arpino, C. Li, F. Xia, A. Alahi, R. Alami, A. Bera, A. Biswas, J. Biswas, R. Chandra, et al. Principles and guidelines for evaluating social robot navigation algorithms. ACM Transactions on Human-Robot Interaction, 14(2):1–65, 2025

    2025

  43. [43]

    D. Fox, W. Burgard, and S. Thrun. The dynamic window approach to collision avoidance. IEEE Robotics & Automation Magazine, 4(1):23–33, 1997

    1997

  44. [44]

    Hirose, C

    N. Hirose, C. Glossop, D. Shah, and S. Levine. Asyncvla: An asynchronous vla for fast and robust navigation on the edge.arXiv preprint arXiv:2602.13476, 2026

    arXiv 2026

  45. [45]

    Ravichandran, I

    Z. Ravichandran, I. Hounie, F. Cladera, A. Ribeiro, G. J. Pappas, and V . Kumar. Distilling on-device language models for robot planning with minimal human intervention.Conference on Robot Learning (CoRL), 2025. URLhttps://arxiv.org/abs/2506.17486

    arXiv 2025

  46. [46]

    Sindhwani, S

    V . Sindhwani, S. Tu, and M. Khansari. Learning contracting vector fields for stable imitation learning.arXiv preprint arXiv:1804.04878, 2018. 11 Appendix A Upstream Verification Example Figure 6: Example of the upstream verification step. The VLM response snippet suggests that both the motion and depth information about person 115 mismatch with visual ob...

    Pith/arXiv arXiv 2018