Pith. sign in

REVIEW 4 major objections 4 minor 4 cited by

HALO: Human Preference Aligned Offline Reward Learning for Robot Navigation

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Offline reward learning from human preference rankings gives a vision-based reward that improves robot navigation success rates by at least a third.

desk verdict A promising mix of preference learning and visual attention is undercut by a sign error that reverses the reward on safety-critical frames, and the evaluation is too thin to support the headline numbers. read the letter →

arxiv 2508.01539 v1 pith:D4P2DV6L submitted 2025-08-03 cs.RO

Gershom Seneviratne , Jianyu An , Sahire Ellahy , Kasun Weerakoon , Mohamed Bashir Elnoor , Jonathan Deepak Kannan , Amogha Thalihalla Sunil , Dinesh Manocha This is my paper
classification cs.RO
keywords offlinerewardlearninghumanpreferencealignmentvision-basednavigationPlackett-LucerankingmobilerobotmodelpredictivecontrolDINOv2featuresfeedback
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

HALO is an offline reward-learning method that converts binary human answers to navigation queries into a vision-based reward function for robot navigation. For each training frame it samples a grid of actions around an expert reference action, scores them with a Boltzmann distribution whose sharpness is set by the human's yes/no preferences and a danger flag, and trains a reward model with the Plackett-Luce ranking loss so that more preferred actions receive higher rewards. The learned reward is then used in two ways: to train an offline reinforcement-learning policy and to augment a dynamic-window/MPC planner as an additional cost term. On a Clearpath Husky in indoor, outdoor, and low-light scenes, the paper reports at least a 33.3% improvement in success rate, a 12.9% reduction in normalized trajectory length, and a 26.6% reduction in Fréchet distance to expert trajectories relative to the compared baselines. The motivation is to replace hand-crafted, LiDAR-dependent reward design with human intuition expressed through cheap binary feedback.

What carries the argument

The load-bearing mechanism is action-conditioned visual feature aggregation: the robot's planned path under a candidate action is projected onto the egocentric image through a homography, making a binary trajectory mask that a lightweight CNN converts into a spatial relevance map, and element-wise multiplication with DINOv2 patch embeddings concentrates the reward on image regions the action actually traverses. Around it sits the preference-scoring scheme: a discrete set of velocities and angular velocities is sampled around an expert action, and binary answers to four directional/velocity queries plus a danger flag set the temperatures of a separable Boltzmann distribution so that the preferred action receives at least 95% of the probability mass and undesirable actions receive less than 5%. These scores rank the actions, and the Plackett-Luce loss trains the reward to reproduce that ranking. This pairing of a human-annotated ranking signal with an action-gated visual encoder is what lets a scalar reward replace hand-designed obstacle and goal terms.

What would settle it

Run HALO on non-planar terrain or with a deliberately wrong homography (for example, an incorrect camera height or tilt), and compare its action rankings against the same model with the trajectory mask disabled or shifted; if success rate and Fréchet distance barely change when the mask points at the wrong image regions, then the homography precondition is not the load-bearing part of the method.

Watch

Extended reading notes

Core claim

The central claim is that a single reward function trained offline from human preferences over egocentric RGB images can guide both a learned policy and a classical planner through unseen real-world environments and hardware setups without LiDAR or hand-designed rewards. The reward model takes the current image and a candidate action, projects the action's trajectory onto the image plane via a homography to form a binary mask, and runs that mask through a CNN to produce a spatial weighting that modulates DINOv2 patch embeddings, yielding a scalar reward $R(s_t,a_t)$. Supervision comes from rankings over a discretized action set: answers to queries such as "Can the robot turn left?" set a Boltzmann distribution centered on the preferred action, the distribution assigns preference scores to all sampled actions, and the Plackett-Luce loss maximizes the likelihood of the resulting ranking. Deployed with IQL or TD3-BC for offline policy learning and with a dynamic-window/MPC planner as an extra cost term, the reward yields the reported gains in success rate, normalized trajectory length, and Fréchet distance to human teleoperation, including socially compliant behavior such as avoiding pedestrians and glass walls.

Load-bearing premise

The method rests on the homography that draws the planned path onto the egocentric image being accurate: every robot used for training and deployment must have a known ground plane and camera pose, and if the mask highlights the wrong pixels, the reward model is trained and evaluated on misaligned visual features.

Editorial extensions

If this is right

  • A single preference-labeled dataset can supply rewards for both an offline RL policy and a classical MPC/DWA planner, so one round of annotation transfers across learning-based and classical navigation stacks.
  • Vision-only navigation becomes viable without LiDAR or depth sensing: the reported deployments run at roughly 50 Hz on a laptop with a camera and an RTX 3060 GPU.
  • The reward's apparent transfer across wheeled and legged robots and unseen scenes implies that preference-based rewards may generalize across hardware form factors without retraining.
  • Because the queries are binary and reusable across frames, annotation cost stays low, allowing the authors to release a human-annotated preference dataset for sub-optimal trajectories.
  • LiDAR-invisible obstacles such as glass walls become learnable for a vision-based planner, addressing a known failure mode of geometric obstacle avoidance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same binary-query protocol could become a general-purpose reward-labeling interface for other egocentric robot skills (manipulation, social interaction) whenever candidate actions can be sampled around an expert action.
  • Because the reward scores single frames, it may underweight temporal dynamics; a natural extension is to feed short action-conditioned image sequences or a memory state so the model can react to obstacles that leave the field of view.
  • The homography precondition limits deployment to near-planar ground; testing on stairs, slopes, or rough terrain would likely expose where the trajectory mask assumption breaks, and a learned alternative to the binary mask could widen applicability.
  • A reward trained purely on human intuition may inherit human blind spots, so a practical system would need an explicit safety layer or uncertainty estimate before relying on the learned reward in dense crowds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes HALO, an offline reward-learning algorithm that maps egocentric RGB observations and candidate robot actions to a scalar reward using human binary feedback. Training actions are sampled around a reference action and ranked by a Boltzmann preference score, with a scalar factor λ shaped by user responses to five feasibility/danger queries; a Plackett-Luce loss trains the reward model to reproduce these rankings. The reward is then used to train IQL and TD3-BC policies and to augment DWA/MPC planners. Real-world deployments on a Clearpath Husky in three scenarios are reported, with the abstract claiming at least 33.3% improvement in success rate, 12.9% reduction in normalized trajectory length, and 26.6% reduction in Fréchet distance relative to human expert trajectories. The paper also introduces an action-conditioned visual aggregation mechanism based on a homographically projected trajectory mask.

Significance. If the method worked as described, it would be a practically useful contribution: it removes hand-crafted reward engineering, uses only RGB input, and integrates learned human preferences into both offline RL and classical planning. The real-robot experiments and the inclusion of LiDAR-equipped baselines are strengths, as is the action-conditioned attention idea. However, the core training objective is internally inconsistent on the very safety-critical frames the method targets, and the paper does not yet release code or data (they are only promised). The reported quantitative gains therefore cannot be attributed to the described preference-alignment mechanism, and the evaluation is partly confounded by closeness to expert trajectories that are already the center of the training distribution. The underlying direction is promising, but this manuscript does not yet substantiate its central claims.

major comments (4)
  1. [§4.3, Eq. (4)–(5)] The preference score inverts exactly on the safety-critical frames. For U_danger = 1, Eq. (4) gives λ = −1/(1 + Σ U_i) < 0, so P_ref(v,ω|U) = λ·P(v,ω|U) is a negative multiple of the positive Boltzmann distribution. Since P is maximized at the reference action (v*,ω*), P_ref is minimized there, directly contradicting the sentence in §4.3 that "the reference action always ranked highest." Training the Plackett-Luce loss in Eq. (6) on this ranking teaches the reward model to assign the lowest reward to the corrective action and higher rewards to actions far from it. Because §4.1 states that roughly 100 of the 116 added scenes are deliberately truncated unsafe trajectories, these inverted labels dominate the danger-flagged training signal. The downstream IQL/TD3-BC policies and the MPC/DWA cost term inherit a reward trained on reversed preferences. This is an internal mathematical contradiction, not a tuning issue; the sign must be corrected or the ranking must be defined on a positive score, and the experiments must be re-run before the headline claims can be assessed.
  2. [§5.2, Table 1] The abstract's quantitative claims are not supported as stated. For success rate, HALO+DWA achieves 70, 70, and 40 across the three scenarios, which is at or below DWA's 80, 100, and 0; HALO+TD3-BC averages only 30%. Only HALO+IQL is consistently high, but no aggregation rule or baseline is specified for the claimed "at least 33.3% improvement." Normalized trajectory length is not consistently reduced: in Scenario 1 HALO+IQL has 1.049 versus DWA's 1.043, and in Scenario 3 HALO+IQL has 1.136 versus DWA's 0.205. The paper reports 10 trials per scenario with no variance, confidence intervals, or per-trial statistics. The "at least" phrasing appears to cherry-pick whichever comparison is favorable; the table needs a pre-specified baseline, an aggregation rule, and uncertainty quantification before these improvements can be verified.
  3. [§4.3 and §5.2] There is a close correlation between the training target and two of the three evaluation metrics. The preference scores are Boltzmann distributions centered on the human expert/reference action (Eqs. (2)–(5)), while the evaluation metrics include normalized trajectory length and Fréchet distance to the same style of human expert trajectory. A model that simply reproduces the expert action distribution will score well on those metrics even if it has not learned a generalizable notion of navigational quality. Success rate is less confounded, but the paper does not isolate the contribution of the preference ranking. A concrete control experiment would be to train the same reward architecture on randomly permuted rankings or to compare against a behavior-cloning policy trained on the same frames and action distribution; the current BC baselines are not sufficient to rule out this confounding. This is not by itself a proof that the method is wrong, but it substantially weakens the claimed evidence for human-aligned reward learning.
  4. [§4.5, Fig. 1] The action-conditioned trajectory mask is generated by a homography transform from the planned path, but the paper never reports the camera-to-ground calibration, the assumed ground plane, or the accuracy of this projection on the legged-robot portion of SCAND. The reward model's action conditioning depends entirely on this mask selecting the image region corresponding to the action. On non-planar terrain or with different camera mounts, the mask can point to incorrect image regions, and the paper provides no sensitivity analysis. Since the authors deploy on a Clearpath Husky while much of the training data comes from legged robots, this is a load-bearing reproducibility gap. Please provide the calibration procedure, the rectification assumptions, and at least a qualitative validation of mask alignment.
minor comments (4)
  1. [§3.3, Eq. (1)] The notation "nY" in the product is a typo; it should be a product with limits, e.g., ∏_{j=1}^{n}.
  2. [§4.3] The notation for user preferences is inconsistent: Eq. (4) uses U_i and U_danger, while the text says U ∈ {0,1}^4 for the four directional queries. Please define the full tuple including the danger flag explicitly.
  3. [§5.1 and Table 1] The naming of the behavioral-cloning variant is inconsistent: Table 1 lists HALO + TD3-BC, but §5.3 and Fig. 3 refer to HALO-BC and HER-BC. Please unify the terminology.
  4. [§7.3.1] The focal regression regularizer minimizes an MSE against the target reward r, which is P_ref from Eq. (5); on danger frames this target is negative. If the sign issue is corrected, the paper should clarify what target magnitudes are used, since the scale of λ affects this regularizer.

Circularity Check

2 steps flagged · score 6.0 of 10

Partial circularity: the reward supervision is generated from proximity to the reference/expert action, and two headline metrics (Fréchet distance and normalized trajectory length) measure closeness to that same reference; Eq. 4-5 additionally invert the ranking on danger frames, contradicting the paper's stated 'reference action always ranked highest.'

  1. self definitional [Section 4.3, Eqs. (4)-(5) and the sentence following Eq. (5)]
    "λ(U , Udanger) = { −1/(1+P4 i=1 Ui), if Udanger = 1, 1+P4 i=1 Ui, otherwise. (4) ... Pref(v, ω| U) = λ(U , Udanger) · P (v, ω| U). (5) These scores are used to rank thenv ×nω candidate actions, with the reference action always ranked highest."

    When Udanger=1, Eq. (4) sets λ = −1/(1+Σ4i=1 Ui) < 0. Because P(v,ω|U) is a Boltzmann distribution centered on the reference action (v∗,ω∗), the reference action has the largest P, and multiplying by negative λ makes its Pref the smallest. Hence on every danger-flagged frame the reference/corrective action is ranked last, not 'always ranked highest.' The Plackett-Luce loss in Eq. (6) is then trained on inverted rankings, so the learned reward is anti-aligned with the corrective action by the paper's own definitions. This is a self-definitional contradiction in the training-signal construction.

  2. fitted input called prediction [Section 4.3, Eqs. (2)-(3), and Section 5.2 evaluation metrics]
    "Each action (v, ω) ∈ Alocal is assigned a probability score based on its proximity to (v∗, ω∗), using a separable Boltzmann distribution (Eq. 2, 3): ... These scores are used to rank thenv ×nω candidate actions, with the reference action always ranked highest. The resulting preference-ordered list σ serves as supervision for reward model training."

    The ranking supervision is generated by Eq. (3) as a decreasing function of distance to the reference action (v∗,ω∗), and the reward model is trained via Plackett-Luce to reproduce that ranking. The evaluation then reports Normalized Trajectory Length (Lactual/Lexpert) and Fréchet Distance to the expert trajectory, both of which measure how closely the executed trajectory tracks the same expert/reference actions used to create the labels. The reported reductions in these metrics are therefore largely a readback of the training-label construction rather than an independent test of human alignment. The success-rate metric is not tied to the label construction in the same way, which is why the circularity is partial.

full rationale

HALO's pipeline is mostly externally grounded: SCAND is an independent public dataset, DINOv2 is a frozen pretrained encoder, and the success-rate metric is an external performance benchmark. Self-citations in Related Work are not load-bearing for the central claims. However, two steps reduce the headline results to internal construction. First, the preference scores used as supervision are generated as a Boltzmann distribution centered on the reference action (Eq. 2-3), so the reward model is trained to output high reward for actions near the expert action; the paper's headline Fréchet-distance and normalized-trajectory-length metrics measure exactly that nearness, making those improvements partly self-fulfilling. Second, Eq. (4)-(5) assign a negative scale when Udanger=1, which reverses the ranking the paper explicitly asserts ('the reference action always ranked highest'); on danger frames the corrective action is ranked last, so the Plackett-Luce loss is trained on inverted labels. The latter is an internal contradiction rather than an imported circularity, but it is load-bearing because roughly 100 of the 116 added scenes are negative safety examples. Because success-rate improvements and cross-environment generalization are not directly encoded in the label-generation mechanism, the circularity is partial, not total.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The method relies on several hand-set constants and domain assumptions. The preference scores are produced by a Boltzmann formula with manually tuned temperatures and a scaling factor, rather than learned or validated against independent human judgments. The homography projection assumes a planar ground plane. No new physical entities are introduced.

free parameters (7)
  • Boltzmann temperatures tau_v(U), tau_omega(U) = hand-set to place 95% probability mass on the preferred direction
    Eq. (3); chosen by design, not learned; determines sharpness of preference scores for each binary user response.
  • Lambda scaling coefficients = 1 + sum(U_i) (safe); -1/(1 + sum(U_i)) (danger)
    Eq. (4); hand-designed to scale scores; introduces an ordering inversion for danger cases.
  • Action grid size n_v, n_omega = 5, 5
    Table 3; discretizes the action space for preference scoring and Plackett-Luce supervision.
  • Probability mass threshold = 95%
    Section 4.3; fixed heuristic for temperature selection.
  • Regularization weights lambda1, lambda2, lambda3 = 2.0, 0.01, 0.05
    Table 4; chosen hyperparameters for diversity, L2, and focal losses; the focal exponent gamma and diversity constant c are not reported.
  • HALO weight in MPC/DWA = lambda_halo = 1/5
    Section 7.4.3; hand-set weighting of learned reward in the planner cost.
  • Homography projection parameters = unspecified
    Section 4.5; camera intrinsics, extrinsics, ground plane, and horizon time needed to build the binary trajectory mask; not reported.
assumptions (5)
  • standard math Plackett-Luce model (Eq. 1) and MDP formalism (Section 3) are valid for representing human preference rankings over actions.
    Standard probability/RL background, not proved in the paper.
  • domain assumption A known ground plane and fixed camera pose allow projecting the robot's future path into image coordinates via homography.
    Invoked in Section 4.5; not validated for legged robots or uneven terrain in SCAND.
  • domain assumption Binary answers to four directional queries plus the recorded expert action are sufficient to reconstruct a human preference ranking over the action grid.
    Section 4.3; the mapping is a fixed heuristic, not derived from data or user studies.
  • ad hoc to paper The 95% probability mass heuristic is a reasonable prior for human certainty.
    Section 4.3; chosen by hand, no evidence given.
  • ad hoc to paper The approximately 100 truncated negative trajectories are correctly labeled with the danger flag and corrective reference actions.
    Section 4.1; the labeling procedure is described qualitatively; no inter-annotator agreement or validation is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HALO: Human Preference Aligned Offline Reward Learning for Robot Navigation." pith.science (2026). https://pith.science/paper/D4P2DV6L

@misc{pith2026250801539,
  author       = {Pith},
  title        = {Pith review of: HALO: Human Preference Aligned Offline Reward Learning for Robot Navigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D4P2DV6L}},
  note         = {Machine review of arXiv:2508.01539}
}
read the original abstract

In this paper, we introduce HALO, a novel Offline Reward Learning algorithm that quantifies human intuition in navigation into a vision-based reward function for robot navigation. HALO learns a reward model from offline data, leveraging expert trajectories collected from mobile robots. During training, actions are uniformly sampled around a reference action and ranked using preference scores derived from a Boltzmann distribution centered on the preferred action, and shaped based on binary user feedback to intuitive navigation queries. The reward model is trained via the Plackett-Luce loss to align with these ranked preferences. To demonstrate the effectiveness of HALO, we deploy its reward model in two downstream applications: (i) an offline learned policy trained directly on the HALO-derived rewards, and (ii) a model-predictive-control (MPC) based planner that incorporates the HALO reward as an additional cost term. This showcases the versatility of HALO across both learning-based and classical navigation frameworks. Our real-world deployments on a Clearpath Husky across diverse scenarios demonstrate that policies trained with HALO generalize effectively to unseen environments and hardware setups not present in the training data. HALO outperforms state-of-the-art vision-based navigation methods, achieving at least a 33.3% improvement in success rate, a 12.9% reduction in normalized trajectory length, and a 26.6% reduction in Frechet distance compared to human expert trajectories.

Figures

Figures reproduced from arXiv: 2508.01539 by the authors.

Figure 1
Figure 1. Architecture of the proposed reward model. Given the current observation I RGB t , a frozen DINO-v2 encoder extracts patch embeddings. Simultaneously, the candidate action at = (vt, ωt) is projected into image space via a homography transform to produce a trajectory mask a m t , which is passed through a trainable CNN to yield a spatial relevance weighting. This vector modulates the image patch embeddings, focusing … view at source ↗
Figure 2
Figure 2. Comparison across three diverse real-world navigation scenarios (outdoors, low light, indoors). Each row shows the trajectory taken by different methods—DWA [44], VANP [45], HER + IQL [42]/BC[43], and HALO-based policies—highlighting differences in social compliance, obstacle avoidance, and goal￾reaching behavior. (BC) [43] on the same data used to train the reward model. We also compared our method with a classical… view at source ↗
Figure 3
Figure 3. Qualitative Analysis for Behavioral Cloning (BC) and Implicit Q-Learning (IQL) policies [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Reward Model Training Loss Curves: Hyperparameter Value Optimizer AdamW Learning rate 2 × 10−4 Weight decay 1 × 10−3 Batch size 256 Number of epochs 200 Scheduler Cosine Annealing with Warm Restarts Warm-up epochs 3 Cosine T0 25 Cosine Tmult 2 Validation split 20% λ1 (…
Figure 5
Figure 5. Figure 5: Training loss curves for IQL training [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: Evaluation curve for IQL training [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: Training loss curves for TD3-BC 18 [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Evaluation loss curves for TD3-BC 7.4 Baselines and Classical Control Details 7.4.1 Hand-Engineered Reward for Offline Comparison Methods Each action (v, ω) is simulated forward in time into a trajectory τv,ω = {(xt, yt, θt)} T t=1. The com￾parison methods benefit from…

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. VEGA: Learning Navigation VLAs from In-the-Wild Egocentric Video with Geometric Trajectory Supervision

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    VEGA reconstructs local geometry from monocular egocentric video to create supervised trajectories that train a flow-matching VLA policy, yielding lower collision rates on a new benchmark and in real-world tests.

  2. VEGA: Learning Navigation VLAs from In-the-Wild Egocentric Video with Geometric Trajectory Supervision

    cs.RO 2026-06 conditional novelty 6.0 of 10

    Unlabeled egocentric video can be converted into obstacle-aware goal-conditioned navigation supervision through monocular geometry, ESDFs, and MPPI-planned trajectories.

  3. From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigation

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    FlowPilot combines anchored flow matching for multimodal action pre-training with human-in-the-loop preference learning to improve long-horizon monocular sidewalk navigation, reporting 42% success in simulation and re...

  4. Interpreting Context-Aware Human Preferences for Multi-Objective Robot Navigation

    cs.RO 2026-03 unverdicted novelty 6.0 of 10

    A framework integrates VLMs and LLMs with a pretrained MORL policy to convert visual context and natural language preferences into real-time navigation parameters for adaptive robot behavior.

Reference graph

Works this paper leans on

50 extracted references · 25 canonical work pages · cited by 3 Pith papers

  1. [1]

    Paluch, J

    S. Paluch, J. Wirtz, and W. H. Kunz.Service Robots and the Future of Services, pages 423–435. Springer Fachmedien Wiesbaden, Wiesbaden, 2020. ISBN 978-3-658-31563-4. doi:10.1007/ 978-3-658-31563-4 21. URL https://doi.org/10.1007/978-3-658-31563-4_21

    work page doi:10.1007/978-3-658-31563-4_21 2020

  2. [2]

    A. H. Raj, Z. Hu, H. Karnan, R. Chandra, A. Payandeh, L. Mao, P. Stone, J. Biswas, and X. Xiao. Rethinking social robot navigation: Leveraging the best of two worlds. In2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 16330–16337. IEEE, 2024

    work page 2024

  3. [3]

    Weerakoon, M

    K. Weerakoon, M. Elnoor, G. Seneviratne, V . Rajagopal, S. H. Arul, J. Liang, M. K. M. Jaffar, and D. Manocha. Behav: Behavioral rule guided autonomy using vlms for robot navigation in outdoor scenes. arXiv preprint arXiv:2409.16484, 2024

    arXiv 2024

  4. [4]

    D. Shah, A. Sridhar, N. Dashora, K. Stachowicz, K. Black, N. Hirose, and S. Levine. Vint: A foundation model for visual navigation. arXiv preprint arXiv:2306.14846, 2023

    arXiv 2023

  5. [5]

    Seneviratne, K

    G. Seneviratne, K. Weerakoon, M. Elnoor, V . Rajgopal, H. Varatharajan, M. K. M. Jaffar, J. Pusey, and D. Manocha. Cross-gait: Cross-attention-based multimodal representation fusion for parametric gait adaptation in complex terrains. arXiv preprint arXiv:2409.17262, 2024

    arXiv 2024

  6. [6]

    Payandeh, D

    A. Payandeh, D. Song, M. Nazeri, J. Liang, P. Mukherjee, A. H. Raj, Y . Kong, D. Manocha, and X. Xiao. Social-llava: Enhancing robot navigation through human-language reasoning in social spaces. arXiv preprint arXiv:2501.09024, 2024

    arXiv 2024

  7. [7]

    D. Shah, A. Bhorkar, H. Leen, I. Kostrikov, N. Rhinehart, and S. Levine. Offline reinforcement learning for visual navigation. arXiv preprint arXiv:2212.08244, 2022

    arXiv 2022

  8. [8]

    T. X. Tung and T. D. Ngo. Socially aware robot navigation using deep reinforcement learning. In 2018 IEEE Canadian Conference on Electrical & Computer Engineering (CCECE) , pages 1–5. IEEE, 2018. 10

    work page 2018

Show all 50 references
  1. [9]

    Sigal, H.-C

    A. Sigal, H.-C. Lin, and A. Moon. Improving generalization in reinforcement learning training regimes for social robot navigation. arXiv preprint arXiv:2308.14947, 2023

    2023 arXiv

  2. [10]

    J. Yao, X. Zhang, Y . Xia, Z. Wang, A. K. Roy-Chowdhury, and J. Li. Sonic: Safe social navigation with adaptive conformal inference and constrained reinforcement learning. arXiv preprint arXiv:2407.17460, 2024

    2024 arXiv

  3. [11]

    Jiang, P

    P. Jiang, P. Osteen, M. Wigness, and S. Saripalli. Rellis-3d dataset: Data, benchmarks and analysis. In 2021 IEEE international conference on robotics and automation (ICRA) , pages 1110–1116. IEEE, 2021

    2021

  4. [12]

    Karnan, A

    H. Karnan, A. Nair, X. Xiao, G. Warnell, S. Pirk, A. Toshev, J. Hart, J. Biswas, and P. Stone. Socially compliant navigation dataset (scand): A large-scale dataset of demonstrations for social navigation. IEEE Robotics and Automation Letters, 7(4):11807–11814, 2022

    2022

  5. [13]

    Weerakoon, A

    K. Weerakoon, A. J. Sathyamoorthy, M. Elnoor, and D. Manocha. Vapor: Legged robot navigation in outdoor vegetation using offline reinforcement learning. arXiv preprint arXiv:2309.07832, 2023

    2023 arXiv

  6. [14]

    Caesar, V

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 11621–11631, 2020

    2020

  7. [15]

    Geyer, Y

    J. Geyer, Y . Kassahun, M. Mahmudi, X. Ricou, R. Durgesh, A. S. Chung, L. Hauswald, V . H. Pham, M. M ¨uhlegg, S. Dorn, et al. A2d2: Audi autonomous driving dataset. arXiv preprint arXiv:2004.06320, 2020

    2004 arXiv

  8. [17]

    Kumar, A

    A. Kumar, A. Zhou, G. Tucker, and S. Levine. Conservative q-learning for offline reinforce- ment learning. Advances in neural information processing systems, 33:1179–1191, 2020

    2020

  9. [18]

    Kapoor, S

    A. Kapoor, S. Swamy, P. Bachiller, and L. J. Manso. Socnavgym: a reinforcement learning gym for social navigation. In 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pages 2010–2017. IEEE, 2023

    2023

  10. [19]

    Liang, U

    J. Liang, U. Patel, A. J. Sathyamoorthy, and D. Manocha. Crowd-steer: Realtime smooth and collision-free robot navigation in densely crowded scenarios trained using high-fidelity simulation. In Proceedings of the Twenty-Ninth International Conference on International Joint Con...

    2021

  11. [20]

    Patel, N

    U. Patel, N. K. S. Kumar, A. J. Sathyamoorthy, and D. Manocha. Dwa-rl: Dynamically feasible deep reinforcement learning policy for robot navigation among mobile obstacles. In2021 IEEE International Conference on Robotics and Automation (ICRA), pages 6057–6063. IEEE, 2021

    2021

  12. [21]

    Mondorf and B

    P. Mondorf and B. Plank. Beyond accuracy: Evaluating the reasoning behavior of large lan- guage models–a survey. arXiv preprint arXiv:2404.01869, 2024

    2024 arXiv

  13. [22]

    Huang and K

    J. Huang and K. C.-C. Chang. Towards reasoning in large language models: A survey. arXiv preprint arXiv:2212.10403, 2022

    2022 arXiv

  14. [23]

    Parmar, N

    M. Parmar, N. Patel, N. Varshney, M. Nakamura, M. Luo, S. Mashetty, A. Mitra, and C. Baral. Logicbench: Towards systematic evaluation of logical reasoning ability of large language mod- els. arXiv preprint arXiv:2404.15522, 2024

    2024 arXiv

  15. [24]

    D. Song, J. Liang, A. Payandeh, A. H. Raj, X. Xiao, and D. Manocha. Vlm-social-nav: Socially aware robot navigation through scoring using vision-language models. IEEE Robotics and Automation Letters, 2024. 11

    2024

  16. [25]

    Huang, O

    C. Huang, O. Mees, A. Zeng, and W. Burgard. Visual language maps for robot navigation.2023 IEEE International Conference on Robotics and Automation (ICRA) , pages 10608–10615,

    2023

  17. [26]

    A. J. Sathyamoorthy, K. Weerakoon, M. B. Elnoor, A. Zore, B. Ichter, F. Xia, J. Tan, W. Yu, and D. Manocha. Convoi: Context-aware navigation using vision language models in outdoor and indoor environments. 2024 IEEE/RSJ International Conference on Intelligent Robots and System...

    2024

  18. [27]

    Elnoor, K

    M. Elnoor, K. Weerakoon, G. Seneviratne, R. Xian, T. Guan, M. K. M. Jaffar, V . Rajagopal, and D. Manocha. Robot navigation using physically grounded vision-language models in outdoor environments. arXiv preprint arXiv:2409.20445, 2024

    2024 arXiv

  19. [28]

    D. F. W. B. S. Thrunyz. The dynamic window approach to collision avoidance. 1997

    1997

  20. [29]

    Kulathunga, A

    G. Kulathunga, A. Yilmaz, Z. Huang, I. Hroob, H. Arunachalam, L. Guevara, A. Klimchik, G. Cielniak, and M. Hanheide. Resilient timed elastic band planner for collision-free naviga- tion in unknown environments. arXiv preprint arXiv:2412.03174, 2024

    2024 arXiv

  21. [30]

    Dosovitskiy, G

    A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V . Koltun. Carla: An open urban driving simulator. In Conference on robot learning, pages 1–16. PMLR, 2017

    2017

  22. [31]

    J. Kim, S. Kang, S. Yang, B. Kim, J. Yura, and D. Kim. Transformable gaussian reward function for socially aware navigation using deep reinforcement learning. Sensors (Basel, Switzerland), 24(14):4540, 2024

    2024

  23. [32]

    P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems , 30, 2017

    2017

  24. [33]

    K. Lee, L. Smith, and P. Abbeel. Pebble: Feedback-efficient interactive reinforcement learning via relabeling experience and unsupervised pre-training. arXiv preprint arXiv:2106.05091 , 2021

    2021 arXiv

  25. [34]

    Y . Wang, Z. Sun, J. Zhang, Z. Xian, E. Biyik, D. Held, and Z. M. Erickson. Rl-vlm-f: Rein- forcement learning from vision language foundation model feedback. ArXiv, abs/2402.03681,

    arXiv

  26. [35]

    R. A. Bradley and M. E. Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39:324, 1952. URL https://api.semanticscholar. org/CorpusID:125209808

    1952

  27. [36]

    Y . Zhai, H. Bai, Z. Lin, J. Pan, S. Tong, Y . Zhou, A. Suhr, S. Xie, Y . LeCun, Y . Ma, and S. Levine. Fine-tuning large vision-language models as decision-making agents via reinforce- ment learning. ArXiv, abs/2405.10292, 2024. URL https://api.semanticscholar.org/ CorpusID:269790773

    2024 arXiv

  28. [37]

    Zhang, K

    J. Zhang, K. Wang, R. Xu, G. Zhou, Y . Hong, X. Fang, Q. Wu, Z. Zhang, and W. He. Navid: Video-based vlm plans the next step for vision-and-language navigation. ArXiv, abs/2402.15852, 2024. URL https://api.semanticscholar.org/CorpusID: 267938569

    2024 arXiv

  29. [38]

    D. Song, J. Liang, X. Xiao, and D. Manocha. Vl-tgs: Trajectory generation and selection using vision language models in mapless outdoor environments, 2025. URL https://arxiv.org/ abs/2408.02454

    2025 arXiv

  30. [39]

    Levine, A

    S. Levine, A. Kumar, G. Tucker, and J. Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643, 2020. 12

    2005 arXiv

  31. [40]

    R. L. Plackett. The analysis of permutations. Journal of The Royal Statistical Society Se- ries C-applied Statistics, 24:193–202, 1975. URL https://api.semanticscholar.org/ CorpusID:116534299

    1975

  32. [41]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V . V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, R. Howes, P.-Y . Huang, H. Xu, V . Sharma, S.-W. Li, W. Galuba, M. Rabbat, M. Assran, N. Ballas, G. Synnaeve, I. Misra, H. Jegou, J. Mairal, P. Labatut,...

    2023

  33. [42]

    Kostrikov, A

    I. Kostrikov, A. Nair, and S. Levine. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06169, 2021

    2021 arXiv

  34. [43]

    Fujimoto, H

    S. Fujimoto, H. Hoof, and D. Meger. Addressing function approximation error in actor-critic methods. In International conference on machine learning, pages 1587–1596. PMLR, 2018

    2018

  35. [44]

    D. Fox, W. Burgard, and S. Thrun. The dynamic window approach to collision avoidance. IEEE Robotics & Automation Magazine, 4(1):23–33, 1997

    1997

  36. [45]

    Nazeri, J

    M. Nazeri, J. Wang, A. Payandeh, and X. Xiao. Vanp: Learning where to see for navigation with self-supervised vision-action pre-training. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2741–2746, 2024. doi:10.1109/IROS58592. 2024.10802451

    2024

  37. [46]

    Alt and M

    H. Alt and M. Godau. Computing the fr ´echet distance between two polygonal curves. Inter- national Journal of Computational Geometry & Applications, 5(01n02):75–91, 1995

    1995

  38. [47]

    Fujimoto and S

    S. Fujimoto and S. S. Gu. A minimalist approach to offline reinforcement learning. Advances in neural information processing systems, 34:20132–20145, 2021

    2021

  39. [48]

    Tarasov, A

    D. Tarasov, A. Nikulin, D. Akimov, V . Kurenkov, and S. Kolesnikov. Corl: Research-oriented deep offline reinforcement learning library. In 3rd Offline RL Workshop: Offline RL as a ”Launchpad”, 2022. URL https://openreview.net/forum?id=SyAS49bBcv. 13 7 Appendix 7.1 Results and...

    2022

  40. [51]

    The overlay of the loss shown is a rolling average over 20 steps, while the actual loss values are faded. The overlay for the policy evaluation loss over a holdout set was averaged over only 5 steps, since this evaluation loss was already an average over evaluating the validat...

  41. [2022]

    URL https://api.semanticscholar.org/CorpusID:252846548

  42. [2024]

    URL https://api.semanticscholar.org/CorpusID:267499679

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.