Pith. sign in

REVIEW 5 major objections 5 minor 40 references

UESF-Bench introduces a large-scale unified seek-and-follow benchmark and shows that a stage-supervised task-driven router substantially outperforms single-head and unsupervised dual-head baselines.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 04:37 UTC pith:EOFRJHZL

load-bearing objection A genuinely useful seek-then-follow benchmark, but the method's recovery claim is undercut by its own sticky stage label, which may mislead readers if not caught in review. the 5 major comments →

arxiv 2607.13621 v1 pith:EOFRJHZL submitted 2026-07-15 cs.AI

UESF-Bench: Benchmarking and Probing for Unified Embodied Seeking and Following

Kun Yu , Jianhua Yang , Yixiang Chen , Changwei Wang , Hongyuan Yu , Yan Huang , Fushuo Huo , Ya Jing
show 2 more authors
Zhumin Chen Keji He
This is my paper
classification cs.AI
keywords Embodied AIEmbodied Visual Seek-and-FollowVision-Language-Action ModelsHuman-Robot Interactionbenchmark constructiontask-driven routingdelayed identity groundingsimulation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces UESF-Bench, a simulation benchmark for a task it argues is closer to real service robotics than prior human-following benchmarks: an agent receives one fused instruction naming a person, must first explore an unseen indoor environment to find that person (who is initially out of view), then switch to following them while keeping identity correct among distractors. The paper claims this is the first large-scale benchmark of this unified kind, with 1.43 million samples built from diverse avatars and scenes in single- and multi-person settings. On it, the paper proposes SeekFollow-VLA, a vision-language-action model, and shows that its task-driven router—dual action heads supervised by a stage label—clearly beats a single shared head and a dual-head router without stage supervision. The central claim is that explicit modeling of the implicit seek-to-follow transition is what makes unified seek-and-follow tractable.

Core claim

On the paper's own terms, the central finding is that an embodied agent can learn to seek a language-described person who is initially out of view and then follow them, provided the action head is explicitly structured around the two behavioral modes. SeekFollow-VLA encodes the observation history and fused instruction with a pre-trained vision-language backbone and predicts waypoints through a seeking head and a following head. The task-driven router between them outputs both routing weights and task logits, with the logits supervised by a stage label that flips from 0 to 1 at first target discovery (within 1.7 m and facing the target). In the reported experiments this variant reaches 0.35

What carries the argument

The Task-Driven Router is the load-bearing mechanism: a three-layer MLP that takes the shared multimodal representation and outputs (i) routing weights that blend a dedicated seeking head and a dedicated following head, and (ii) task logits trained with cross-entropy against a ground-truth stage label. The stage label is 0 until the agent's first successful discovery (target within 1.7 m and facing it) and 1 afterward, which turns the implicit seek-to-follow transition into a supervised switch.

Load-bearing premise

The benchmark's validity rests on the assumption that the LLM-generated appearance descriptions and fused seek-follow instructions uniquely and accurately describe the target avatar; without human validation, ambiguity filtering, or inter-annotator checks, noisy instructions could make scores reflect instruction-following artifacts rather than true embodied seeking and following.

What would settle it

Take a random sample of UESF-Bench test episodes, extract the instruction, and show it to human annotators alongside a lineup of the scene's avatars; if annotators cannot pick the intended target reliably, or if rephrasing the instruction changes model rankings, the benchmark's scores are not measuring embodied seek-and-follow.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Future human-following agents can be evaluated on episodes where the target is initially out of view, closing a gap with real service-robot deployments.
  • Stage-supervised routing is shown to be the decisive design choice: without the task-driven router, the two tested alternatives stay near 0.04–0.05 task success; with it, success reaches 0.35 in single-person and 0.20 in multi-person episodes.
  • Because multi-person episodes lower the following ratio from 0.92 to 0.82 for the best model, the benchmark quantifies the extra cost of identity ambiguity under distractors.
  • Search SPL gains (0.53–0.55 vs 0.26–0.31) indicate the router improves not just success but the efficiency of the search phase.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A human-annotation probe of the generated instructions would test whether the benchmark's difficulty is language grounding or perception; if paraphrased instructions change model rankings, text artifacts are inflating the gap.
  • The same task-driven routing idea could be transplanted to other implicit two-phase tasks (e.g., find-then-manipulate an object) whenever a stage label can be defined by first contact.
  • The absence of reported human validation on avatar descriptions leaves open the possibility that some test episodes are unsolvable by construction; a cheap fix is filtering by description uniqueness.
  • A smaller diagnostic—training only the auxiliary stage head on a frozen backbone—could reveal how much of the gain comes from supervision alone versus the dual-head decomposition.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces UESF-Bench, a large-scale benchmark (1.43M samples, 4,859 avatars, 777 scenes) for the unified task of embodied language-guided seeking and following, in which the target is initially out of view and the agent must first search for the person and then persistently follow them. The authors also propose SeekFollow-VLA, a Qwen3-4B-based VLA framework, and compare three action-head designs: a single shared head, dual heads with a phase-aware router, and dual heads with a task-driven router that is trained with an auxiliary stage label. Experiments in single-person and multi-person settings report that the task-driven router substantially improves Task Success Rate (TSR) and Search SPL over the other two variants, though with a higher Collision Rate. The paper claims this demonstrates reliable behavior switching and recovery, including the ability to re-seek after target loss.

Significance. If the benchmark is validated and the method claims hold, UESF-Bench would fill a real gap: existing human-following benchmarks assume the target is initially visible, while search-only benchmarks stop at discovery. The proposed task formulation, dataset scale, and scene diversity are valuable contributions, and the internal comparison across action-head designs provides an initial baseline. The paper also presents a clean experimental setup with a clear metric set. However, the central claim about recovery and behavior switching is not supported by the training objective as written, and the benchmark's validity relies on unvalidated LLM-generated instructions. These issues, plus the lack of statistical rigor, currently limit the paper's reliability as a benchmark contribution.

major comments (5)
  1. [Sec. 4.5, Eq. (17)] The stage label z_T is defined as 0 until the target is first discovered and then 1 'for all subsequent time steps.' The auxiliary cross-entropy loss therefore actively penalizes the router whenever it outputs any weight to the seeking head after first discovery. This makes the router's behavior monotonic: the model is trained never to re-enter seeking, even if the target is lost. Consequently, the paper's stated challenge in Sec. 1 and Fig. 1 — that the agent must infer 'whether to recover following or re-seek' — is not addressed. The sharp one-way collapse in Fig. 4 is fully compatible with this label schedule and cannot be cited as evidence of learned recovery/transition. The authors should revise z_T to return to 0 when the target is lost (or otherwise model a non-monotonic phase), or clearly temper the 'recovery' and 'reliable behavior switching' claims. As written, the main methodo
  2. [Sec. 3.2] The benchmark's validity depends on LLM-generated appearance descriptions (GPT-5.2) and seek-follow instructions (Gemini-3-Pro) being accurate and sufficiently specific to uniquely identify the target under distractors. No human validation, ambiguity filtering, or inter-annotator statistics are reported. If a description is vague or incorrect, an episode measures the agent's ability to follow a noisy/under-specified instruction rather than embodied seek-and-follow. The authors should provide at least a human evaluation on a random sample of episodes (e.g., can a human identify the target from the instruction?), and report the distribution of description specificity and failure cases. This is load-bearing for the benchmark's utility.
  3. [Sec. 5.2–5.3, Tables 1–2] All results are reported as point estimates with no confidence intervals, number of seeds, or significance tests. The paper proposes a benchmark for systematic evaluation; without variance information, the claim that DualHead-TD 'clearly outperforms' baselines is not statistically grounded. This is especially important given the low absolute TSR values (0.04–0.05 for single/dual-head baselines) and the large gap to 0.35/0.20. Please report mean ± std over at least 3 seeds (or equivalent) and, if possible, a paired significance test across episodes.
  4. [Sec. 5.3, Tables 1–2] The best method (DualHead-TD) raises the Collision Rate from 0.07/0.06 (baselines) to 0.32/0.33. The paper dismisses this by saying it should be interpreted with task success, but for a human-robot interaction task, a ~5× higher collision rate is a serious safety concern. The authors should analyze the causes (e.g., is the following head too aggressive, or does the router switch too late?) and either propose a mitigation or explicitly discuss the safety/performance trade-off. This is not a minor issue given the task's application domain.
  5. [Sec. 5.3 and Sec. 2.3] The experimental comparison only includes the paper's own three head designs. The paper motivates UESF-Bench as a reusable general testbed, but no existing VLA-based tracker (e.g., TrackVLA, Uni-NaVid, TrackVLA++) is adapted and evaluated on the new benchmark. Without at least one external baseline, the community cannot calibrate the proposed method's performance or see how the benchmark discriminates among published approaches. Please include at least one existing method, even if adapted to the unified task, to demonstrate the benchmark's utility.
minor comments (5)
  1. [Sec. 3.3] The text says '777 of the 890 scenes available in HM3D and MP3D'; clarify whether 890 is the union of both datasets and why the remaining scenes are excluded. The number 890 appears only here.
  2. [Sec. 4.3, Eq. (4)] The temporal window notation is slightly ambiguous: V_T uses coarse tokens for T-k,...,T-1 and fine tokens for T. Clarify whether the current frame is always the latest and whether k=31 means 31 history frames or 31+current.
  3. [Sec. 4.5, Eq. (18)] The relative weighting between L_wp and L_stage is set by β_nav=10, but there is no discussion of the balance or sensitivity to this hyperparameter. A small sensitivity analysis would strengthen the claim that the auxiliary loss is the key factor.
  4. [References [16]] Reference [16] is cited for GPT-5.2, Gemini-3-Pro, and Qwen3-VL, but the title is 'A Safety Report on...' — this is not the primary source for these models. Please cite the actual model cards or technical reports.
  5. [Sec. 1] Minor typo: 'question:how' lacks a space after the colon. Also, the phrase 'reliable behavior switching and recovery' appears in the abstract and conclusion but is not defined as a separate metric; consider defining an explicit recovery metric (e.g., fraction of lost-target episodes in which the agent regains following).

Circularity Check

1 steps flagged

The task-driven router's claimed latent phase inference is directly supervised by a monotonic privileged stage label, so the reported one-way seek-to-follow transition is forced by construction; the benchmark itself remains non-circular.

specific steps
  1. fitted input called prediction [Sec. 4.5 (Training Objective, Eqs. 17-18) and Sec. 5.5 (Routing Dynamics Analysis, Fig. 4)]
    "the task logits are supervised by a task-related label ... z_T remains 0 until the agent first successfully finds the target person, where success is defined as the target being within 1.7 m and the agent facing the target. After this first successful discovery, z_T remains 1 for all subsequent time steps. ... Once the target is found, the routing undergoes a sharp transition: the search weight quickly collapses to nearly zero, while the follow weight rises to nearly one and remains stable thereafter."

    Eq. (17) trains the router logits g_T directly with cross-entropy against the privileged stage label z_T, and Eq. (18) adds this L_stage to the main loss. The sharp search-to-follow collapse in Fig. 4 is therefore the label schedule itself: z_T is 0 until the first within-1.7m/facing discovery and permanently 1 afterward, so the router is optimized to make exactly that one-way transition. Presenting this as evidence of 'latent phase inference and transition modeling' or 'reliable behavior switching and recovery' renames the imposed supervision as an emergent capability. The label never returns to 0, so the model is trained against re-entering seeking; the recovery claim is not derivable from the trained router at all.

full rationale

The benchmark construction, task formulation, and aggregate TSR/SPL comparisons are empirical and do not reduce to their inputs: success is measured against held-out expert trajectories and the environment is built from HM3D/MP3D with LLM-generated instructions. The single significant circularity concern is the task-driven router: its 'latent phase inference' is not latent, because the stage label z_T is provided as privileged supervision and the reported routing dynamics in Fig. 4 are a consequence of that label's monotonic definition. Because this is the paper's central methodological evidence for transition/recovery, the circularity score is 6 rather than 0-2; the non-router benchmark contributions keep it from a higher score.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

This is an empirical benchmark-and-method paper, so the ledger records the domain assumptions the central claims rest on rather than mathematical axioms. No invented entities are introduced. The free parameters are training hyperparameters chosen by hand; the most consequential is β_nav=10, which controls how much weight the router's stage-supervision loss has relative to waypoint regression. The deeper assumptions are that the Habitat simulator, the LLM-generated instructions, and the ground-truth stage labels each faithfully represent the real seek-and-follow problem.

free parameters (3)
  • β_nav = 10
    Weight on waypoint prediction loss in Eq. (18); set by hand without ablation.
  • History length k = 31
    Sliding-window length of historical frames; fixed hyperparameter, no sensitivity analysis.
  • Waypoint horizon M = 10
    Number of predicted waypoints; fixed hyperparameter.
axioms (5)
  • domain assumption Habitat 3.0 simulation faithfully captures indoor navigation, collision, and egocentric perception needed for seek-and-follow.
    All results are generated in simulation; no real-world validation is reported.
  • domain assumption LLM-generated avatar descriptions (GPT-5.2) and instructions (Gemini-3-Pro) are accurate and uniquely identify the target among distractors.
    Sec. 3.2 uses these generations directly; no human validation or ambiguity analysis is reported.
  • domain assumption Ground-truth stage labels z_T defined by target within 1.7m and facing the target (Sec. 4.5) provide correct supervision for routing.
    The stage label is derived from simulator ground truth; the paper assumes it is learnable from visual history at inference.
  • domain assumption Pre-trained Qwen3-4B, DINO-V3, and SigLIP provide useful representations for egocentric RGB waypoint prediction in this novel task.
    The method relies on transfer from these models without analysis of their suitability.
  • domain assumption Expert trajectories generated for each sample are a reliable supervision signal for both seeking and following phases.
    Waypoint MSE (Eq. 15) relies on these trajectories; no analysis of expert optimality is provided.

pith-pipeline@v1.3.0-alltime-deepseek · 13297 in / 15783 out tokens · 139870 ms · 2026-08-02T04:37:52.582180+00:00 · methodology

0 comments
read the original abstract

Language-guided human following is an important capability for embodied agents, but existing benchmarks typically assume that the target person is visible at the start of an episode. This setting simplifies the problem and overlooks a more realistic requirement: an agent often needs to first find a language-described target and then persistently follow that target in a dynamic environment. While recent work has started to study human search, existing settings are typically evaluated in task-specific scenarios and often rely on stronger prior knowledge of the environment. Moreover, they usually treat searching and following as separate tasks and still lack a unified benchmark for systematic evaluation. To address these limitations, we introduce the Unified Embodied Seeking and Following Benchmark (UESF-Bench), a large-scale and diverse benchmark for embodied human seeking and following. The benchmark requires agents to handle semantic-guided exploration, reliable behavior switching and recovery, and delayed identity grounding. To this end, we propose SeekFollow-VLA, a vision-language-action framework with a task-driven routing mechanism for latent phase inference and transition modeling between seeking and following. Experimental results show that SeekFollow-VLA achieves clear improvements over both single-head and dual-head baselines across single-person and multi-person environments, establishing a baseline for unified embodied seek-and-follow.

Figures

Figures reproduced from arXiv: 2607.13621 by Changwei Wang, Fushuo Huo, Hongyuan Yu, Jianhua Yang, Keji He, Kun Yu, Ya Jing, Yan Huang, Yixiang Chen, Zhumin Chen.

Figure 1
Figure 1. Figure 1: Overview of the embodied visual seek-and-follow task, its core challenges. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview and statistics of UESF-Bench. UESF-Bench contains 4800+ diverse humanoid avatars and 1.43M+ embodied [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Architecture of SeekFollow-VLA. The model encodes historical and current observations into visual tokens, projects [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: visualizes the router weights of the task-driven variant in a representative episode. A clear two-stage routing pattern can be observed. During the early part of the episode, the router assigns nearly all weight to the search head, indicating that the model 0.00 0.25 0.50 0.75 1.00 Search Raw Smoothed 0 20 40 60 80 100 120 140 160 Step 0.00 0.25 0.50 0.75 1.00 Follow Raw Smoothed [PITH_FULL_IMAGE:figures/… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 18 linked inside Pith

  1. [1]

    Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünder- hauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. 2018. Vision-and- language navigation: Interpreting visually-grounded navigation instructions in real environments. InProceedings of the IEEE conference on computer vision and pattern recognition. 3674–3683

    2018

  2. [2]

    2024.𝜋0: A Vision-Language-Action Flow Model for General Robot Control.arXiv preprint arXiv:2410.24164(2024)

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. 2024.𝜋0: A Vision-Language-Action Flow Model for General Robot Control.arXiv preprint arXiv:2410.24164(2024). UESF-Bench: Benchmarking and Probing for Unified Embodied Seeking and Following

    Pith/arXiv arXiv 2024

  3. [3]

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. 2017. Matterport3d: Learning from rgb-d data in indoor environments.arXiv preprint arXiv:1709.06158 (2017)

    Pith/arXiv arXiv 2017

  4. [4]

    Devendra Singh Chaplot, Dhiraj Prakashchand Gandhi, Abhinav Gupta, and Russ R Salakhutdinov. 2020. Object goal navigation using goal-oriented semantic exploration.Advances in Neural Information Processing Systems33 (2020), 4247– 4258

    2020

  5. [5]

    Andrea Fortuna, Marta Lorenzini, Mattia Leonori, Juan M Gandarias, Pietro Bal- atti, Younggeol Cho, Elena De Momi, and Arash Ajoudani. 2024. A personalizable controller for the walking assistive omni-directional exo-robot (WANDER). In 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 3212–3218

    2024

  6. [6]

    Angus Fung, Beno Benhabib, and Goldie Nejat. 2025. Ldtrack: dynamic people tracking by service robots using diffusion models.International Journal of Computer Vision133, 6 (2025), 3392–3412

    2025

  7. [7]

    Angus Fung, Aaron Hao Tan, Haitong Wang, Bensiyon Benhabib, and Goldie Nejat. 2025. MLLM-search: A zero-shot approach to finding people using multi- modal large language models.Robotics14, 8 (2025), 102

    2025

  8. [8]

    Sourav Garg, Dustin Craggs, Vineeth Bhat, Lachlan Mares, Stefan Podgorski, Madhava Krishna, Feras Dayoub, and Ian Reid. 2025. Objectreact: Learning object-relative control for visual navigation.arXiv preprint arXiv:2509.09594 (2025)

    Pith/arXiv arXiv 2025

  9. [9]

    Meenakshi Gupta, Swagat Kumar, Laxmidhar Behera, and Venkatesh K Subra- manian. 2016. A novel vision-based tracking algorithm for a human-following mobile robot.IEEE Transactions on Systems, Man, and Cybernetics: Systems47, 7 (2016), 1415–1427

    2016

  10. [10]

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. 2024. Openvla: An open-source vision-language-action model, 2024.URL https://arxiv. org/abs/2406.092461, 2 (2024), 4

    Pith/arXiv arXiv 2024

  11. [11]

    Jin Joo Lee, Amin Atrash, Dylan F Glas, and Hanxiao Fu. 2023. Developing autonomous behaviors for a consumer robot to be near people in the home. In 2023 32nd IEEE International Conference on Robot and Human Interactive Commu- nication (RO-MAN). IEEE, 197–204

    2023

  12. [12]

    Lei Li, Ming Jeat Foo, Jiaye Chen, Kuan Yuee Tan, Jiaying Cai, Rohini Swami- nathan, Karen Sui Geok Chua, Seng Kwee Wee, Christopher Wee Keong Kuah, Huiting Zhuo, et al . 2023. Mobile Robotic Balance Assistant (MRBA): a gait assistive and fall intervention robot for daily living.Journal of NeuroEngineering and Rehabilitation20, 1 (2023), 29

    2023

  13. [13]

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Cheng- gang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437(2024)

    Pith/arXiv arXiv 2024

  14. [14]

    Jiahang Liu, Yunpeng Qi, Jiazhao Zhang, Minghan Li, Shaoan Wang, Kui Wu, Hanjing Ye, Hong Zhang, Zhibo Chen, Fangwei Zhong, et al. 2025. Trackvla++: Unleashing reasoning and memory capabilities in vla models for embodied visual tracking.arXiv preprint arXiv:2510.07134(2025)

    arXiv 2025

  15. [15]

    Yufei Liu, Junwei Zhu, Junshu Tang, Shijie Zhang, Jiangning Zhang, Weijian Cao, Chengjie Wang, Yunsheng Wu, and Dongjin Huang. 2024. Texdreamer: Towards zero-shot high-fidelity 3d human texture generation. InEuropean conference on computer vision. Springer, 184–202

    2024

  16. [16]

    Xingjun Ma, Yixu Wang, Hengyuan Xu, Yutao Wu, Yifan Ding, Yunhan Zhao, Zilong Wang, Jiabin Hua, Ming Wen, Jianan Liu, et al. 2026. A Safety Report on GPT-5.2, Gemini 3 Pro, Qwen3-VL, Doubao 1.8, Grok 4.1 Fast, Nano Banana Pro, and Seedream 4.5.arXiv preprint arXiv:2601.10527(2026)

    arXiv 2026

  17. [17]

    Nidhi Mishra, Teena Bharti, Aviral Kumar Tiwari, and Gregor Pfajfar. 2024. Public and scholarly interest in social robots: An investigation through Google Trends, bibliometric analysis, and systematic literature review.Technological Forecasting and Social Change206 (2024), 123578

    2024

  18. [18]

    Anastasia K Ostrowski, Cynthia Breazeal, and Hae Won Park. 2022. Mixed- method long-term robot usage: Older adults’ lived experience of social robots. In2022 17th ACM/IEEE international conference on human-robot interaction (HRI). IEEE, 33–42

    2022

  19. [19]

    Akhil Padmanabha, Jessie Yuan, Janavi Gupta, Zulekha Karachiwalla, Carmel Majidi, Henny Admoni, and Zackory Erickson. 2024. Voicepilot: Harnessing LLMs as speech interfaces for physically assistive robots. InProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology. 1–18

    2024

  20. [20]

    Jeongeun Park, Jefferson Silveria, Matthew Pan, and Sungjoon Choi. 2023. SOCRATES: Text-based Human Search and Approach using a Robot Dog.arXiv preprint arXiv:2302.05324(2023)

    Pith/arXiv arXiv 2023

  21. [21]

    Xavier Puig, Eric Undersander, Andrew Szot, Mikael Dallaire Cote, Tsung-Yen Yang, Ruslan Partsey, Ruta Desai, Alexander William Clegg, Michal Hlavac, So Yeon Min, et al. 2023. Habitat 3.0: A co-habitat for humans, avatars and robots. arXiv preprint arXiv:2310.13724(2023)

    Pith/arXiv arXiv 2023

  22. [22]

    Santhosh K Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Undersander, Wojciech Galuba, An- drew Westbury, Angel X Chang, et al. 2021. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai.arXiv preprint arXiv:2109.08238 (2021)

    Pith/arXiv arXiv 2021

  23. [23]

    Ranjan Sapkota, Yang Cao, Konstantinos I Roumeliotis, and Manoj Karkee. 2025. Vision-language-action models: Concepts, progress, applications and challenges. arXiv preprint arXiv:2505.04769(2025)

    arXiv 2025

  24. [24]

    Dhruv Shah, Błażej Osiński, Sergey Levine, et al. 2023. Lm-nav: Robotic naviga- tion with large pre-trained models of language, vision, and action. InConference on robot learning. pmlr, 492–504

    2023

  25. [25]

    Oriane Siméoni, Huy V. Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ra- mamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timothée Darcet, Théo Moutakanni, Leonel Sentana, Claire Roberts, Andrea Vedaldi, Jamie Tolan, John Brandt, Camille Couprie, Julie...

    Pith/arXiv arXiv 2025

  26. [26]

    Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al

  27. [27]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. 2023. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805(2023)

    Pith/arXiv arXiv 2023

  28. [28]

    Shaoan Wang, Jiazhao Zhang, Minghan Li, Jiahang Liu, Anqi Li, Kui Wu, Fangwei Zhong, Junzhi Yu, Zhizheng Zhang, and He Wang. 2025. Trackvla: Embodied visual tracking in the wild.arXiv preprint arXiv:2505.23189(2025)

    Pith/arXiv arXiv 2025

  29. [29]

    Karmesh Yadav, Arjun Majumdar, Ram Ramrakhya, Naoki Yokoyama, Alexei Baevski, Zsolt Kira, Oleksandr Maksymets, and Dhruv Batra. 2023. Ovrl-v2: A simple state-of-art baseline for imagenav and objectnav.arXiv preprint arXiv:2303.07798(2023)

    Pith/arXiv arXiv 2023

  30. [30]

    Hanjing Ye, Kuanqi Cai, Yu Zhan, Bingyi Xia, Arash Ajoudani, and Hong Zhang

  31. [31]

    Zhaoshu Yu, Bo Wang, Pengpeng Zeng, Haonan Zhang, Ji Zhang, Lianli Gao, Jingkuan Song, Nicu Sebe, and Heng Tao Shen. 2025. A survey on efficient vision-language-action models.arXiv preprint arXiv:2510.24795(2025)

    arXiv 2025

  32. [32]

    Rpf-search: Field-based search for robot person following in unknown dynamic environments.IEEE/ASME Transactions on Mechatronics(2025)

    2025

  33. [33]

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sigmoid Loss for Language Image Pre-Training. arXiv:2303.15343 [cs.CV]

    Pith/arXiv arXiv 2023

  34. [34]

    Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, et al. 2025. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models.arXiv preprint arXiv:2508.06471 (2025)

    Pith/arXiv arXiv 2025

  35. [35]

    Jiazhao Zhang, Kunyu Wang, Rongtao Xu, Gengze Zhou, Yicong Hong, Xiaomeng Fang, Qi Wu, Zhizheng Zhang, and He Wang. 2024. Navid: Video-based vlm plans the next step for vision-and-language navigation.arXiv preprint arXiv:2402.15852 (2024)

    Pith/arXiv arXiv 2024

  36. [36]

    Jiazhao Zhang, Kunyu Wang, Shaoan Wang, Minghan Li, Haoran Liu, Songlin Wei, Zhongyuan Wang, Zhizheng Zhang, and He Wang. 2024. Uni-navid: A video-based vision-language-action model for unifying embodied navigation tasks.arXiv preprint arXiv:2412.06224(2024)

    Pith/arXiv arXiv 2024

  37. [37]

    Yifan Zhong, Fengshuo Bai, Shaofei Cai, Xuchuan Huang, Zhang Chen, Xiaowei Zhang, Yuanfei Wang, Shaoyang Guo, Tianrui Guan, Ka Nam Lui, et al. 2025. A survey on vision-language-action models: An action tokenization perspective. arXiv preprint arXiv:2507.01925(2025)

    Pith/arXiv arXiv 2025

  38. [38]

    Fangwei Zhong, Kui Wu, Hai Ci, Churan Wang, and Hao Chen. 2024. Empower- ing embodied visual tracking with visual foundation models and offline rl. In European Conference on Computer Vision. Springer, 139–155

    2024

  39. [40]

    Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. 2023. Rt-2: Vision-language- action models transfer web knowledge to robotic control. InConference on Robot Learning. PMLR, 2165–2183

    2023

  40. [2025]

    Openai gpt-5 system card.arXiv preprint arXiv:2601.03267(2025)

    Pith/arXiv arXiv 2025