Pith. sign in

REVIEW 4 major objections 4 minor 32 references

VLM Can Be a Good Assistant: Enhancing Embodied Visual Tracking with Self-Improving Vision-Language Models

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A vision-language model used only as a recovery module, triggered by a segmentation-based failure detector, raises embodied tracking success rates by up to 72 percent over an RL tracker and up to 220 percent over a PID controller.

desk verdict A genuinely new VLM-as-recovery-assistant system for embodied tracking, but the central attribution claim needs a non-VLM control and cleaner evaluation. read the letter →

arxiv 2505.20718 v2 pith:MFJDG6UJ submitted 2025-05-27 cs.CV cs.AI

Kui Wu , Shuhang Xu , Hao Chen , Churan Wang , Zhoujun Li , Yizhou Wang , Fangwei Zhong This is my paper
classification cs.CVcs.AI
keywords embodiedvisualtrackingvision-languagemodelsfailurerecoverymemory-augmentedself-reflectionactiveGPT-4osegmentation-baseddetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a vision-language model can act as a recovery assistant for embodied visual tracking: instead of trying to make the VLM do the whole tracking task, the system runs a fast conventional tracker while the target is visible and hands control to VLM reasoning only when the target is lost. The authors claim this division of labor turns what used to be terminal tracking failures into recoverable events, and that a memory-augmented self-reflection loop lets the VLM improve over time. They report that adding this recovery layer to a state-of-the-art RL tracker raises success rates by 21–72 percent across four simulated environments, and raises a PID controller's success rate by 29–220 percent. If correct, the result matters because it offers a practical path to making existing robot trackers robust to occlusion without retraining them.

What carries the argument

The load-bearing object is the memory-augmented self-reflection loop. Each memory entry stores a failure context tuple (occlusion flag, occluding object, last known target position), the movement plan, the executed action sequence, and the reflection insight produced after the attempt. On a new failure, the system retrieves the top-3 similar entries by TF-IDF cosine similarity between the text of the failure context and movement plan, and feeds them to the VLM as exemplars before planning the recovery action sequence. The reflection insight, generated by prompting the VLM to compare expected versus actual behavior when recovery fails, is what lets the system improve across episodes. This mechanism is what the paper claims compensates for the VLM's weak 3D spatial reasoning.

What would settle it

Run the same framework with the same failure detector but replace the VLM's recovery planning with a scripted heuristic (for example, a fixed sequence of rotations and forward moves). If the heuristic recovers the target about as often as the VLM module in the same four environments, the paper's central claim that VLM reasoning drives recovery is not supported.

Watch

Extended reading notes

Core claim

The central claim is that the bottleneck in embodied visual tracking is not the nominal tracking policy but the absence of a way to recover after the target disappears, and that VLM reasoning can fill that gap. The paper demonstrates this by pairing an off-the-shelf active tracker with a GPT-4o-based recovery module that is activated by a segmentation-based failure detector. When triggered, the VLM performs chain-of-thought failure analysis on the last frames, emits a structured movement plan of direction–landmark–trigger tuples, retrieves similar past episodes from memory, and refines a five-step action sequence. A reflection phase records what worked and what did not, so later recoveries improve. The authors state this is the first VLM-based proactive failure recovery for embodied visual tracking.

Load-bearing premise

The whole recovery layer is triggered by a single rule: the target's segmentation mask must be invisible for more than three consecutive steps, and that threshold plus the segmentation model's reliability are assumed to hold in the deployed environment.

Editorial extensions

If this is right

  • A tracker that loses the target no longer has to fail: the episode continues if the VLM can bring the target back into view.
  • The same recovery layer can be stacked on very different base policies, from an RL policy to a classical PID controller, without retraining the base tracker.
  • The memory-and-reflection loop reduces repeated mistakes as the system accumulates experience in a deployment environment.
  • The performance gap between cheap classical controllers and expensive learned policies narrows substantially once a VLM recovery module is added.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test not run in the paper: compare the recovery module against a scripted search heuristic using the same failure detector. If a fixed spiral or random search recovers targets equally well, the VLM's reasoning is not the active ingredient.
  • The failure detector's 3-step invisibility threshold and its dependence on a segmentation model suggest the gains may shrink if transferred to a real robot with noisier masks; the recovery layer would need a perception-stack-agnostic trigger.
  • The memory format is text-only TF-IDF over VLM outputs; pairing the same mechanism with visual or multimodal embeddings could make retrieval robust to phrasing differences and is a natural extension.
  • The architecture is a template for other embodied tasks with intermittent observability, such as visual navigation or person following, where a fast policy handles nominal conditions and a slow reasoner handles loss-of-observation events.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a modular framework that augments an off-the-shelf embodied visual tracking policy with a vision-language-model (VLM) recovery module. While the base tracker runs continuously, a segmentation-based detector triggers a recovery phase when the target mask is absent for more than three consecutive steps. The VLM (GPT-4o) then analyzes the last frames, produces a structured movement plan, retrieves similar past failure cases from a growing memory, refines a fixed-length action sequence, and, if recovery fails, generates reflection insights that are stored for future use. Experiments in four UnrealZoo environments compare the framework against PID, OpenVLA, GPT-4o, and a SOTA RL tracker, reporting substantial success-rate improvements (up to 72% over SOTA RL and 220% over PID in the Chemical Plant environment), along with ablations of the reflection and memory-retrieval components.

Significance. If the reported effects are causal, the framework is practically valuable: it turns tracking failures from terminal events into recoverable phases, it is modular with respect to the base tracker, and it offers a concrete way to inject VLM spatial reasoning into a low-level control loop without incurring VLM latency during normal operation. The memory-augmented reflection mechanism is a sensible approach to mitigating VLM weaknesses in 3D reasoning, and the open-sourced code and four-environment evaluation are assets. However, the current experimental design does not isolate the VLM's contribution from the mere presence of an active search behavior, and the 'self-improving' property is not directly demonstrated; these gaps are load-bearing for the paper's central claims.

major comments (4)
  1. [Section IV-C and Section V-A] The action space is described inconsistently. Section IV-C states that the action set A contains six executable actions, but only five are listed: Move Forward, Move Backward, Turn Left, Turn Right, and Jump Over. Section V-A then says GPT-4o employs a six-dimensional discrete action space. The recovery sequence R is also defined as a five-element tuple (a1,...,a5). This ambiguity directly affects reproducibility and the fairness of the comparison with the GPT-4o baseline, since the baseline must use the same action space and sequence length. Please correct the list, define the exact six actions, and state the sequence length unambiguously.
  2. [Section IV-C and Section V] The 'self-improving' claim is not directly demonstrated. The memory is populated during the evaluation episodes (Section IV-C describes storing new memories, and Table II counts recovery attempts across the 50 evaluation episodes), but the paper provides no analysis of whether recovery success improves over episode index, and no separate warm-up or validation split for memory. Because the baselines do not have access to this evolving memory, the aggregate SR comparison may conflate a system that improves over time with a static one. Please report recovery success as a function of episode index (e.g., first vs. second half), or clarify whether memory is pre-populated from a separate dataset and frozen during evaluation.
  3. [Table I] The magnitude of the claimed improvements is uncertain because all metrics are reported on 50 episodes with no variance, confidence intervals, or significance tests. For example, in Chemical Plant the PID success rate is 0.10 and Ours (PID) is 0.32; with 50 episodes this is a difference of about 11 episodes, and the paper does not report whether the same 50 episodes are used across methods. Please provide multiple seeds or bootstrap confidence intervals, and state whether the evaluation episodes are identical across all compared systems. The absence of error bars is especially important given that the abstract quotes the largest gains (72% and 220%) from a single environment.
  4. [Section V-C, Table I] The text claims that 'both ablated variants still outperformed all baseline methods,' but this is contradicted by Table I: in Old Factory, Ours w/o Reflection achieves SR 0.68, which is lower than the SOTA RL baseline's 0.76. This discrepancy undermines the conclusion that each module contributes positively in all environments. Please correct the claim or discuss why removing reflection can hurt performance relative to the base tracker in some environments, and adjust the ablation interpretation accordingly.
minor comments (4)
  1. [Section III] The reward function r = 1 - |rho-rho*|/rho_max - |theta-theta*|/theta_max is defined in terms of the target's distance and angle relative to the tracker. During the recovery phase the target is, by definition, not visible, so it is unclear how r_t is computed at those steps. Please clarify how the reward is obtained when the target is out of view.
  2. [Abstract and Table I] The abstract reports only the largest gains (72% and 220%), which are the Chemical Plant values; the gains in the other three environments are substantially smaller (21-57% for SOTA RL and 29-85% for PID). Please report the range of improvements or state the environment explicitly to avoid overstating the general result.
  3. [Section II-B and Section IV-C] There are minor grammatical errors: 'we building a self-improving reasoning framework' in Section II-B should be 'we build', and 'We empirically retrieves' in Section IV-C should be 'We empirically retrieve'.
  4. [Section IV-A] The failure-detection threshold ('more than 3 consecutive steps') and the episode-termination threshold (50 steps) are fixed constants with no sensitivity analysis. Since the entire recovery phase is gated by the segmentation-based detector, a brief missed segmentation or a different environment with noisier masks could change when recovery is triggered. A short discussion of the threshold choice or a sensitivity study would strengthen the transferability argument.

Circularity Check

0 steps flagged · score 1.0 of 10

No derivation-level circularity; empirical gains are measured against independent baselines, with self-citation only in the SOTA RL reference.

full rationale

This is an empirical systems evaluation rather than a mathematical derivation, so the circularity burden is low. The central claim—that adding a VLM-based recovery module with memory and reflection improves embodied visual tracking success rates—is tested by comparing the framework against PID, OpenVLA, GPT-4o, and an RL baseline in simulated environments. The SOTA RL baseline [29] is prior work by overlapping authors, but it is a published, code-reproduced policy that the paper uses as an external reference point, and the reported improvements are measured outcomes rather than quantities defined into existence by the method itself. The ablations 'Ours w/o Reflection' and 'Ours w/o Memory Retrieval' still use GPT-4o for recovery, so the experiments do not isolate VLM reasoning from the presence of any active search behavior; however, that is an attribution gap, not a circularity, because the compared systems are not equated by construction. The empirically tuned three-step failure-detection threshold is a parameter of the system, not a fitted value that is later presented as a prediction. No equation or definition in the paper reduces the claimed result to its own inputs. The only mild concern is self-citation in selecting the RL baseline, which is not load-bearing for the derivation and does not make the comparison tautological. Score 0-2 is therefore appropriate; I assign 1 to acknowledge the self-citation without treating it as circular.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central result rests on the reliability of the base tracker, the VLM's spatial reasoning, the segmentation-based failure detector, and the text-similarity memory retrieval. The listed free parameters are hand-chosen constants that are not swept or reported with sensitivity. No new physical entities are introduced.

free parameters (6)
  • failure_detection_threshold = 3 consecutive steps
    Chosen empirically in Section IV-A via extensive testing; determines when the recovery phase is triggered, affecting all reported metrics.
  • observation_sampling_interval = 5 steps
    Frames o_{t-5k}, k=0..3, are sampled at five-step intervals in Section IV-B; a hand-chosen context window for the VLM.
  • recovery_action_sequence_length = 5 actions
    The fixed-length recovery sequence R=(a1,...,a5) in Section IV-C is chosen as a balance between effectiveness and computational cost.
  • retrieved_memory_cases = top-3
    Top-3 most similar cases are retrieved as exemplars in Section IV-C, with no sensitivity analysis reported.
  • memory_similarity_weights = equal weights (1 and 1)
    score_i = sim_text(Psi_cur, Psi_i) + sim_text(Gamma_cur, Gamma_i) in Section IV-C, with no learned or tuned weighting.
  • episode_termination_threshold = 50 steps
    Episodes terminate if the target is lost for more than 50 steps, following prior protocols [26],[29]; this constant shapes the success rate metric.
assumptions (6)
  • domain assumption The offline RL tracking policy from [29] generalizes to the four UnrealZoo test environments after training on 50k-step trajectories with domain randomization.
    Used as the base tracker in Section V-A; if it does not transfer, the recovery layer is evaluated against a weak base.
  • domain assumption GPT-4o's chain-of-thought analysis of three RGB frames can yield reliable recovery decisions in 3D space.
    Core reasoning module in Section IV-B; the paper relies on VLM spatial reasoning despite acknowledging its known limitations.
  • domain assumption The segmentation model [3] reliably identifies the target whenever it is in view.
    Failure detection and tracking both depend on segmentation masks in Section IV-A; no segmentation failure-rate analysis is provided.
  • ad hoc to paper TF-IDF cosine similarity on short text tuples is a valid measure of recovery-case similarity.
    Memory retrieval in Section IV-C uses this text similarity without validation against behavioral similarity.
  • domain assumption The four UnrealZoo environments are representative enough to support claims about real-world robotic applications.
    All experiments are in simulation (Section V-A), yet the conclusion claims real-world applicability.
  • domain assumption Memory behavior across evaluation episodes is consistent and does not introduce test-set leakage.
    The paper does not state whether memory is reset per episode; Section IV-C stores all episodes, so later episodes may benefit from earlier test episodes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VLM Can Be a Good Assistant: Enhancing Embodied Visual Tracking with Self-Improving Vision-Language Models." pith.science (2026). https://pith.science/paper/MFJDG6UJ

@misc{pith2026250520718,
  author       = {Pith},
  title        = {Pith review of: VLM Can Be a Good Assistant: Enhancing Embodied Visual Tracking with Self-Improving Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MFJDG6UJ}},
  note         = {Machine review of arXiv:2505.20718}
}
abstract

We introduce a novel self-improving framework that enhances Embodied Visual Tracking (EVT) with Vision-Language Models (VLMs) to address the limitations of current active visual tracking systems in recovering from tracking failure. Our approach combines the off-the-shelf active tracking methods with VLMs' reasoning capabilities, deploying a fast visual policy for normal tracking and activating VLM reasoning only upon failure detection. The framework features a memory-augmented self-reflection mechanism that enables the VLM to progressively improve by learning from past experiences, effectively addressing VLMs' limitations in 3D spatial reasoning. Experimental results demonstrate significant performance improvements, with our framework boosting success rates by $72\%$ with state-of-the-art RL-based approaches and $220\%$ with PID-based methods in challenging environments. This work represents the first integration of VLM-based reasoning to assist EVT agents in proactive failure recovery, offering substantial advances for real-world robotic applications that require continuous target monitoring in dynamic, unstructured environments. Project website: https://sites.google.com/view/evt-recovery-assistant.

Figures

Figures reproduced from arXiv: 2505.20718 by the authors.

Figure 1
Figure 1. Comparison of tracking capabilities between traditional tracking [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The framework for integrating Vision-Language Models (VLMs) with active tracking policies. The framework follows a structured recovery [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Four high-fidelity virtual environments used for testing the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Visualization of a successful recovery sequence in [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 26 canonical work pages

  1. [29]

    Empowering embodied visual tracking with visual foundation models and offline rl

    Fangwei Zhong, Kui Wu, Hai Ci, Churan Wang, and Hao Chen. Empowering embodied visual tracking with visual foundation models and offline rl. InEuropean Conference on Computer Vision, pages 139–155, 2024

    work page 2024

  2. [1]

    Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.Arxiv Preprint Arxiv:2308.12966, 1(2):3, 2023

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.Arxiv Preprint Arxiv:2308.12966, 1(2):3, 2023

    arXiv 2023

  3. [2]

    Spatialvlm: Endowing vision-language models with spatial reasoning capabilities

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14455–14465, 2024

    2024

  4. [3]

    Tracking anything with decoupled video segmenta- tion

    Ho Kei Cheng, Seoung Wug Oh, Brian Price, Alexander Schwing, and Joon-Young Lee. Tracking anything with decoupled video segmenta- tion. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 1316–1326, 2023

    work page 2023

  5. [4]

    Changhyun Choi and Henrik I Christensen. Robust 3d visual tracking using particle filtering on the special euclidean group: A combined approach of keypoint and edge features.The International Journal of Robotics Research, 31(4):498–519, 2012

    work page 2012

  6. [5]

    Proactive multi-camera collaboration for 3d human pose estimation

    Hai Ci, Mickel Liu, Xuehai Pan, Fangwei Zhong, and Yizhou Wang. Proactive multi-camera collaboration for 3d human pose estimation. In The Eleventh International Conference on Learning Representations, 2023

    work page 2023

  7. [6]

    Enhancing continuous control of mobile robots for end-to-end visual active tracking.Robotics and Autonomous Systems, 142:103799, 2021

    Alessandro Devo, Alberto Dionigi, and Gabriele Costante. Enhancing continuous control of mobile robots for end-to-end visual active tracking.Robotics and Autonomous Systems, 142:103799, 2021

    work page 2021

  8. [7]

    E-vat: An asymmetric end-to-end approach to visual active exploration and tracking.IEEE Robotics and Automation Letters, 7(2):4259–4266, 2022

    Alberto Dionigi, Alessandro Devo, Leonardo Guiducci, and Gabriele Costante. E-vat: An asymmetric end-to-end approach to visual active exploration and tracking.IEEE Robotics and Automation Letters, 7(2):4259–4266, 2022

    work page 2022

Show all 32 references
  1. [8]

    D-vat: End-to-end visual active tracking for micro aerial vehicles.IEEE Robotics and Automation Letters, 2024

    Alberto Dionigi, Simone Felicioni, Mirko Leomanni, and Gabriele Costante. D-vat: End-to-end visual active tracking for micro aerial vehicles.IEEE Robotics and Automation Letters, 2024

    2024

  2. [9]

    Memory sharing for large language model based agents.Arxiv Preprint Arxiv:2404.09982, 2024

    Hang Gao and Yongfeng Zhang. Memory sharing for large language model based agents.Arxiv Preprint Arxiv:2404.09982, 2024

    2024

  3. [10]

    An embodied generalist agent in 3d world

    Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, Yan Wang, Qing Li, Song-Chun Zhu, Baoxiong Jia, and Siyuan Huang. An embodied generalist agent in 3d world. In The Twelfth International Conference on Learning Representations Workshop: How Far Are We From AGI, 2024

    2024

  4. [11]

    Conquering ghosts: Relation learning for information reliability representation and end-to-end robust navigation

    Kefan Jin and Xingyao Han. Conquering ghosts: Relation learning for information reliability representation and end-to-end robust navigation. Arxiv Preprint Arxiv:2203.09952, 2022

    2022 arXiv

  5. [12]

    OpenVLA: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan P Foster, Pannag R Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. OpenVLA: An op...

    2024

  6. [13]

    A novel performance evaluation methodology for single-target trackers.IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(11):2137–2155, Nov 2016

    Matej Kristan, Jiri Matas, Ale ˇs Leonardis, Tomas V ojir, Roman Pflugfelder, Gustavo Fernandez, Georg Nebehay, Fatih Porikli, and Luka ˇCehovin. A novel performance evaluation methodology for single-target trackers.IEEE Transactions on Pattern Analysis and Machine Intelligenc...

    2016

  7. [14]

    Person following robot based on real time single object tracking and rgb-d image

    JP Kulkarni and PD Pantawane. Person following robot based on real time single object tracking and rgb-d image. InInternational Conference on Signal and Information Processing, pages 1–5, 2022

    2022

  8. [15]

    Blip-2: Boot- strapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Boot- strapping language-image pre-training with frozen image encoders and large language models. InInternational Conference on Machine Learning, pages 19730–19742, 2023

    2023

  9. [16]

    Vi- sual instruction tuning.Advances in Neural Information Processing Systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Vi- sual instruction tuning.Advances in Neural Information Processing Systems, 36:34892–34916, 2023

    2023

  10. [17]

    End-to-end active object tracking via reinforcement learning

    Wenhan Luo, Peng Sun, Fangwei Zhong, Wei Liu, Tong Zhang, and Yizhou Wang. End-to-end active object tracking via reinforcement learning. InInternational Conference on Machine Learning, pages 3286–3295, 2018

    2018

  11. [18]

    Curious george: An attentive semantic robot.Robotics and Autonomous Systems, 56(6):503–511, 2008

    David Meger, Per-Erik Forss ´en, Kevin Lai, Scott Helmer, Sancho McCann, Tristram Southey, Matthew Baumann, James J Little, and David G Lowe. Curious george: An attentive semantic robot.Robotics and Autonomous Systems, 56(6):503–511, 2008

    2008

  12. [19]

    The hands-free push-cart: Autonomous following in front by predicting user trajectory around obstacles

    Payam Nikdel, Rakesh Shrestha, and Richard Vaughan. The hands-free push-cart: Autonomous following in front by predicting user trajectory around obstacles. InProceedings of the IEEE International Conference on Robotics and Automation, pages 4548–4554, 2018

    2018

  13. [20]

    Unrealcv: Virtual worlds for computer vision

    Weichao Qiu, Fangwei Zhong, Yi Zhang, Siyuan Qiao, Zihao Xiao, Tae Soo Kim, Yizhou Wang, and Alan Yuille. Unrealcv: Virtual worlds for computer vision. InProceedings of the 2017 ACM on Multimedia Conference, pages 1221–1224, 2017

    2017

  14. [21]

    Tracking multiple moving targets with a mobile robot using particle filters and statistical data association

    Dirk Schulz, Wolfram Burgard, Dieter Fox, and Armin B Cremers. Tracking multiple moving targets with a mobile robot using particle filters and statistical data association. InProceedings of the IEEE International Conference on Robotics and Automation, volume 2, pages 1665–1670...

    2001

  15. [22]

    Accurate and real-time 3-d tracking for the following robots by fusing vision and ultrasonar information

    Mengmeng Wang, Yong Liu, Daobilige Su, Yufan Liao, Lei Shi, Jinhong Xu, and Jaime Valls Miro. Accurate and real-time 3-d tracking for the following robots by fusing vision and ultrasonar information. IEEE/ASME Transactions On Mechatronics, 23(3):997–1006, 2018

    2018

  16. [23]

    Vlfm: Vision-language frontier maps for zero-shot semantic navigation

    Naoki Yokoyama, Sehoon Ha, Dhruv Batra, Jiuguang Wang, and Bernadette Bucher. Vlfm: Vision-language frontier maps for zero-shot semantic navigation. InIEEE International Conference on Robotics and Automation, pages 42–48, 2024

    2024

  17. [24]

    Navid: Video-based vlm plans the next step for vision-and-language navigation.Arxiv Preprint Arxiv:2402.15852, 2024

    Jiazhao Zhang, Kunyu Wang, Rongtao Xu, Gengze Zhou, Yicong Hong, Xiaomeng Fang, Qi Wu, Zhizheng Zhang, and He Wang. Navid: Video-based vlm plans the next step for vision-and-language navigation.Arxiv Preprint Arxiv:2402.15852, 2024

    2024 arXiv

  18. [25]

    Vision- language models for vision tasks: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. Vision- language models for vision tasks: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  19. [26]

    Ad-vat+: An asymmetric dueling mechanism for learning and understanding visual active tracking.IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(5):1467–1482, 2019

    Fangwei Zhong, Peng Sun, Wenhan Luo, Tingyun Yan, and Yizhou Wang. Ad-vat+: An asymmetric dueling mechanism for learning and understanding visual active tracking.IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(5):1467–1482, 2019

    2019

  20. [27]

    AD-V AT: An asymmetric dueling mechanism for learning visual active tracking

    Fangwei Zhong, Peng Sun, Wenhan Luo, Tingyun Yan, and Yizhou Wang. AD-V AT: An asymmetric dueling mechanism for learning visual active tracking. InInternational Conference on Learning Representations, 2019

    2019

  21. [28]

    Towards distraction-robust active visual tracking

    Fangwei Zhong, Peng Sun, Wenhan Luo, Tingyun Yan, and Yizhou Wang. Towards distraction-robust active visual tracking. InInterna- tional Conference on Machine Learning, pages 12782–12792, 2021

    2021

  22. [30]

    Unrealzoo: Enriching photo-realistic virtual worlds for embodied ai.ArXiv Preprint ArXiv:2412.20977, 2024

    Fangwei Zhong, Kui Wu, Churan Wang, Hao Chen, Hai Ci, Zhoujun Li, and Yizhou Wang. Unrealzoo: Enriching photo-realistic virtual worlds for embodied ai.ArXiv Preprint ArXiv:2412.20977, 2024

    2024 arXiv

  23. [31]

    On deep recurrent reinforcement learning for active visual tracking of space noncooperative objects.IEEE Robotics and Automation Letters, 8(8):4418–4425, 2023

    Dong Zhou, Guanghui Sun, Zhao Zhang, and Ligang Wu. On deep recurrent reinforcement learning for active visual tracking of space noncooperative objects.IEEE Robotics and Automation Letters, 8(8):4418–4425, 2023

    2023

  24. [32]

    Navgpt-2: Unleashing navigational reasoning capability for large vision-language models

    Gengze Zhou, Yicong Hong, Zun Wang, Xin Eric Wang, and Qi Wu. Navgpt-2: Unleashing navigational reasoning capability for large vision-language models. InEuropean Conference on Computer Vision, pages 260–278, 2025

    2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.