REVIEW 17 cited by
TrackVLA: Embodied Visual Tracking in the Wild
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
TrackVLA: Embodied Visual Tracking in the Wild
read the original abstract
Embodied visual tracking is a fundamental skill in Embodied AI, enabling an agent to follow a specific target in dynamic environments using only egocentric vision. This task is inherently challenging as it requires both accurate target recognition and effective trajectory planning under conditions of severe occlusion and high scene dynamics. Existing approaches typically address this challenge through a modular separation of recognition and planning. In this work, we propose TrackVLA, a Vision-Language-Action (VLA) model that learns the synergy between object recognition and trajectory planning. Leveraging a shared LLM backbone, we employ a language modeling head for recognition and an anchor-based diffusion model for trajectory planning. To train TrackVLA, we construct an Embodied Visual Tracking Benchmark (EVT-Bench) and collect diverse difficulty levels of recognition samples, resulting in a dataset of 1.7 million samples. Through extensive experiments in both synthetic and real-world environments, TrackVLA demonstrates SOTA performance and strong generalizability. It significantly outperforms existing methods on public benchmarks in a zero-shot manner while remaining robust to high dynamics and occlusion in real-world scenarios at 10 FPS inference speed. Our project page is: https://pku-epic.github.io/TrackVLA-web.
Forward citations
Cited by 17 Pith papers
-
USS: Unified Spatial-Semantic Prompts for Embodied Visual Tracking with Latent Dynamics Learning
USS is an end-to-end framework for embodied visual tracking that fuses text, point, box, and mask prompts via modality-specific encoders and hybrid attention, augmented by a latent world model, and demonstrates higher...
-
RescueBench: Can Embodied Agents Save Lives in the Wild ?
RescueBench is a new diagnostic benchmark for multi-stage embodied search-and-rescue that shows no tested baselines complete the hardest tasks and identifies exploration and memory as independent failure modes.
-
Beyond Isolation: A Unified Benchmark for General-Purpose Navigation
OmniNavBench is a unified benchmark for general-purpose navigation featuring composite multi-skill instructions, support for humanoid, quadrupedal and wheeled robots, and 1779 human teleoperated trajectories across 17...
-
Towards Backdoor-Based Ownership Verification for Vision-Language-Action Models
GuardVLA embeds a stealthy backdoor watermark in VLAs via secret messages in visual data and uses a swap-and-detect mechanism for post-release ownership verification that preserves task performance.
-
Instance-level Visual Active Tracking with Occlusion-Aware Planning
OA-VAT improves visual active tracking by combining instance-level prototype discrimination with occlusion-aware diffusion planning, reporting gains over prior SOTA on simulated and real drone benchmarks.
-
UESF-Bench: Benchmarking and Probing for Unified Embodied Seeking and Following
UESF-Bench is a 1.43M-sample simulated benchmark for embodied agents that must first find a language-described person and then follow them; SeekFollow-VLA with task-driven routing outperforms the paper's internal baselines.
-
ABot-N1: Toward a General Visual Language Navigation Foundation Model
ABot-N1 decouples navigation into a slow CoT-plus-pixel-goal reasoner and a fast waypoint controller, claiming state-of-the-art results on five VLN benchmarks and releasing two new urban navigation benchmarks.
-
ABot-N1: Toward a General Visual Language Navigation Foundation Model
ABot-N1 decouples VLN into a slow CoT reasoner that outputs pixel goals and a fast action expert, claiming large SOTA gains on urban POI and multi-task navigation.
-
ABot-N1: Toward a General Visual Language Navigation Foundation Model
A slow–fast VLN model that routes five navigation tasks through CoT plus image-space pixel goals reaches SOTA on established and new urban benchmarks, including 77.3% POI arrival.
-
AsyncShield: A Plug-and-Play Edge Adapter for Asynchronous Cloud-based VLA Navigation
AsyncShield restores VLA geometric intent from latency via kinematic pose mapping and uses PPO-Lagrangian to balance tracking with LiDAR safety constraints in a plug-and-play module.
-
A Vision-Language-Action Model for Adaptive Ultrasound-Guided Needle Insertion and Needle Tracking
A VLA model with Cross-Depth Fusion tracking head and TraCon register unifies needle tracking and adaptive insertion control, outperforming prior trackers and manual operation in experiments.
-
Structured Observation Language for Efficient and Generalizable Vision-Language Navigation
SOL-Nav encodes RGB-D observations as grid-organized text and uses a 0.6B text-embedding model with four classification heads to predict navigation action blocks, reporting SOTA/comparable R2R-CE/RxR-CE results with 1...
-
AstraNav-World: World Model for Foresight Control and Consistency
AstraNav-World unifies diffusion video generation and vision-language action planning in a single bidirectional model that improves trajectory accuracy, success rates, and zero-shot real-world adaptation in embodied n...
-
EffiNav: Fusing Depth and Vision-Language for Efficient Object Goal Navigation
EffiNav combines depth and vision-language inputs for efficient object goal navigation, matching or exceeding baselines on success rate and path-length-weighted success across simulation benchmarks and real-robot tests.
-
GN0: Toward a Unified Paradigm for Generation, Evaluation, and Policy Learning in Visual-Language Navigation
GN0 curates GN-Matrix dataset, builds 3DGS simulator and GN-Bench, and trains BAE model via supervised learning plus DAgger and RL to unify VLN tasks and outperform prior methods on GN-Bench and VLN-CE.
-
Vision-and-Language Navigation for UAVs: Progress, Challenges, and a Research Roadmap
A survey of UAV vision-and-language navigation that establishes a methodological taxonomy, reviews resources and challenges, and proposes a forward-looking research roadmap.
-
Structured Observation Language for Efficient and Generalizable Vision-Language Navigation
A 0.6B language model navigates by reading grid-structured text descriptions of depth, object class, and color instead of images, with reported R2R-CE/RxR-CE scores near the top of the leaderboard.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.