REVIEW 4 major objections 6 minor 68 references
WorldScape Policy 2.0 reaches 94.3% average success in bimanual manipulation by layering event-level memory over short-term visual memory.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 14:09 UTC pith:MSBUMKDP
load-bearing objection A real engineering advance in WAM memory and event-grounded pretraining, but the headline RoboTwin result is not established — the gap to the runner-up is likely within noise and the comparison is not data-matched. the 4 major comments →
WorldScape Policy 2.0: Empowering Steerable World Action Modeling with Reasoning-Augmented Memory
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim: a world action model that jointly predicts future video and robot actions becomes controllable and long-horizon capable when history is organized at two levels—a causal buffer of recent visual latents and an event memory of VLM outputs stored as global-history, local-active, and event-boundary views. Gated retrieval fuses that memory into current reasoning tokens; a semantic-forcing loss aligns them with fine-grained caption embeddings, transferring event semantics into autonomous planning. On RoboTwin 2.0 it reports 94.3% average success (94.3% clean, 94.2% randomized) and 47.9% on the clean-to-randomized protocol; real-world dual-arm tasks reach 75–80% on folding, cleani
What carries the argument
The load-bearing mechanism is the reasoning-augmented long short-term memory: a causal short-term visual memory feeds recent observation latents into the diffusion transformer as prefill, while a long short-term event memory organizes historical vision-language-model outputs into global-history, local-active, and event-boundary views, retrieved by learned gated attention into current perception and planning tokens. These memory-enhanced tokens form an implicit subgoal condition, and a semantic-forcing loss aligns them with fine-grained caption embeddings, transferring explicit event semantics into autonomous planning. ManipEvent-5M supplies the captions, goal images, and video prompts that m
Load-bearing premise
The load-bearing premise is that ManipEvent-5M's automatically generated event boundaries and fine-grained captions are accurate enough to supervise fine-grained grounding and semantic forcing; the paper reports no human agreement or error rate, only that outputs were parsed and validated as structured records.
What would settle it
Shuffle or delay the event boundaries in ManipEvent-5M by a few seconds for a matched pretraining run, or replace fine-grained captions with the episode-level instruction; if success on RoboTwin 2.0 barely moves, then event-level semantic supervision is not the active ingredient. A reader could also compare the model trained on the automatic labels against one trained on a small human-verified subset of the same episodes.
If this is right
- If the reported numbers are representative, memory-grounded world action models can replace the separate planner-plus-executor stacks used in many long-horizon systems.
- The negligible clean-to-randomized gap (94.3% vs 94.2%) implies the design generalizes to visual and physical perturbations when trained on both, while the 47.9% clean-to-randomized score shows a meaningful gain even without seeing randomized data.
- Unified multimodal prompting—text, goal image, video—becomes a single interface, so the same backbone can be steered at different granularities depending on what the user supplies.
- Since ablations show monotonic gains from short-term memory, event memory, and latent reasoning, the paper's design claims each component is load-bearing rather than decorative.
- The dataset contributes a reusable asset: nearly five million event-segmented manipulation trajectories with aligned captions and visual prompts, which other world-model and vision-language-action training pipelines could adopt.
Where Pith is reading between the lines
- One testable extension: apply the same reasoning-augmented memory to a non-world-model vision-language-action policy; if gains persist, the benefit is the memory structure, not joint video-action pretraining.
- A stronger test of the causal story would be a human-verified subset of ManipEvent-5M; if caption noise is removed and performance does not improve, then semantic forcing may be learning from scale rather than from event semantics.
- The event-boundary detection via latent cosine change suggests a generic progress-tracking prior that could transfer to other long-horizon domains, such as navigation or assembly, where subtask boundaries are not annotated.
- The paper does not report breakdowns by data source; a natural follow-up is to measure how much of the gain comes from the egocentric human-video portion versus real robot data, which would speak to the data-efficiency of the approach.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes WorldScape Policy 2.0, a world action model that couples a causal short-term visual memory (recent frames as DiT prefill) with a long-term event memory built from VLM outputs (global-history, local-active, and event-boundary views) and an autoregressively generated latent subgoal condition. A semantic-forcing loss aligns the latent planning tokens with fine-grained event captions. To support training, the authors construct ManipEvent-5M, an event-grounded dataset of about 4.89M segments with hierarchical captions, goal images, and video prompts, and use a three-stage curriculum. The paper reports 94.3% average success on RoboTwin 2.0, monotonic gains in component and curriculum ablations, and real-robot results on dual-arm tasks including folding, table cleaning, shell game, and cross-embodiment stacking.
Significance. If the claims hold, the paper makes a useful step toward long-horizon, steerable WAMs: the combination of frame-level and event-level memory within a single WAM is well motivated, and the real-robot evaluation spans four capabilities. The constructed ManipEvent-5M dataset and the three-stage training recipe are potentially valuable contributions. The internal ablations (Tables 5 and 6) are a strength: they show monotonic improvements when memory stages and semantic forcing are added. However, the headline comparative claims are not established as stated: the main RoboTwin table is not data-matched, no variance is reported, the C2R comparison omits the closest baselines, and the automatically generated event labels are not validated against human judgments. These issues are fixable and do not invalidate the approach, but they must be addressed before the central claims can be accepted.
major comments (4)
- [§4.3, Table 2] The main comparison is not data-matched. WorldScape Policy 2.0 is pretrained on ManipEvent-5M (§3.5, Table 1, ~4.89M segments) and initialized from Wan2.2-5B, while the baselines are described only as fine-tuned for 50K steps on clean-plus-randomized data. Reported advantages over Abot-M0.5 (94.3% vs 94.1%) and LingBot-VA 2.0 (94.3% vs 93.6%) are within a range that can easily be sampling noise; no error bars, confidence intervals, or significance tests are provided anywhere. This prevents attributing the gains to the proposed memory and semantic-forcing mechanisms. Please report variance estimates and, if possible, data-matched baselines (e.g., the same backbone trained with and without ManipEvent-5M, or with episode-level captions only).
- [Figure 10] In the C2R benchmark, the paper compares only against pi0, pi0.5, and Fast-WAM and omits Abot-M0.5 and LingBot-VA 2.0, which are the two closest competitors in Table 2. Without their C2R scores, the claim that WorldScape Policy 2.0 'achieves the highest average success rate' at 47.9% is unsupported. Report those numbers or explicitly explain why they are unavailable.
- [Tables 5 and 6] The component and curriculum ablations show monotonic improvements, and that is a strength. However, no multiple-seed variance is reported. The increments from Stage-2 mid-training and semantic forcing are small (average 47.30% to 47.89% in Table 6; randomized-condition gains around 0.4–0.7 points), and with 100 trials per task across 50 tasks these differences may be within noise. Please add seeds, error bars, and, ideally, a significance test, particularly for the contribution of semantic forcing, which is central to the paper's claim that event semantics transfer to latent subgoal planning.
- [§3.5, Eq. (16)] The event boundaries and captions in ManipEvent-5M are generated automatically by Qwen3-VL-32B. The paper states only that outputs are 'parsed and validated as structured records' with retries and fallbacks; no human agreement, error rate, or manual sample audit is reported. These captions are the supervision targets of the semantic-forcing loss (Eq. 16) and the event-boundary selections in Eq. (11). If the labels are noisy or boundaries are misaligned, the claimed progress-aware reasoning is trained on unreliable targets. Please provide a human-validated evaluation of caption and boundary accuracy on a representative sample, and, if feasible, a label-noise sensitivity analysis.
minor comments (6)
- [Table 1] The segment counts for RoboMIND, RoboCOIN, and DROID are shown as '–', while 'Single-Seg. Ratio' is 100%. Clarify whether these datasets are treated as single-segment episodes and whether their segment counts are included in the total or not.
- [Eqs. (10)–(11)] The notation TopKΔ and the 'Expand' operation are used without a precise formal definition. Specify the greedy selection procedure with the minimum temporal separation Δ and the exact slot expansion used for global-history latents.
- [§4.2] Several hyperparameters that appear in the method are not given concrete values: S_v (short-term visual window), S_b and Δ (event-boundary count/separation), λ_s, and α. The text gives S_e=8 and K=4, but the others should be listed for reproducibility.
- [Figure 2] The caption text appears garbled: 'First-view of Goal ImageThird-view of Goal Image' is missing a separator. Also, the figure's 'Long Short-Term Memory WAM' label is ambiguous because the paper distinguishes short-term visual memory from long-term event memory.
- [§3.3] The sentence 'Following the implementation' is informal and does not identify which implementation is meant. Replace it with a precise description of how the compact full-history bank is concatenated with the three memory views.
- [References] Reference [62] for WorldScape Policy 1.0 is formatted as an organization-only citation with no authors or technical report number. If this is a self-citation to an unpublished predecessor, please provide full bibliographic information or a publicly accessible technical report.
Circularity Check
No significant circularity: the paper's central claims are empirical results of supervised pretraining and external-benchmark evaluation, not reconstructions of their inputs.
full rationale
The derivation chain is not circular. Semantic forcing (Eq. 16) uses fine-grained captions as training-time targets only; at inference the caption and its T5 embedding are absent (Eq. 17 selects the memory-enhanced latent q_hat), so the model is trained to predict a latent subgoal from observations, memory, and the high-level instruction rather than copying the caption into the output. Event-boundary selection (Eq. 11) computes boundary indices from historical VLM latents, but these are memory-retrieval anchors, not the predicted actions or the benchmark success labels; no equation retrofits the reported success rate into the model's input. ManipEvent-5M's auto-generated captions and boundaries (Sec. 3.5) are a supervision-quality risk, since validation is only described as 'parsed and validated as structured records,' but that is a data-quality/correctness concern, not an equivalence-by-construction. The self-citation to WorldScape Policy 1.0 [62] and other author-affiliated works is used as a baseline or contextual comparison, not as the load-bearing justification for the central result. The benchmark gaps over Abot-M0.5 are small and lack reported variance, which is a statistical-robustness concern, not circularity. Overall, the core training and evaluation pipeline is self-contained against external RoboTwin 2.0 and real-robot tasks, so no circular step meets the quoting-and-reduction standard.
Axiom & Free-Parameter Ledger
free parameters (6)
- Short-term visual memory window S_v =
4 chunks
- Long-term event memory size S_e =
8 chunks (default)
- Planning tokens K =
4
- Event-boundary count S_b and separation Delta =
not specified
- Semantic forcing weight lambda_s =
0.001 when active
- Residual scaling factor alpha =
not specified
axioms (4)
- domain assumption ManipEvent-5M event boundaries and captions generated by Qwen3-VL (with robot-signal priors) are accurate enough to serve as training targets.
- domain assumption T5 embeddings of fine-grained captions form a semantic space to which planning latents can be aligned via L_sem.
- domain assumption K=4 greedy-decoded VLM tokens provide a sufficient latent subgoal representation.
- domain assumption The Wan2.2-5B text-to-video pretrained DiT is a suitable backbone for joint video-action modeling.
invented entities (3)
-
Implicit subgoal latent (planning tokens)
no independent evidence
-
Three memory views (global-history, local-active, event-boundary latents)
no independent evidence
-
ManipEvent-5M dataset
no independent evidence
read the original abstract
World Action Models (WAMs) offer a promising paradigm for robotic manipulation by jointly modeling visual state transitions and robot actions. However, existing WAMs are constrained by limited temporal context, coarse episode-level language supervision, and predominantly text-only conditioning, which hinder task-progress tracking and fine-grained language-video-action grounding while limiting visual-context reasoning and cross-embodiment transfer. In this paper, we introduce WorldScape Policy 2.0, a controllable WAM with reasoning-augmented long short-term memory. Its causal short-term visual memory supplies recent observations as DiT prefill to preserve local interaction dynamics, while its long short-term event memory organizes historical VLM outputs into global-history, local-active, and event-boundary representations for progress-aware retrieval. The retrieved history augments perception and autoregressively generated planning tokens, yielding an implicit subgoal condition for autonomous planning; semantic forcing further transfers event-level instruction semantics into this latent planning pathway. To establish fine-grained multimodal controllability, we construct ManipEvent-5M, an event-grounded embodied pretraining dataset containing nearly 5 million event segments with aligned action trajectories, episode-level task instructions, segment-level subtask captions, goal images, and video demonstrations. These designs provide a unified interface for autonomous planning from high-level instructions and controllable execution from fine-grained text, goal-image, or video-context prompts. Experiments in both simulation and real-world platforms demonstrate superior capabilities in long-horizon autonomous planning, fine-grained instruction following and in-context adaptation.
Figures
Reference graph
Works this paper leans on
-
[1]
Yu Shang, Zhuohang Li, Yiding Ma, Weikang Su, Xin Jin, Ziyou Wang, Lei Jin, Xin Zhang, Yinzhou Tang, Haisheng Su, et al. Worldarena: A unified benchmark for evaluating perception and functional utility of embodied world models.arXiv preprint arXiv:2602.08971, 2026
arXiv 2026
-
[2]
Robosense: Large-scale dataset and benchmark for egocentric robot perception and navigation in crowded and unstructured environments
Haisheng Su, Feixiang Song, Cong Ma, Wei Wu, and Junchi Yan. Robosense: Large-scale dataset and benchmark for egocentric robot perception and navigation in crowded and unstructured environments. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 27446–27455, 2025
2025
-
[3]
Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Robert Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Allen Z. Ren, ...
-
[4]
𝜋0.7: A steerable generalist robotic foundation model with emergent capabilities, 2026
Physical Intelligence, Bo Ai, Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Greg Balke, Kevin Black, et al. 𝜋0.7: A steerable generalist robotic foundation model with emergent capabilities, 2026. URLhttps: //arxiv.org/abs/2604.15483
Pith/arXiv arXiv 2026
-
[5]
Drivemoe: Mixture-of-expertsforvision-language-actionmodelinend-to-endautonomousdriving
Zhenjie Yang, Yilin Chai, Xiaosong Jia, Qifeng Li, Yuqian Shao, Xuekai Zhu, Haisheng Su, and Junchi Yan. Drivemoe: Mixture-of-expertsforvision-language-actionmodelinend-to-endautonomousdriving. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10678–10688, 2026
2026
-
[6]
Tianyuan Yuan, Zibin Dong, Yicheng Liu, and Hang Zhao. Fast-wam: Do world action models need test-time future imagination?arXiv preprint arXiv:2603.16666, 2026
Pith/arXiv arXiv 2026
-
[7]
Motus: A unified latent action world model
Hongzhe Bi, Hengkai Tan, Shenghao Xie, Zeyuan Wang, Shuhe Huang, Haitian Liu, Ruowen Zhao, Yao Feng, Chendong Xiang, Yinze Rong, Hongyan Zhao, Hanyu Liu, Zhizhong Su, Lei Ma, Hang Su, and Jun Zhu. Motus: A unified latent action world model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 35101–35113, June ...
2026
-
[8]
Jun Guo, Qiwei Li, Peiyan Li, Zilong Chen, Nan Sun, Yifei Su, Heyun Wang, Yuan Zhang, Xinghang Li, and Huaping Liu. Unified 4d world action modeling from video priors with asynchronous denoising.arXiv preprint arXiv:2604.26694, 2026
Pith/arXiv arXiv 2026
-
[9]
Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Hao Li, Hengtao Li, Jie Li, Jindi Lv, Jingyu Liu, et al. Gigaworld-policy: An efficient action-centered world–action model.arXiv preprint arXiv:2603.17240, 2026
arXiv 2026
-
[10]
Causal world modeling for robot control
Lin Li, Qihang Zhang, Yiming Luo, Shuai Yang, Ruilin Wang, Luyao Zhang, Mingrui Yu, Zelin Gao, Nan Xue, Boyu Zhou, Xing Zhu, Mingyu Ding, Yujun Shen, and Yinghao Xu. Causal world modeling for robot control. In Proceedings of Robotics: Science and Systems, 2026. URLhttps://roboticsconference.org/program/ papers/16/
2026
-
[11]
World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026
Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, Suneel Indupuru, You Liang Tan, Chuning Zhu, Jiannan Xiang, et al. World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026
Pith/arXiv arXiv 2026
-
[12]
Sizhe Yang, Juncheng Mu, Tianming Wei, Chenhao Lu, Xiaofan Li, Linning Xu, Zhengrong Xue, Zhecheng Yuan, Dahua Lin, Jiangmiao Pang, and Huazhe Xu. MemoryWAM: Efficient world action modeling with persistent memory.arXiv preprint arXiv:2606.20562, 2026. 20
Pith/arXiv arXiv 2026
-
[13]
DINO-WM: World models on pre-trained visual features enable zero-shot planning
Gaoyue Zhou, Hengkai Pan, Yann Lecun, and Lerrel Pinto. DINO-WM: World models on pre-trained visual features enable zero-shot planning. InProceedings of the 42nd International Conference on Machine Learning, volume 267 ofProceedings of Machine Learning Research, pages 79115–79135. PMLR, 2025. URLhttps: //proceedings.mlr.press/v267/zhou25t.html
2025
-
[14]
Enerverse: Envisioning em- bodied future space for robotics manipulation
Siyuan Huang, Liliang Chen, Pengfei Zhou, Shengcong Chen, Yue Liao, Zhengkai Jiang, Yue Hu, Peng Gao, Hongsheng Li, Maoqing Yao, and Guanghui Ren. Enerverse: Envisioning em- bodied future space for robotics manipulation. InAdvances in Neural Information Processing Sys- tems, volume 38, 2025. URL https://proceedings.neurips.cc/paper_files/paper/2025/hash/ ...
2025
-
[15]
Video prediction policy: A generalist robot policy with predictive visual representations
Yucheng Hu, Yanjiang Guo, Pengchao Wang, Xiaoyu Chen, Yen-Jen Wang, Jianke Zhang, Koushil Sreenath, Chaochao Lu, and Jianyu Chen. Video prediction policy: A generalist robot policy with predictive visual representations. InProceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 24...
2025
-
[16]
Dreamgen: Unlocking generalization in robot learning through video world models
Joel Jang, Seonghyeon Ye, Zongyu Lin, Jiannan Xiang, Johan Bjorck, Yu Fang, Fengyuan Hu, Spencer Huang, Kaushil Kundalia, Yen-Chen Lin, Loïc Magne, Ajay Mandlekar, Avnish Narayan, You Liang Tan, Guanzhi Wang, Jing Wang, Qi Wang, Yinzhen Xu, Xiaohui Zeng, Kaiyuan Zheng, Ruijie Zheng, Ming-Yu Liu, Luke Zettlemoyer, Dieter Fox, Jan Kautz, Scott Reed, Yuke Zh...
2025
-
[17]
Learning universal policies via text-guided video generation.Advances in neural information processing systems, 36:9156–9172, 2023
YilunDu, SherryYang, BoDai, HanjunDai, OfirNachum, JoshTenenbaum, DaleSchuurmans, andPieterAbbeel. Learning universal policies via text-guided video generation.Advances in neural information processing systems, 36:9156–9172, 2023
2023
-
[18]
Yao Feng, Hengkai Tan, Xinyi Mao, Guodong Liu, Shuhe Huang, Chendong Xiang, Hang Su, and Jun Zhu. Vidar: Embodied video diffusion model for generalist bimanual manipulation.arXiv preprint arXiv:2507.12898, 2025
Pith/arXiv arXiv 2025
-
[19]
Gen2Act: Human video generation in novel scenarios enables generalizable robot manipulation
Homanga Bharadhwaj, Debidatta Dwibedi, Abhinav Gupta, Shubham Tulsiani, Carl Doersch, Ted Xiao, Dhruv Shah, Fei Xia, Dorsa Sadigh, and Sean Kirmani. Gen2Act: Human video generation in novel scenarios enables generalizable robot manipulation. InProceedings of The 9th Conference on Robot Learning, volume 305 of Proceedings of Machine Learning Research, page...
2025
-
[20]
Haisheng Su, Kunchang Li, Jinyuan Feng, Dongliang Wang, Weihao Gan, Wei Wu, and Yu Qiao. Tsi: temporal saliency integration for video action recognition.arXiv preprint arXiv:2106.01088, 2021
Pith/arXiv arXiv 2021
-
[21]
Discovering a variety of objects in spatio-temporal human-object interactions
Yong-Lu Li, Hongwei Fan, Zuoyu Qiu, Yiming Dou, Liang Xu, Hao-Shu Fang, Peiyang Guo, Haisheng Su, Dongliang Wang, Wei Wu, et al. Discovering a variety of objects in spatio-temporal human-object interactions. arXiv preprint arXiv:2211.07501, 2022
Pith/arXiv arXiv 2022
-
[22]
Haisheng Su, Jing Su, Dongliang Wang, Weihao Gan, Wei Wu, Mengmeng Wang, Junjie Yan, and Yu Qiao. Collaborative distillation in the parameter and spectrum domains for video action recognition.arXiv preprint arXiv:2009.06902, 2020
Pith/arXiv arXiv 2009
-
[23]
RoboDreamer: Learning compositional world models for robot imagination
Siyuan Zhou, Yilun Du, Jiaben Chen, Yandong Li, Dit-Yan Yeung, and Chuang Gan. RoboDreamer: Learning compositional world models for robot imagination. InProceedings of the 41st International Conference on Machine Learning, volume 235 ofProceedings of Machine Learning Research, pages 61885–61896. PMLR, 2024. URLhttps://proceedings.mlr.press/v235/zhou24f.html
2024
-
[24]
Haisheng Su, Wei Wu, Feixiang Song, Junjie Zhang, Zhenjie Yang, and Junchi Yan. Drivemamba: Task-centric scalable state space model for efficient end-to-end autonomous driving.arXiv preprint arXiv:2602.13301, 2026. 21
arXiv 2026
-
[25]
Egofsd: Ego-centricfullysparseparadigmwithuncertainty denoising and iterative refinement for efficient end-to-end self-driving
HaishengSu,WeiWu,ZhenjieYang,andIsabelGuan. Egofsd: Ego-centricfullysparseparadigmwithuncertainty denoising and iterative refinement for efficient end-to-end self-driving. In2026 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2026
2026
-
[26]
Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, et al. V-jepa 2: Self-supervised video models enable understanding, prediction and planning.arXiv preprint arXiv:2506.09985, 2025
Pith/arXiv arXiv 2025
-
[27]
Gigaworld-0: Worldmodelsasdataenginetoempowerembodiedai.arXivpreprint arXiv:2511.19861, 2025
GigaWorld Team, Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Haoyun Li, Jiagang Zhu, KeruiLi,MengyuanXu, etal. Gigaworld-0: Worldmodelsasdataenginetoempowerembodiedai.arXivpreprint arXiv:2511.19861, 2025
arXiv 2025
-
[28]
Cosmos policy: Fine-tuning video models for visuomotor control and planning
Moo Jin Kim, Yihuai Gao, Tsung-Yi Lin, Yen-Chen Lin, Yunhao Ge, Grace Lam, Percy Liang, Shuran Song, Ming-Yu Liu, Chelsea Finn, and Jinwei Gu. Cosmos policy: Fine-tuning video models for visuomotor control and planning. InInternational Conference on Learning Representations, 2026. URLhttps://iclr.cc/virtual/ 2026/poster/10006732
arXiv 2026
-
[29]
OA-WAM: Object-addressable world action model for robust robot manipulation, 2026
YushanLiu,PeiboSun,ShoujieLi,YifanXie,LingfengZhang,XintaoChao,ShiyuanDong,FangChen,Xiao-Ping Zhang, and Wenbo Ding. OA-WAM: Object-addressable world action model for robust robot manipulation, 2026. URLhttps://arxiv.org/abs/2605.06481
Pith/arXiv arXiv 2026
-
[30]
Long-VLA: Unleashing long-horizon capability of vision language action model for robot manipulation
Yiguo Fan, Shuanghao Bai, Xinyang Tong, Pengxiang Ding, Yuyang Zhu, Hongchao Lu, Fengqi Dai, Wei Zhao, Yang Liu, Siteng Huang, Zhaoxin Fan, Badong Chen, and Donglin Wang. Long-VLA: Unleashing long-horizon capability of vision language action model for robot manipulation. InProceedings of The 9th Conference on Robot Learning, volume 305 ofProceedings of Ma...
2018
-
[31]
VLA-OS: Structuring and dissecting planning representations and paradigms in vision-language-action models
Chongkai Gao, Zixuan Liu, Zhenghao Chi, Junshan Huang, Xin Fei, Yiwen Hou, Yuxuan Zhang, Yudi Lin, Zhirui Fang, and Lin Shao. VLA-OS: Structuring and dissecting planning representations and paradigms in vision-language-action models. InAdvances in Neural Information Processing Sys- tems, volume 38, 2025. URL https://proceedings.neurips.cc/paper_files/pape...
2025
-
[32]
MEM: Multi-scale embodied memory for vision language action models, 2026
Marcel Torne, Karl Pertsch, Homer Walke, Kyle Vedder, Suraj Nair, Brian Ichter, et al. MEM: Multi-scale embodied memory for vision language action models, 2026. URLhttps://arxiv.org/abs/2603.03596
arXiv 2026
-
[33]
MemoryVLA: Perceptual-cognitive memory in vision-language-action models for robotic manipulation
Hao Shi, Bin Xie, Yingfei Liu, Lin Sun, Fengrong Liu, Tiancai Wang, Erjin Zhou, Haoqiang Fan, Xiangyu Zhang, and Gao Huang. MemoryVLA: Perceptual-cognitive memory in vision-language-action models for robotic manipulation. InInternational Conference on Learning Representations, 2026. URLhttps://openreview. net/forum?id=54U3XHf7qq
2026
-
[34]
Long-horizon manipulation via trace-conditioned VLA planning, 2026
Isabella Liu, An-Chieh Cheng, Rui Yan, Geng Chen, Ri-Zhao Qiu, Xueyan Zou, Sha Yi, Hongxu Yin, Xiaolong Wang, and Sifei Liu. Long-horizon manipulation via trace-conditioned VLA planning, 2026. URLhttps: //arxiv.org/abs/2604.21924
Pith/arXiv arXiv 2026
-
[35]
Goal2Skill: Long-horizon manipulation with adaptive planning and reflection,
Zhen Liu, Xinyu Ning, Zhe Hu, Xinxin Xie, Weize Li, Zhipeng Tang, Chongyu Wang, Zejun Yang, Hanlin Wang, Yitong Liu, and Zhongzhu Pu. Goal2Skill: Long-horizon manipulation with adaptive planning and reflection,
-
[36]
DSWAM: A dual-system world action foundation model for fine-grained robot manipulation,
JianZhu,JianjunZhang,TaiyiSu,TianbinLiu,ZhangyuanWang,KaiXie,ZitaiHuang,ChongMa,YouzhangHe, Tianjian Wang, et al. DSWAM: A dual-system world action foundation model for fine-grained robot manipulation,
-
[37]
Barry, Kris Kitani, and George Konidaris
Jiahui Fu, Junyu Nan, Lingfeng Sun, Hongyu Li, Jianing Qian, Jennifer L. Barry, Kris Kitani, and George Konidaris. NovaPlan: Zero-shot long-horizon manipulation via closed-loop video language planning, 2026. URL https://arxiv.org/abs/2602.20119. 22
arXiv 2026
-
[38]
Qiuyue Wang, Mingsheng Li, Jian Guan, Jinhui Ye, Sicheng Xie, Yitao Liu, Junhao Chen, Zhixuan Liang, Jie Zhang, Xintong Hu, et al. Qwen-VLA: Unifying vision-language-action modeling across tasks, environments, and robot embodiments, 2026. URLhttps://arxiv.org/abs/2605.30280
Pith/arXiv arXiv 2026
-
[39]
URLhttps://arxiv.org/abs/2607.04927
-
[40]
PixelVLA: Advancing pixel-level understanding in vision-language-action model
Wenqi Liang, Gan Sun, Yao He, Jiahua Dong, Suyan Dai, Ivan Laptev, Salman Khan, and Yang Cong. PixelVLA: Advancing pixel-level understanding in vision-language-action model. InInternational Conference on Learning Representations, 2026. URLhttps://openreview.net/forum?id=7M6ryCABIc
2026
-
[41]
SpatialVLA: Exploring spatial representations for vision-language-action models
Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Jiayuan Gu, Zhigang Wang, Yan Ding, Bin Zhao, Dong Wang, and Xuelong Li. SpatialVLA: Exploring spatial representations for vision-language-action models. InProceedings of Robotics: Science and Systems, Los Angeles, CA, USA, June 2025. doi: 10.15607/RSS.2025. XXI.011
-
[42]
FineVLA: Fine-grained instruction alignment for steerable vision-language-action policies, 2026
Xintong Hu, Xuhong Huang, Jinyu Zhang, Yutong Yao, Yuchong Sun, Qiuyue Wang, Mingsheng Li, Sicheng Xie, Yitao Liu, Junhao Chen, et al. FineVLA: Fine-grained instruction alignment for steerable vision-language-action policies, 2026. URLhttps://arxiv.org/abs/2605.27284
Pith/arXiv arXiv 2026
-
[43]
SG- VLA: Learning spatially-grounded vision-language-action models for mobile manipulation, 2026
Ruisen Tu, Arth Shukla, Sohyun Yoo, Xuanlin Li, Junxi Li, Jianwen Xie, Hao Su, and Zhuowen Tu. SG- VLA: Learning spatially-grounded vision-language-action models for mobile manipulation, 2026. URLhttps: //arxiv.org/abs/2603.22760
arXiv 2026
-
[44]
TraceVLA: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies
Ruijie Zheng, Yongyuan Liang, Shuaiyi Huang, Jianfeng Gao, Hal Daumé III, Andrey Kolobov, Furong Huang, and Jianwei Yang. TraceVLA: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies. InInternational Conference on Learning Representations, 2025. URLhttps://openreview.net/ forum?id=b1CVu9l5GO
2025
-
[45]
ReconVLA: Reconstructive vision-language-action model as effective robot perceiver
Wenxuan Song, Ziyang Zhou, Han Zhao, Jiayi Chen, Pengxiang Ding, Haodong Yan, Yuxin Huang, Feilong Tang, Donglin Wang, and Haoang Li. ReconVLA: Reconstructive vision-language-action model as effective robot perceiver. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 18549–18557, 2026. doi: 10.1609/aaai.v40i22.38921. URLhtt...
-
[46]
Learning generalizable robot policy with human demonstration video as a prompt, 2025
Xiang Zhu, Yichen Liu, Hezhong Li, and Jianyu Chen. Learning generalizable robot policy with human demonstration video as a prompt, 2025. URLhttps://arxiv.org/abs/2505.20795
Pith/arXiv arXiv 2025
-
[47]
AgiBot-World Team. AgiBot World Colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. In2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3549–3556. IEEE, 2025. doi: 10.1109/IROS60139.2025.11247088. URLhttps://ieeexplore.ieee.org/ document/11247088/
arXiv 2025
-
[48]
Qingqing Zhao, Yao Lu, Moo Jin Kim, Zipeng Fu, Zhuoyang Zhang, Yecheng Wu, Zhaoshuo Li, Qianli Ma, Song Han,ChelseaFinn,AnkurHanda,Tsung-YiLin,GordonWetzstein,Ming-YuLiu,andDonglaiXiang.CoT-VLA:Vi- sualchain-of-thoughtreasoningforvision-language-actionmodels. InProceedingsoftheIEEE/CVFConferenceon Computer Vision and Pattern Recognition (CVPR), pages 1702...
arXiv 2025
-
[49]
RoboCOIN: An open-sourced bimanual robotic data collection for integrated manipulation,
ShihanWu,XuechengLiu,ShaoxuanXie,PengweiWang,XinghangLi,ZheLi,KaiZhu,HongyuWu,YihengLiu, Zhaoye Long, et al. RoboCOIN: An open-sourced bimanual robotic data collection for integrated manipulation,
-
[50]
DROID: A large-scale in-the-wild robot manipulation dataset
Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. DROID: A large-scale in-the-wild robot manipulation dataset. InProceedings of Robotics: Science and Systems, Delft, Netherlands, July
-
[51]
Robomind: Benchmark on multi-embodiment intelligence normative data for robot manipulation
Kun Wu, Chengkai Hou, Jiaming Liu, Zhengping Che, Xiaozhu Ju, Zhuqin Yang, Meng Li, Yinuo Zhao, Zhiyuan Xu, Guang Yang, et al. Robomind: Benchmark on multi-embodiment intelligence normative data for robot manipulation. InRobotics: Science and Systems (RSS) 2025. Robotics: Science and Systems Foundation, 2025. URLhttps://www.roboticsproceedings.org/rss21/p152.pdf
2025
-
[52]
LIBERO: Benchmarking knowledge transfer for lifelong robot learning
Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. LIBERO: Benchmarking knowledge transfer for lifelong robot learning. InAdvances in Neural Information Processing Systems, volume 36, pages 44776–44791, 2023. URLhttps://proceedings.neurips.cc/paper_files/paper/2023/hash/ 8c3c666820ea055a77726d66fc7d447f-Abstract-Datasets_a...
2023
-
[53]
URLhttps://arxiv.org/abs/2511.17441
-
[54]
Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025
Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan ...
Pith/arXiv arXiv 2025
-
[55]
Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025
Pith/arXiv arXiv 2025
-
[56]
RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation
Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, Weiliang Deng, Yubin Guo, Tian Nian, Xuanbing Xie, Qiangyu Chen, Kailun Su, Tianling Xu, Guodong Liu, Mengkang Hu, Huan-ang Gao, Kaixuan Wang, Zhixuan Liang, Yusen Qin, Xiaokang Yang, Ping Luo, and Yao Mu. RoboTwin 2.0: A scalable d...
-
[57]
URLhttps://icml.cc/virtual/2026/poster/62192
2026
-
[58]
Ronghan Chen, Yandan Yang, Zuojin Tang, Dongjie Huo, Tong Lin, Haoning Wu, Haoyun Liu, Yuzhi Chen, Lulu Zheng, Botai Yuan, et al. Abot-m0. 5: Unified mobility-and-manipulation world action model.arXiv preprint arXiv:2607.00678, 2026
Pith/arXiv arXiv 2026
-
[59]
Yoon, Mouli Sivapurapu, and Jian Zhang
Ryan Hoque, Peide Huang, David J. Yoon, Mouli Sivapurapu, and Jian Zhang. EgoDex: Learning dexterous manipulation from large-scale egocentric video. InInternational Conference on Learning Representations, 2026. URLhttps://mlanthology.org/iclr/2026/hoque2026iclr-egodex/
2026
-
[60]
Holobrain-0 technical report.arXiv preprint arXiv:2602.12062, 2026
XuewuLin,TianweiLin,YunDu,HongyuXie,YiweiJin,JiaweiLi,ShijieWu,QingzeWang,MengdiLi,Mengao Zhao, et al. Holobrain-0 technical report.arXiv preprint arXiv:2602.12062, 2026
arXiv 2026
-
[61]
Qihang Zhang, Lin Li, Luyao Zhang, Shuai Yang, Yiming Luo, Shuaiting Li, Ruilin Wang, Junke Wang, Jiahao Shao, Gangwei Xu, et al. Native video-action pretraining for generalizable robot control.arXiv preprint arXiv:2607.08639, 2026
Pith/arXiv arXiv 2026
-
[62]
InProceedings of Robotics: Science and Systems, Los Angeles, CA, USA, June 2025
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Robert Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, Laura Smith, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Z...
2025
-
[63]
X-VLA: Soft-prompted transformerasscalablecross-embodimentvision-language-actionmodel
Jinliang Zheng, Jianxiong Li, Zhihao Wang, Dongxiu Liu, Xirui Kang, Yuchun Feng, Yinan Zheng, Jiayin Zou, Yilun Chen, Jia Zeng, Tai Wang, Ya-Qin Zhang, Jingjing Liu, and Xianyuan Zhan. X-VLA: Soft-prompted transformerasscalablecross-embodimentvision-language-actionmodel. InInternationalConferenceonLearning Representations, 2026. URLhttps://openreview.net/...
2026
-
[65]
A pragmatic vla foundation model.arXiv preprint arXiv:2601.18692, 2026
Wei Wu, Fan Lu, Yunnan Wang, Shuai Yang, Shi Liu, Fangjing Wang, Qian Zhu, He Sun, Yong Wang, Shuailei Ma, et al. A pragmatic vla foundation model.arXiv preprint arXiv:2601.18692, 2026
Pith/arXiv arXiv 2026
-
[68]
Worldscape policy: Generalizable robotic learning via a foundation world model, 2026
Manifold AI. Worldscape policy: Generalizable robotic learning via a foundation world model, 2026. 24
2026
-
[2024]
doi: 10.15607/RSS.2024.XX.120. 23
-
[2025]
URLhttps://proceedings.mlr.press/v305/black25a.html
-
[2026]
URLhttps://arxiv.org/abs/2604.13942
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.