REVIEW 4 major objections 6 minor 62 references
A frozen VLA's own action chunk, plus recent multi-view history, is enough to regenerate interception trajectories for moving objects without retraining the VLA.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-12 09:09 UTC pith:SIEFSJD7
load-bearing objection Clean frozen-base adapter plus a useful dynamic sim suite; gains are real in the reported setup, but the “world foundation / interception” framing outruns the pure imitation training. the 4 major comments →
DynaWM: A Base-VLA-Guided World Foundation Model for Moving-Object Manipulation
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
DynaWM shows that a single base-action-conditioned world foundation model—fed only the frozen base VLA's emitted action chunk, multi-view observation history, and proprioceptive state—can regenerate motion-aware trajectories that improve closed-loop success on every one of eight tested base-VLA/checkpoint combinations on DynaGrasp-32, with gains between 1.88 and 45.31 percentage points, without modifying or continuing to train the base VLA.
What carries the argument
Action-conditioned trajectory regeneration: a Mamba-3 encoder turns the base action chunk into a persistent conditioning vector; a V-JEPA 2.1 encoder supplies multi-view motion tokens; both, plus arm state, condition a flow-matching Diffusion Transformer that starts from noise and rewrites the full action chunk rather than applying a residual correction.
Load-bearing premise
That success measured entirely in simulation with human teleoperation demos, fixed short visual history, and a separate DynaWM trained for each base VLA will transfer as a general world-foundation capability to real robots and new embodiments.
What would settle it
Run the identical frozen base VLAs plus the corresponding DynaWM weights on a physical arm with independently moving targets under realistic sensing delay and contact; if the closed-loop success gap over the unaugmented base VLA collapses or reverses relative to the reported Isaac Sim numbers, the central claim fails.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DynaWM, a base-VLA-guided world foundation model for moving-object manipulation. A frozen base VLA emits an action chunk that is encoded by Mamba-3; multi-view short-horizon history is encoded by V-JEPA 2.1; proprioceptive state is encoded by an MLP; these conditions jointly drive a flow-matching DiT that regenerates a motion-aware action chunk without accessing base-VLA tokens or gradients (Eqs. 1–2, 5–9; Figs. 1–2). The authors introduce DynaGrasp-32 (six diagnostic families, 32 Isaac Sim tasks) and DynaGrasp-1600 (1,600 teleop demos, ~1.53M images). Across four base VLAs and fine-/coarse-tuned checkpoints (eight combinations), closed-loop success rises by 1.88–45.31 percentage points (Tables 2–3), with ablations on visual encoder size and action-conditioning formulation (Tables 4–5).
Significance. If the result holds, the work offers a practical, architecture-agnostic interface for adding dynamic interception to heterogeneous frozen VLAs without joint fine-tuning that can degrade pretrained policies. Strengths include a clean freeze-and-cache protocol that attributes gains to the adapter rather than base-VLA retraining, a diagnostic multi-family benchmark that isolates velocity, collision, appearance, and multi-object factors, and ablations that separate visual history from action-sequence encoding. The contribution is primarily empirical and systems-level rather than a new theoretical principle of world modeling; its lasting value depends on whether the gains reflect online interception of independent object motion rather than supervised imitation of teleop demos under base-action conditioning, and on eventual hardware transfer.
major comments (4)
- §3.5 Eq. (9) and Fig. 1a: training is pure supervised flow matching to human teleop demonstration chunks, with base actions only cached offline from the same demos. There is no explicit motion-prediction, interception, or future-state loss, and no held-out velocity/trajectory distribution. Combined with separate DynaWM weights per base VLA, this leaves open whether gains are online interception of independent object motion (§3.2) or offline imitation of demo-like actions given base chunk + history. A load-bearing control is missing: a non-dynamic residual/imitation adapter that receives the same base chunk, multi-view history, and state, or evaluation on motion parameters outside the demo distribution.
- Table 5(b) and §4.4: removing the Mamba action encoder and flowing from the base chunk collapses success from 74.44% to 29.00% (below frozen π0 at 72.56%), with severe drops on Amount and Obstacle. This shows sequential encoding is necessary under the current recipe, but does not establish that the model computes interception geometry rather than remapping base-action structure onto demo trajectories. The paper should either (i) report a residual/direct-start variant that still uses an action encoder, or (ii) provide qualitative/quantitative evidence that regenerated contact points systematically lead the observed target rather than replaying demo timing.
- §4.1–4.2 and Conclusion: all results are Isaac Sim only (50 episodes × 32 tasks), with fixed L=5, K=3, H=10, and human teleop demos. The manuscript positions DynaGrasp-32 as a foundation for later hardware validation, but sim-to-real sensing delay, contact/rebound fidelity, and camera synchronization are load-bearing for the “world foundation” claim. At minimum, the paper should report variance/error bars or multi-seed statistics for the eight model-level averages in Table 2, and clearly bound claims to simulation until real-robot evidence exists.
- Table 3 / §4.3: fine-tuned π0 gains only +1.88% overall and is Up/Tie/Down = 14/4/14, with regressions concentrated on multi-object Amount tasks; fine-tuned π0.5 also drops on Amount despite a net gain. This is an important boundary of the method (local motion correction can disturb language-conditioned sequencing). The central claim of broad adaptation across checkpoints should be qualified, and the paper should analyze when regeneration should defer to the base chunk rather than always replace it.
minor comments (6)
- Abstract vs Table 4: the abstract’s “visual encoding enhances success by 27.50%” and “reducing success by 45.44% if action conditioning is removed” do not map cleanly onto the tabulated numbers (e.g., 39.94→67.44 is +27.5 points on a different backbone; Table 5(b) is a 45.44-point drop from 74.44). Align abstract percentages with explicit table references.
- §3.1 / §3.5: action dimension Da=7 and state “9D” are stated, but the exact joint/gripper/state layout and normalization statistics source should be specified for reproducibility.
- Fig. 3 caption: clarify that frame indices are rollout time, not model-predicted object trajectories (already noted, but easy to misread in multi-panel layout).
- Related work: AHEAD is correctly distinguished as OpenVLA-specific latent prediction; a short explicit comparison table (frozen base, action-only interface, multi-VLA support) would help readers place DynaWM.
- Terminology: “world foundation model” is used for an action-regeneration adapter without future image/latent rollout; a brief definition early in §3 would reduce ambiguity relative to Cosmos-style WFMs.
- Minor prose: “with having performance gains” (Conclusion); “percentage” vs “percentage points” in abstract gains; ensure consistent π0 / π0.5 notation.
Circularity Check
No circular derivation: empirical closed-loop gains on DynaGrasp-32 do not reduce by construction to training inputs or self-citation.
full rationale
DynaWM is an empirical systems paper, not a first-principles derivation. The load-bearing claim is that a frozen base VLA’s emitted action chunk, multi-view history, and proprioception jointly condition a flow-matching DiT that regenerates actions and raises closed-loop success on DynaGrasp-32 (Eq. 2; Tables 2–3). Training is supervised flow matching to human teleop demonstration chunks with cached base actions (Eq. 9; Fig. 1a); evaluation is task success (lift height, basket placement) over 50 closed-loop episodes per task, not reconstruction of a fitted scalar or of the training loss. Ablations (Tables 4–5) remove visual features or action conditioning and report success drops, which is independent measurement rather than a tautology. There is no self-definitional loop (X defined as Y then “derived”), no parameter fitted to a subset and renamed a prediction of a near-identical quantity, and no uniqueness theorem or ansatz imported from overlapping-author prior work as a load-bearing external fact. Self-citations are ordinary related-work pointers; the architecture (Mamba-3, V-JEPA 2.1, DiT flow matching) is justified by external literature. Constructing the benchmark and training set is standard ML practice and does not make reported success rates equal the training objective by construction. Overclaim concerns (imitation vs interception framing, sim-only transfer) are correctness/interpretation issues, not circularity under the stated criteria.
Axiom & Free-Parameter Ledger
free parameters (6)
- action horizon H =
10
- observation history length L =
5
- camera views K =
3
- DiT depth/width/heads and Euler steps =
8 blocks / 512-d / 8 heads / 10 steps
- Mamba action embedding dimensions =
128→256
- per-base-VLA DynaWM weights =
one model per base VLA
axioms (5)
- domain assumption Emitted physical action chunks from heterogeneous VLAs share enough task-relevant structure to serve as a common conditioning interface without internal tokens.
- domain assumption Short multi-view history plus proprioception supplies sufficient motion evidence for interception without explicit velocity labels or future image rollout.
- domain assumption Isaac Sim closed-loop success with fixed success criteria is a valid diagnostic of dynamic manipulation capability for comparing frozen VLAs.
- standard math Flow matching from Gaussian noise with linear probability path is an appropriate generative objective for action-chunk regeneration.
- ad hoc to paper Base VLA checkpoints remain frozen and are not updated during DynaWM training or evaluation.
invented entities (2)
-
DynaWM (base-VLA-guided world foundation model)
no independent evidence
-
DynaGrasp-32 / DynaGrasp-1600
no independent evidence
read the original abstract
Although vision-language-action (VLA) models have received widespread attention, many challenges remain in manipulating dynamic moving objects. In most existing approaches, end-to-end forward or inverse dynamics models, i.e., world models, are incorporated into high-performance base VLA architectures, which may degrade the performance of well-pretrained base VLA models due to inappropriate fine-tuning. In this paper, we propose DynaWM, a base-VLA-guided world foundation model that adapts to a wide variety of fine-tuned and coarse-tuned base-VLA checkpoints for moving-object manipulation. DynaWM uses a Mamba-3-based action encoder to encode the base action chunk produced by the base VLA into an action-conditioning representation, a V-JEPA 2.1 vision encoder to extract features from multi-view observation history, and a proprioceptive state encoder to encode robotic-arm proprioceptive states. These feature representations jointly condition a flow-matching DiT to regenerate motion-aware action trajectories for moving-object manipulation. For systematic evaluation, we construct the DynaGrasp-32 benchmark, covering six categories of moving-object manipulation tasks, including velocity variation, trajectory variation, and multi-object manipulation, as well as the DynaGrasp-1600 dataset, which consists of 32 scenarios, 1,600 demonstration trajectories, and approximately 1.53M images. For fine-tuned base-VLA checkpoints, DynaWM achieves percentage improvements of 7.19, 45.31, 1.88, and 10.94 over SmolVLA, X-VLA, {\pi}0, and {\pi}0.5, respectively. For coarse-tuned base-VLA checkpoints, performance increases by 35.13, 44.06, 35.69, and 26.13 percentage, respectively. Ablation experiments show that visual encoding enhances success by 27.50%, while reducing success by 45.44% if action conditioning is removed.
Figures
Reference graph
Works this paper leans on
-
[1]
Akinola, I., Xu, J., Song, S., Allen, P.K.: Dynamic grasping with reachability and motion awareness (2021)
2021
-
[2]
Assran,M.,Bardes,A.,Fan,D.,Garrido,Q.,Howes,R.,Mojtaba,etal.:V-JEPA2: Self-supervised video models enable understanding, prediction and planning (2025)
2025
-
[3]
Bardes, A., Garrido, Q., Ponce, J., Chen, X., Rabbat, M., LeCun, Y., et al.: Re- visiting feature prediction for learning visual representations from video (2024)
2024
-
[4]
Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., et al.:π0: A vision-language-action flow model for general robot control (2024)
2024
-
[5]
Black, K., Galliker, M.Y., Levine, S.: Real-time execution of action chunking flow policies (2025)
2025
-
[6]
Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Chen, X., Choromanski, K., et al.: RT-2: Vision-language-action models transfer web knowledge to robotic con- trol (2023)
2023
-
[7]
Burgess-Limerick, B., Lehnert, C., Leitner, J., Corke, P.: DGBench: An open- source, reproducible benchmark for dynamic grasping (2022)
2022
-
[8]
Chen, J., Song, W., Chen, S., Wang, J., Li, Z., Li, H.: DFM-VLA: Iterative action refinement for robot manipulation via discrete flow matching (2026) DynaWM for Moving-Object Manipulation 15
2026
-
[9]
Chen, T., Chen, Z., Chen, B., Cai, Z., Liu, Y., Li, Z., et al.: RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation (2025)
2025
-
[10]
Chen, Y., Ge, Y., Zhou, H., Ding, M., Ge, Y., Liu, X.: DIAL: Decoupling intent and action via latent world modeling for end-to-end VLA (2026)
2026
-
[11]
Chi, C., Xu, Z., Feng, S., Cousineau, E., Du, Y., Burchfiel, B., et al.: Diffusion policy: Visuomotor policy learning via action diffusion (2023)
2023
-
[12]
Embodiment Collaboration, O’Neill, A., Rehman, A., Gupta, A., Maddukuri, A., Gupta, A., et al.: Open X-Embodiment: Robotic learning datasets and RT-X mod- els (2023)
2023
-
[13]
Fang, H., Li, S., Wang, S., Xi, X., Liang, D., Bai, X.: Towards generalizable robotic manipulation in dynamic environments (2026)
2026
-
[14]
Feng, X., Cheng, Y., Shi, C., Han, B., Yan, Y., Hong, Y., et al.: Denoising tells when to replan: Denoising-variance adaptive chunking for flow-based robot policies (2026)
2026
-
[15]
Gemini Robotics Team, Abdolmaleki, A., Abeyruwan, S., Ainslie, J., Alayrac, J.B., Arenas, M.G., et al.: Gemini Robotics 1.5: Pushing the frontier of generalist robots with advanced embodied reasoning, thinking, and motion transfer (2025)
2025
-
[16]
Gu, A., Dao, T.: Mamba: Linear-time sequence modeling with selective state spaces (2023)
2023
-
[17]
James, S., Ma, Z., Arrojo, D.R., Davison, A.J.: RLBench: The robot learning benchmark & learning environment (2019)
2019
-
[18]
Jiang, S., Fang, X., Roy, N., Lozano-Pérez, T., Kaelbling, L.P., Ancha, S.: Stream- ing flow policy: Simplifying diffusion/flow-matching policies by treating action tra- jectories as flow trajectories (2025)
2025
-
[19]
Khazatsky, A., Pertsch, K., Nair, S., Balakrishna, A., Dasari, S., Karamcheti, S., et al.: DROID: A large-scale in-the-wild robot manipulation dataset (2024)
2024
-
[20]
Kim, M.J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., et al.: OpenVLA: An open-source vision-language-action model (2024)
2024
-
[21]
Lahoti, A., Li, K.Y., Chen, B., Wang, C., Bick, A., Kolter, J.Z., et al.: Mamba-3: Improved sequence modeling using state space principles (2026)
2026
-
[22]
Lian, S., Yu, B., Lin, X., Shen, Z., Yang, L.T., Jin, Y., et al.: IntentVLA: Short- horizon intent modeling for aliased robot manipulation (2026)
2026
-
[23]
Liang, Q., Cai, B., He, R., Li, H., Teng, T., Duan, H., et al.: Whole-body coordi- nation for dynamic object grasping with legged manipulators (2025)
2025
-
[24]
Lipman, Y., Chen, R.T.Q., Ben-Hamu, H., Nickel, M., Le, M.: Flow matching for generative modeling (2022)
2022
-
[25]
Liu,B.,Zhu,Y.,Gao,C.,Feng,Y.,Liu,Q.,Zhu,Y.,etal.:LIBERO:Benchmarking knowledge transfer for lifelong robot learning (2023)
2023
-
[26]
Liu, H., Xu, C., Ji, J., Mu, Y.:π 0-EqM: Equilibrium matching for closed-loop vision-language-action control (2026)
2026
-
[27]
Liu, S., Wu, L., Li, B., Tan, H., Chen, H., Wang, Z., et al.: RDT-1B: a diffusion foundation model for bimanual manipulation (2024)
2024
-
[28]
Liu, Y., Yu, H., Zhao, J., Li, B., Zhang, D., Li, M., et al.: Learning native contin- uation for action chunking flow policies (2026)
2026
-
[29]
Lu, Y., Liu, Z., Fan, X., Yang, Z., Hou, J., Li, J., et al.: FASTER: Rethinking real-time flow VLAs (2026)
2026
-
[30]
Chang and Z
Ma, T., Zheng, J., Wang, Z., Jiang, C., Cui, A., Liang, J., et al.: DiT4DiT: Jointly modeling video dynamics and actions for generalizable robot control (2026) 16 C. Chang and Z. Deng
2026
-
[31]
Mandlekar, A., Nasiriany, S., Wen, B., Akinola, I., Narang, Y., Fan, L., et al.: Mim- icgen: A data generation system for scalable robot learning using human demon- strations (2023)
2023
-
[32]
Mees, O., Hermann, L., Rosete-Beas, E., Burgard, W.: CALVIN: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks (2021)
2021
-
[33]
Mu, Y., Chen, T., Peng, S., Chen, Z., Gao, Z., Zou, Y., et al.: RoboTwin: Dual-arm robot benchmark with generative digital twins (early version) (2024)
2024
-
[34]
Mur-Labadia, L., Muckley, M., Bar, A., Assran, M., Sinha, K., Rabbat, M., et al.: V-JEPA 2.1: Unlocking dense features in video self-supervised learning (2026)
2026
-
[35]
Nasiriany, S., Maddukuri, A., Zhang, L., Parikh, A., Lo, A., Joshi, A., et al.: RoboCasa: Large-scale simulation of everyday tasks for generalist robots (2024)
2024
-
[36]
NVIDIA, et al.: Cosmos World Foundation Model Platform for Physical AI. arXiv:2501.03575 (2025)
Pith/arXiv arXiv 2025
-
[37]
NVIDIA, et al.: Cosmos 3: Omnimodal World Models for Physical AI. arXiv:2606.02800 (2026)
Pith/arXiv arXiv 2026
-
[38]
NVIDIA: NVIDIA Isaac Sim documentation (2026)
2026
-
[39]
NVIDIA,Bjorck,J.,Castañeda,F.,Cherniadev,N.,Da,X.,Ding,R.,etal.:GR00T N1: An open foundation model for generalist humanoid robots (2025)
2025
-
[40]
Octo Model Team, Ghosh, D., Walke, H., Pertsch, K., Black, K., Mees, O., et al.: Octo: An open-source generalist robot policy (2024)
2024
-
[41]
Physical Intelligence:π 0.7: A steerable generalist robotic foundation model with emergent capabilities. Tech. rep., Physical Intelligence (Apr 2026)
2026
-
[42]
Physical Intelligence, Black, K., Brown, N., Darpinian, J., Dhabalia, K., Driess, D., et al.:π 0.5: a vision-language-action model with open-world generalization (2025)
2025
-
[43]
Shukor, M., Aubakirova, D., Capuano, F., Kooijmans, P., Palma, S., Zouitine, A., et al.: SmolVLA: A vision-language-action model for affordable and efficient robotics (2025)
2025
-
[44]
Sun, J., Zhang, W., Qi, Z., Ren, S., Liu, Z., Zhu, H., et al.: VLA-JEPA: Enhancing vision-language-action model with latent world model (2026)
2026
-
[45]
Sun, X., Xu, Z., Cao, C., Liu, Z., Sun, Y., Pang, J., et al.: AtomVLA: Scalable post-training for robotic manipulation via predictive latent world models (2026)
2026
-
[46]
Syed, S.N., Jakobsson, A., Hao, H., Ichnowski, J.: Intercepting the future: Latent- space predictive world model for dynamic vla manipulation (2026)
2026
-
[47]
Tian, Y., Jin, Y., Yu, B., Shi, Y., Wu, H., Liu, C.H., et al.: Starry: Spatial-temporal action-centric world modeling for robotic manipulation (2026)
2026
-
[48]
Walke, H., Black, K., Lee, A., Kim, M.J., Du, M., Zheng, C., et al.: BridgeData V2: A dataset for robot learning at scale (2023)
2023
-
[49]
Wang, K., Fan, K., Qiu, C., Shangguan, Z., Fu, Y., Fu, Y., et al.: OFlow: Injecting object-aware temporal flow matching for robust robotic manipulation (2026)
2026
-
[50]
Wang, Q., Li, M., Guan, J., Ye, J., Xie, S., Liu, Y., et al.: Qwen-VLA: Unifying vision-language-action modeling across tasks, environments, and robot embodi- ments (2026)
2026
-
[51]
Wang, X., Wang, C., Xu, Y., Ye, M., Zhang, F.C., Tian, J., et al.: LaMP: Learning vision-language-action policies with 3D scene flow as latent motion prior (2026)
2026
-
[52]
Wei, H., Xu, X., Cheng, Z., Yin, H., Ma, A., Yu, B., et al.: F2F-AP: Flow-to-future asynchronous policy for real-time dynamic manipulation (2026)
2026
-
[53]
Wu, T., Zhong, F., Geng, Y., Wang, H., Zhu, Y., Wang, Y., et al.: GraspARL: Dynamic grasping via adversarial reinforcement learning (2022)
2022
-
[54]
Wu, Y., Chen, Z., Ou, C., Wang, W., Huang, W., Cao, L., et al.: ST-VLA: Enabling 4D-aware spatiotemporal understanding for general robot manipulation (2026) DynaWM for Moving-Object Manipulation 17
2026
-
[55]
Xie, H., Wen, B., Zheng, J., Chen, Z., Hong, F., Diao, H., et al.: DynamicVLA: A vision-language-action model for dynamic object manipulation (2026)
2026
-
[56]
Yu, T., Quillen, D., He, Z., Julian, R., Narayan, A., Shively, H., et al.: Meta-World: A benchmark and evaluation for multi-task and meta reinforcement learning (2019)
2019
-
[57]
Zhang, C., Lu, R., Tong, J., Li, X., Wang, Y., Li, H.: EvoScene-VLA: Evolving scene beliefs inside the action decoder for chunked robot control (2026)
2026
-
[58]
Zhang, S., Xu, Z., Liu, P., Yu, X., Li, Y., Gao, Q., et al.: VLABench: A large- scale benchmark for language-conditioned robotics manipulation with long-horizon reasoning tasks (2024)
2024
-
[59]
Zhang, W., Liu, H., Qi, Z., Wang, Y., Yu, X., Zhang, J., et al.: DreamVLA: A vision-language-action model dreamed with comprehensive world knowledge (2025)
2025
-
[60]
Zhao, T.Z., Kumar, V., Levine, S., Finn, C.: Learning fine-grained bimanual ma- nipulation with low-cost hardware (2023)
2023
-
[61]
Zheng, J., Li, J., Wang, Z., Liu, D., Kang, X., Feng, Y., et al.: X-VLA: Soft- prompted transformer as scalable cross-embodiment vision-language-action model (2025)
2025
-
[62]
Zhu, C., Yu, R., Feng, S., Burchfiel, B., Shah, P., Gupta, A.: Unified world models: Coupling video and action diffusion for pretraining on large robotic datasets (2025)
2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.