Pith. sign in

REVIEW 4 major objections 5 minor 98 references

LeapBot-WA claims that world modeling for robot manipulation can be moved entirely out of pixel space: predictive semantic alignment in a JEPA latent, with an isotropic bottleneck and an asymmetric transformer pair, matches video-generating

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-07-31 23:23 UTC pith:TCU2ZH2D

load-bearing objection Fresh and plausible latent-WAM design, but the paper's internal number and objective inconsistencies make the central mechanism unverified. the 4 major comments →

arxiv 2607.23969 v2 pith:TCU2ZH2D submitted 2026-07-27 cs.RO

LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments

Pei Liu , Nan Zheng , Lang Zhang , Daojie Peng , Yanan Zhang , Feilong Kong , Mingyue Feng , Jiachao Liu
show 3 more authors
Yaonong Wang Qifeng Chen Jun Ma
This is my paper
classification cs.RO
keywords world action modelspredictive latent alignmentJEPArobot manipulationdiffusion policylatent world modelzero-shot robustnessflow matching
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that a robot manipulation policy can learn world dynamics without generating future pixels. It does this by using a fine-tuned JEPA encoder as a "predictive anchor" that produces semantic tokens, an isotropic autoencoder (ISAE) that reshapes those tokens into a diffusion-compatible latent space, and an asymmetric mixture-of-transformers that lets a heavy "Anchor DiT" teach a light "Action DiT" during training, then prunes the heavy branch at inference. If the claim holds, the expensive video-reconstruction branch of today's world action models is replaceable by latent predictive alignment, with the same or better task success, much lower inference cost, and improved resilience to visual distractors. The paper reports top predictive-model performance on LIBERO, parity with top generative world action models on RoboTwin 2.0, and zero-shot robustness gains on LIBERO-Plus, plus a real-world UR5 transfer.

Core claim

LeapBot-WA's central discovery is that predictive semantic alignment in a latent foundation space is sufficient for robust manipulation: a policy does not need to render future pixels. The paper supports this by taking a fine-tuned V-JEPA encoder as a frozen predictive anchor, compressing its 1664-dimensional tokens through an Isotropic Semantic Autoencoder (ISAE) into a 96-dimensional Gaussian-like latent, and training an asymmetric pair of diffusion transformers in which an Anchor DiT predicts future semantic states while an Action DiT reads them through masked attention. At deployment the Anchor DiT is removed and a static semantic cache conditions action generation. On LIBERO, LIBERO-Plu

What carries the argument

Predictive Semantic Alignment via a JEPA-as-World-Anchor: a LoRA-fine-tuned V-JEPA encoder turns observations into abstract semantic tokens that capture physical change without reconstruction. The Isotropic Semantic Autoencoder (ISAE) bridges the modality gap by reshaping those non-Gaussian tokens into a 96-dimensional isotropic Gaussian manifold (reconstruction + KL + sliced-isotropy regularization), preventing off-manifold drift in flow matching. The Asymmetric Mixture-of-Transformers couples an Anchor Diffusion Transformer, masked so it cannot see actions and trained to predict future semantics, with an Action Diffusion Transformer that attends to both semantic and action tokens; the heav

Load-bearing premise

The load-bearing premise is that the Anchor DiT's future-semantic training signal is actually active in the reported model; the main text sets its weight to zero (λf = 0) while an appendix reports a different nonzero configuration, so if the final model never optimized future prediction, the claimed privileged-dynamics mechanism cannot be the source of the reported gains.

What would settle it

Retrain the same architecture with the future-semantic loss weight λf explicitly positive and exactly zero, using identical data and seeds, and compare success rates on LIBERO and RoboTwin 2.0; if the two policies are statistically indistinguishable, the Anchor DiT's future prediction is not the active ingredient. A stronger test: remove the Anchor DiT entirely, keep only the ISAE semantic cache, and check whether the Action DiT still reaches ~97% on LIBERO.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Training a world action model no longer requires a video decoder or pixel-reconstruction loss; the dynamics signal can come entirely from latent semantic alignment.
  • The deployed policy inherits world-model guidance without running the dynamics branch: LeapBot-WA reports 546 ms inference on 2.9B parameters, about 90% faster than high-capacity generative WAMs.
  • Because the latent features are appearance-invariant, the policy transfers zero-shot to camera, lighting, background, and layout perturbations; LIBERO-Plus average success rises to 73.1%, the best reported among predictive WAMs.
  • Without large-scale robot trajectory pretraining, the model reaches 91.76% average success on RoboTwin 2.0 and 97.3% on LIBERO, suggesting representation quality matters more than data volume for this class of policies.
  • The static-semantic-cache inference scheme gives the policy a closed-loop, receding-horizon controller with stable conditioning, enabling real-world sim-to-real transfer.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own appendix sets the future-prediction loss weight to zero in the flagship configuration (λf = 0), while other training sections report nonzero λjepa and an ablation uses λ = 0.2; this discrepancy leaves open whether the Anchor DiT's future-semantic prediction is the active ingredient or whether the ISAE bottleneck plus a static semantic cache accounts for the gains. A clean experimen
  • If the mechanism survives that check, the world-anchor recipe is modular: any latent foundation model that produces structured, non-Gaussian features could be plugged into the same ISAE + asymmetric-distillation pipeline, extending the idea to audio, tactile, or multimodal scene understanding.
  • The 96-dimensional isotropic latent may be doing more work than the predictive objective; one testable extension is whether a fixed random projection of JEPA features, with the Anchor DiT removed, already gives most of the robustness benefit, which would point to geometry rather than prediction as the core enabler.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes LeapBot-WA, a latent-space world action model that replaces pixel-level video prediction with predictive semantic alignment in a V-JEPA latent space. An ISAE maps JEPA features to a 96-dimensional isotropic latent; during training an Anchor DiT is intended to predict future semantic latents and asymmetrically condition an Action DiT; at inference the anchor branch is pruned and a static semantic cache is used. The paper reports state-of-the-art predictive-WAM results on LIBERO and competitive results on RoboTwin 2.0, with zero-shot LIBERO-Plus robustness and real-world transfer.

Significance. If the reported numbers were internally consistent, the contribution would be significant: it provides a concrete architecture for latent-only WAMs and evidence that semantic prediction plus an isotropic bottleneck can match generative WAMs at lower inference cost. The paper attacks a real bottleneck (pixel reconstruction) and offers a plausible system design, including a public code link, a three-stage training recipe, and comparisons against many recent VLAs/WAMs. However, the load-bearing evidence is currently compromised by contradictory statements of the training objective and by mutually inconsistent result tables. The central claim cannot be evaluated until a single, reproducible configuration is specified and the headline numbers reconciled.

major comments (4)
  1. [Eq. (12); Appendix 'Training Objectives', 'Training and Inference', 'Stage III'] The future-prediction objective, which is the core of the claimed 'Predictive-Latent' mechanism, is never specified consistently. Main-text Eq. (12) defines Ltrain = λ_a L_action + λ_s L_semantic + λ_f L_future. The appendix 'Training Objectives' then states for main experiments λ_a=1.0, λ_s=1.0, λ_f=0 (also Table 7 lists λ_f=0). The later 'Training and Inference' appendix gives L = L_action + λ_jepa L_jepa with λ_jepa=0.1, while 'Stage III' gives λ_act=0.1 and λ_jepa=1.0. Table 9's ablation uses λ=0.2. These are mutually exclusive. If the headline model was trained with λ_f=0, the 'privileged dynamics expert' did not receive any future-semantic supervision, so the paper's central mechanism is not the source of the reported gains; if it was trained with λ=0.2, the printed objective is wrong. The authors must state one objective, give the exact weights used for every table, and release th
  2. [Table 2 vs. Tables 4 and 9] The headline LIBERO result and the full ablation are irreconcilable. Table 2 reports LeapBot-WA on LIBERO as 96.2/99.6/97.6/95.6, average 97.3. Table 4's full configuration reports 94.6/99.0/89.2/85.0, average 92.0, and Table 9's final row reports exactly the same four numbers. Thus the same method is credited with two different results on the same suites, one 5.3 points higher. Either the table labels are wrong, the protocols differ (not stated), or the numbers are not reproducible. Since Table 2 is the basis for the 'state-of-the-art among predictive models' claim, this must be resolved.
  3. [Experiment Results and Table 8] The RoboTwin results in the text contradict the table. The 'Experiment Results' section claims 92.64% clean and 89.80% randomized average success, and a 'marginal performance gap (only 2.84%)'. Table 8's averages are 91.04 clean and 92.48 randomized, while Table 1 reports 91.04/92.48/91.76. The clean/randomized text numbers are swapped and inconsistent, and the claimed 2.84% gap does not exist in Table 8 (randomized is higher by 1.44 points). The text also says LeapBot-WA achieves 'perfect 100% success rate' in Open Laptop, Pick Dual Bottles and Place Shoe, but Table 8 lists 99/97, 97/98, and 97/100 for those tasks. The paper's robustness claim is overstated on the printed record.
  4. [Table 9, row 'Semantic Context ✓, Future Prediction 0'] The ablation's baseline rows are not sufficiently specified. The second row uses frozen V-JEPA features as a semantic stream, while the first row lacks this stream; but the method description always encodes observations through the Predictive Anchor, so it is unclear what architecture the first row actually trains (raw pixels? a different encoder?). This prevents the reader from attributing the 47.7→90.7 jump to the semantic stream rather than to a mismatched baseline. A precise definition of each ablation configuration is needed.
minor comments (5)
  1. [Throughout] The method is called 'Mixture-of-Transformers (MoT)' in Section 3 but 'Mixture-of-Tokens (MoT)' in the contributions list; the appendix also inconsistently uses 'LeapWAM' instead of 'LeapBot-WA'.
  2. [Eq. (4), Eq. (7), Table 7] Hyperparameters λ_cos, β_max, and λ_iso are never given numerical values; only the SIGReg projection count (1024) is reported. Without these, the ISAE objective cannot be reproduced.
  3. [Eq. (12) and Appendix Eq. (34)] Lfuture is used in the main text but only defined in the appendix. Please define it in the main text or explicitly defer the definition.
  4. [Appendix 'Detailed Formulation'] The appendix contains incomplete references ('used in Sec.') and uses undefined notation z_{future} in Eq. (34); clarify how the future target is obtained from the dataset.
  5. [Table 8] The table shows only 5 of 50 tasks plus an ellipsis, so the reported averages cannot be verified from the printed record. Please include the full per-task table or a link to a public leaderboard.

Circularity Check

0 steps flagged

No circular derivation: LeapBot-WA's claims are externally benchmarked; the lambda inconsistencies are correctness/evidence issues, not circularity.

full rationale

The paper's central equations define an ISAE bottleneck (Eqs. 2-7), an asymmetric masked-attention MoT (Eqs. 9-11), and flow-matching losses (Eqs. 12, 30-34). None of these defines its target in terms of a fitted output: Lfuture is a supervised objective against future V-JEPA features, Laction is a standard velocity loss, and the reported gains are compared with external baselines on LIBERO, LIBERO-Plus, and RoboTwin. There is no fitted parameter that is then reported as a prediction. The one same-group citation (AttenA+) is only a baseline table entry, not load-bearing. The appendix's internal inconsistency about whether lambda_f/lambda_jepa is 0, 0.1, 0.2, or 1.0, and the Table 2/Table 4/Table 9 numerical mismatches, genuinely undermine the causal attribution of the results to the future-prediction mechanism; however, that is a missing-support/consistency problem rather than a derivational circularity, so it does not raise the circularity score.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 2 invented entities

The method rests on several hand-chosen hyperparameters and two invented network components. Their independent validity is not established outside the paper's own pipeline: the isotropy claim is not directly verified, the future-prediction objective is inconsistently specified, and the real-world transfer evidence is qualitative.

free parameters (5)
  • ISAE latent dimension = 96
    Chosen bottleneck size for diffusion compatibility; no sensitivity study is reported in the main text.
  • loss weights lambda_a, lambda_s, lambda_f / lambda_jepa = 1.0, 1.0, 0 (main); 0.1/1.0 (appendix)
    Conflicting values; these determine whether the future-prediction branch is active and therefore whether the central mechanism was actually trained.
  • KL weight beta_max and SIGReg weight lambda_iso = not specified
    beta is warmed to beta_max but beta_max is not given; lambda_iso never appears numerically. Both directly control the claimed isotropy.
  • noise scheduler shift = 5.0
    Global shift parameter for both action and semantic flow-matching schedulers.
  • SIGReg projection count = 1024
    Number of random 1D projections used in the sliced isotropy regularizer; no ablation is shown.
axioms (4)
  • domain assumption V-JEPA 2.1 features, after LoRA fine-tuning on robot videos, capture action-relevant physical dynamics.
    Stage I relies on this transfer; no independent measurement of dynamics fidelity is provided beyond downstream task success.
  • domain assumption Diffusion models require an isotropic Gaussian latent space to avoid off-manifold drift.
    Motivates ISAE, but the paper provides no direct isotropy metric or off-manifold diagnostic.
  • domain assumption SIGReg enforces aggregate isotropy as intended.
    Adopted from Kolouri et al. 2018; no diagnostic shows the posterior is actually isotropic.
  • domain assumption Point estimates on RoboTwin/LIBERO plus qualitative UR5 runs are sufficient to establish real-world transfer.
    The real-world section reports no success counts, no baselines, and no confidence intervals.
invented entities (2)
  • ISAE latent space (96-d isotropic semantic manifold) no independent evidence
    purpose: Reshape non-Gaussian JEPA features into a diffusion-friendly Gaussian latent space.
    No external or falsifiable handle; only internal ablations and downstream scores, with no isotropy diagnostics reported.
  • Semantic cache z0 no independent evidence
    purpose: Compute semantic tokens once from the current observation and reuse them across all denoising steps at inference.
    An algorithmic artifact; its adequacy for long action horizons is not measured beyond aggregate success rates.

pith-pipeline@v1.3.0-alltime-deepseek · 19398 in / 17367 out tokens · 161957 ms · 2026-07-31T23:23:05.434130+00:00 · methodology

0 comments
read the original abstract

World Action Models (WAMs) have emerged as a powerful paradigm for embodied intelligence, yet the prevailing reliance on pixel-level video generation creates a fundamental bottleneck. Forcing models to reconstruct task-irrelevant visual details dissipates representational capacity and renders policies vulnerable to visual distractors. In this paper, we propose LeapBot-WA, which establishes a novel Predictive-Latent paradigm for WAMs by operationalizing the Joint-Embedding Predictive Architecture (JEPA) as a World-Anchor. Departing from the traditional reliance on visual synthesis, LeapBot-WA shifts the core of world modeling to Predictive Semantic Alignment, extracting abstract physical dynamics directly within a latent foundation space. To bridge the modality gap between non-Gaussian predictive features and diffusion priors, we introduce the Isotropic Semantic Autoencoder (ISAE), which reshapes the anchor's latent space into a diffusion-friendly manifold to prevent off-manifold drift. Furthermore, we design an Asymmetric Mixture-of-Transformers (MoT) architecture. During training, an Anchor Diffusion Transformer acts as a privileged dynamics expert to guide the Action Diffusion Transformer; at inference, this heavy dynamics branch is pruned, enabling zero-overhead execution. LeapBot-WA achieves state-of-the-art performance among predictive models on LIBERO and matches top-tier generative WAMs on RoboTwin 2.0 without requiring large-scale trajectory pre-training. It further demonstrates superior zero-shot robustness to unseen environments and successful real-world transfer, establishing a highly efficient and robust latent-centric paradigm for scalable robotic control. Code: https://github.com/LeapWM/leapbot-wa.

Figures

Figures reproduced from arXiv: 2607.23969 by Daojie Peng, Feilong Kong, Jiachao Liu, Jun Ma, Lang Zhang, Mingyue Feng, Nan Zheng, Pei Liu, Qifeng Chen, Yanan Zhang, Yaonong Wang.

Figure 1
Figure 1. Figure 1: Overview of LeapBot-WA. Visual observations are processed by a Predictive Anchor (fine-tuned JEPA) and com￾pressed by an ISAE into a diffusion-friendly latent space. During joint flow-matching, an Asymmetric MoT decouples world modeling from policy execution. The Anchor DiT predicts semantic evolution and distills these dynamics into the Action DiT via asymmetric masked attention. Crucially, this heavy dyn… view at source ↗
Figure 2
Figure 2. Figure 2: Architecture of the ISAE. The semantic autoen [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Asymmetric MoT denoising. The Anchor DiT is prevented from attending to action tokens, while the Ac￾tion DiT attends to both action and semantic representations through asymmetric masked attention. within the diffusion-friendly ISAE latent space. The primary objective of this branch is to anticipate the future seman￾tic states of the world. Bypassing the representation bottle￾neck of pixel-level video gene… view at source ↗
Figure 5
Figure 5. Figure 5: Real-world robot experiments. The UR5 robot [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 4
Figure 4. Figure 4 [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Overview of the LeapBot-WA training and inference pipeline. Left (Training): An asymmetric joint denoising process where the Anchor DiT predicts future V-JEPA features to provide K, V for the Action DiT (Q). This auxiliary branch is supervised by Ljepa and exists only during training. Right (Inference): A decoupled conditioning strategy where the current observation is converted into a Semantic Cache z0 to… view at source ↗
Figure 7
Figure 7. Figure 7: The three-stage training paradigm of LeapBot-WA. Stage I involves domain-adaptive semantic pretraining where [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative results of LeapBot-WA on the LIBERO benchmark. We visualize the execution of various manipulation [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative results of LeapBot-WA on the LIBERO-Plus benchmark. This dataset introduces significant visual [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Qualitative execution trajectories of LeapBot-WA on the RoboTwin benchmark. Each row illustrates the temporal [PITH_FULL_IMAGE:figures/full_fig_p018_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

98 extracted references · 57 linked inside Pith

  1. [1]

    arXiv preprint arXiv:1803.10122 , volume=

    World models , author=. arXiv preprint arXiv:1803.10122 , volume=

    Pith/arXiv arXiv

  2. [2]

    2, 2022-06-27 , author=

    A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27 , author=. Open Review , volume=

    2022

  3. [3]

    arXiv preprint arXiv:2301.04104 , year=

    Mastering diverse domains through world models , author=. arXiv preprint arXiv:2301.04104 , year=

    Pith/arXiv arXiv

  4. [4]

    arXiv preprint arXiv:2602.15922 , year=

    World action models are zero-shot policies , author=. arXiv preprint arXiv:2602.15922 , year=

    Pith/arXiv arXiv

  5. [5]

    arXiv preprint arXiv:2510.16732 , year=

    A comprehensive survey on world models for embodied ai , author=. arXiv preprint arXiv:2510.16732 , year=

    Pith/arXiv arXiv

  6. [6]

    arXiv preprint arXiv:2605.12090 , year=

    World Action Models: The Next Frontier in Embodied AI , author=. arXiv preprint arXiv:2605.12090 , year=

    Pith/arXiv arXiv

  7. [7]

    arXiv preprint arXiv:2603.16666 , year=

    Fast-WAM: Do World Action Models Need Test-time Future Imagination? , author=. arXiv preprint arXiv:2603.16666 , year=

    Pith/arXiv arXiv

  8. [8]

    7: A Latent World-Action Model from Egocentric Videos , author=

    Being-H0. 7: A Latent World-Action Model from Egocentric Videos , author=. arXiv preprint arXiv:2605.00078 , year=

    Pith/arXiv arXiv

  9. [9]

    2024 IEEE International Conference on Robotics and Automation (ICRA) , pages=

    Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0 , author=. 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages=. 2024 , organization=

    2024

  10. [10]

    arXiv preprint arXiv:2512.05230 , year=

    Invariance Co-training for Robot Visual Generalization , author=. arXiv preprint arXiv:2512.05230 , year=

    arXiv

  11. [11]

    Neural Processing Letters , volume=

    Reinforcement learning with decoupled state representation for robot manipulations , author=. Neural Processing Letters , volume=. 2024 , publisher=

    2024

  12. [12]

    arXiv preprint arXiv:2402.06665 , year=

    The essential role of causality in foundation world models for embodied ai , author=. arXiv preprint arXiv:2402.06665 , year=

    Pith/arXiv arXiv

  13. [13]

    arXiv preprint arXiv:2603.19312 , year=

    Leworldmodel: Stable end-to-end joint-embedding predictive architecture from pixels , author=. arXiv preprint arXiv:2603.19312 , year=

    Pith/arXiv arXiv

  14. [14]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Self-supervised learning from images with a joint-embedding predictive architecture , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  15. [15]

    V-jepa: Latent video prediction for visual representation learning , author=

  16. [16]

    arXiv preprint arXiv:2602.10098 , year=

    Vla-jepa: Enhancing vision-language-action model with latent world model , author=. arXiv preprint arXiv:2602.10098 , year=

    arXiv

  17. [17]

    2019 IEEE/RSJ international conference on intelligent robots and systems (IROS) , pages=

    Robot learning of shifting objects for grasping in cluttered environments , author=. 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS) , pages=. 2019 , organization=

    2019

  18. [18]

    arXiv preprint arXiv:1910.11215 , year=

    Robonet: Large-scale multi-robot learning , author=. arXiv preprint arXiv:1910.11215 , year=

    Pith/arXiv arXiv 1910

  19. [19]

    2024 IEEE International Conference on Robotics and Automation (ICRA) , pages=

    Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot , author=. 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages=. 2024 , organization=

    2024

  20. [20]

    arXiv preprint arXiv:2311.16098 , year=

    On bringing robots home , author=. arXiv preprint arXiv:2311.16098 , year=

    Pith/arXiv arXiv

  21. [21]

    Conference on Robot Learning , pages=

    Rt-2: Vision-language-action models transfer web knowledge to robotic control , author=. Conference on Robot Learning , pages=. 2023 , organization=

    2023

  22. [22]

    arXiv preprint arXiv:2212.06817 , year=

    Rt-1: Robotics transformer for real-world control at scale , author=. arXiv preprint arXiv:2212.06817 , year=

    Pith/arXiv arXiv

  23. [23]

    arXiv preprint arXiv:2406.09246 , year=

    Openvla: An open-source vision-language-action model , author=. arXiv preprint arXiv:2406.09246 , year=

    Pith/arXiv arXiv

  24. [24]

    arXiv preprint arXiv:2412.03555 , year=

    Paligemma 2: A family of versatile vlms for transfer , author=. arXiv preprint arXiv:2412.03555 , year=

    Pith/arXiv arXiv

  25. [25]

    arXiv preprint arXiv:2501.14818 , year=

    Eagle 2: Building post-training data strategies from scratch for frontier vision-language models , author=. arXiv preprint arXiv:2501.14818 , year=

    Pith/arXiv arXiv

  26. [26]

    arXiv preprint arXiv:2409.12191 , year=

    Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution , author=. arXiv preprint arXiv:2409.12191 , year=

    Pith/arXiv arXiv

  27. [27]

    International Conference on Learning Representations , volume=

    From pixels to tokens: Byte-pair encoding on quantized visual modalities , author=. International Conference on Learning Representations , volume=

  28. [28]

    Advances in Neural Information Processing Systems , volume=

    OpenMMEgo: Enhancing egocentric understanding for LMMs with open weights and data , author=. Advances in Neural Information Processing Systems , volume=

  29. [29]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    VideoOrion: Tokenizing Object Dynamics in Videos , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  30. [30]

    arXiv preprint arXiv:2501.15830 , year=

    Spatialvla: Exploring spatial representations for visual-language-action model , author=. arXiv preprint arXiv:2501.15830 , year=

    Pith/arXiv arXiv

  31. [31]

    arXiv preprint arXiv:2503.14734 , year=

    Gr00t n1: An open foundation model for generalist humanoid robots , author=. arXiv preprint arXiv:2503.14734 , year=

    Pith/arXiv arXiv

  32. [32]

    arXiv preprint arXiv:2507.15597 , year=

    Being-h0: vision-language-action pretraining from large-scale human videos , author=. arXiv preprint arXiv:2507.15597 , year=

    Pith/arXiv arXiv

  33. [33]

    Advances in neural information processing systems , volume=

    Denoising diffusion probabilistic models , author=. Advances in neural information processing systems , volume=

  34. [34]

    arXiv preprint arXiv:2411.19650 , year=

    Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation , author=. arXiv preprint arXiv:2411.19650 , year=

    Pith/arXiv arXiv

  35. [35]

    arXiv preprint arXiv:2508.20072 , year=

    Discrete diffusion vla: Bringing discrete diffusion to action decoding in vision-language-action policies , author=. arXiv preprint arXiv:2508.20072 , year=

    Pith/arXiv arXiv

  36. [36]

    Black, Kevin and Brown, Noah and Darpinian, James and Dhabalia, Karan and Driess, Danny and Esmail, Adnan and Equi, Michael Robert and Finn, Chelsea and Fusai, Niccolo and Galliker, Manuel Y and others , booktitle=. _

  37. [37]

    arXiv preprint arXiv:2502.05855 , year=

    Dexvla: Vision-language model with plug-in diffusion expert for general robot control , author=. arXiv preprint arXiv:2502.05855 , year=

    Pith/arXiv arXiv

  38. [38]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Dexgraspvla: A vision-language-action framework towards general dexterous grasping , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  39. [39]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  40. [40]

    arXiv preprint arXiv:2407.08693 , year=

    Robotic control via embodied chain-of-thought reasoning , author=. arXiv preprint arXiv:2407.08693 , year=

    Pith/arXiv arXiv

  41. [41]

    arXiv preprint arXiv:2505.11917 , year=

    Onetwovla: A unified vision-language-action model with adaptive reasoning , author=. arXiv preprint arXiv:2505.11917 , year=

    arXiv

  42. [42]

    arXiv preprint arXiv:2502.03729 , year=

    Action-free reasoning for policy generalization , author=. arXiv preprint arXiv:2502.03729 , year=

    Pith/arXiv arXiv

  43. [43]

    arXiv preprint arXiv:2510.12276 , year=

    Spatial forcing: Implicit spatial representation alignment for vision-language-action model , author=. arXiv preprint arXiv:2510.12276 , year=

    arXiv

  44. [44]

    Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages=

    Mobile robot manipulation using pure object detection , author=. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages=

  45. [45]

    International conference on machine learning , pages=

    Curl: Contrastive unsupervised representations for reinforcement learning , author=. International conference on machine learning , pages=. 2020 , organization=

    2020

  46. [46]

    Robotics research: volume 1 , pages=

    Using geometry to detect grasp poses in 3d point clouds , author=. Robotics research: volume 1 , pages=. 2017 , publisher=

    2017

  47. [47]

    arXiv preprint arXiv:2508.07917 , year=

    Molmoact: Action reasoning models that can reason in space , author=. arXiv preprint arXiv:2508.07917 , year=

    Pith/arXiv arXiv

  48. [48]

    arXiv preprint arXiv:2601.20540 , year=

    Advancing Open-source World Models , author=. arXiv preprint arXiv:2601.20540 , year=

    Pith/arXiv arXiv

  49. [49]

    arXiv preprint arXiv:2412.14803 , year=

    Video prediction policy: A generalist robot policy with predictive visual representations , author=. arXiv preprint arXiv:2412.14803 , year=

    Pith/arXiv arXiv

  50. [50]

    arXiv preprint arXiv:2512.15692 , year=

    mimic-video: Video-action models for generalizable robot control beyond vlas , author=. arXiv preprint arXiv:2512.15692 , year=

    Pith/arXiv arXiv

  51. [51]

    arXiv preprint arXiv:2507.12898 , year=

    Vidar: Embodied video diffusion model for generalist manipulation , author=. arXiv preprint arXiv:2507.12898 , year=

    Pith/arXiv arXiv

  52. [52]

    arXiv preprint arXiv:2508.05635 , year=

    Genie envisioner: A unified world foundation platform for robotic manipulation , author=. arXiv preprint arXiv:2508.05635 , year=

    Pith/arXiv arXiv

  53. [53]

    arXiv preprint arXiv:2503.00200 , year=

    Unified video action model , author=. arXiv preprint arXiv:2503.00200 , year=

    Pith/arXiv arXiv

  54. [54]

    arXiv preprint arXiv:2504.02792 , year=

    Unified world models: Coupling video and action diffusion for pretraining on large robotic datasets , author=. arXiv preprint arXiv:2504.02792 , year=

    Pith/arXiv arXiv

  55. [55]

    arXiv preprint arXiv:2508.00795 , year=

    Video generators are robot policies , author=. arXiv preprint arXiv:2508.00795 , year=

    Pith/arXiv arXiv

  56. [56]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Motus: A unified latent action world model , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  57. [57]

    Advances in neural information processing systems , volume=

    Videovla: Video generators can be generalizable robot manipulators , author=. Advances in neural information processing systems , volume=

  58. [58]

    arXiv preprint arXiv:2601.16163 , year=

    Cosmos policy: Fine-tuning video models for visuomotor control and planning , author=. arXiv preprint arXiv:2601.16163 , year=

    Pith/arXiv arXiv

  59. [59]

    arXiv preprint arXiv:2601.21998 , year=

    Causal World Modeling for Robot Control , author=. arXiv preprint arXiv:2601.21998 , year=

    Pith/arXiv arXiv

  60. [60]

    Advances in Neural Information Processing Systems , volume=

    Libero: Benchmarking knowledge transfer for lifelong robot learning , author=. Advances in Neural Information Processing Systems , volume=

  61. [61]

    arXiv preprint arXiv:2506.18088 , year=

    Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation , author=. arXiv preprint arXiv:2506.18088 , year=

    Pith/arXiv arXiv

  62. [62]

    Forty-first International Conference on Machine Learning , year=

    Genie: Generative interactive environments , author=. Forty-first International Conference on Machine Learning , year=

  63. [63]

    arXiv preprint arXiv:2311.01455 , year=

    Robogen: Towards unleashing infinite data for automated robot learning via generative simulation , author=. arXiv preprint arXiv:2311.01455 , year=

    Pith/arXiv arXiv

  64. [64]

    arXiv preprint arXiv:2401.09985 , year=

    Worlddreamer: Towards general world models for video generation via predicting masked tokens , author=. arXiv preprint arXiv:2401.09985 , year=

    Pith/arXiv arXiv

  65. [65]

    Conference on Robot Learning , pages=

    Masked world models for visual control , author=. Conference on Robot Learning , pages=. 2023 , organization=

    2023

  66. [66]

    International Conference on Learning Representations , volume=

    Td-mpc2: Scalable, robust world models for continuous control , author=. International Conference on Learning Representations , volume=

  67. [67]

    arXiv preprint arXiv:2404.08471 , year=

    Revisiting feature prediction for learning visual representations from video , author=. arXiv preprint arXiv:2404.08471 , year=

    Pith/arXiv arXiv

  68. [68]

    2024 IEEE International Conference on Robotics and Automation (ICRA) , pages=

    Roboagent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking , author=. 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages=. 2024 , organization=

    2024

  69. [69]

    5: Unified Mobility-and-Manipulation World Action Model , author=

    ABot-M0. 5: Unified Mobility-and-Manipulation World Action Model , author=. arXiv preprint arXiv:2607.00678 , year=

    Pith/arXiv arXiv

  70. [70]

    Advances in Neural Information Processing Systems , volume=

    Where are we in the search for an artificial visual cortex for embodied intelligence? , author=. Advances in Neural Information Processing Systems , volume=

  71. [71]

    arXiv preprint arXiv:2203.12601 , year=

    R3m: A universal visual representation for robot manipulation , author=. arXiv preprint arXiv:2203.12601 , year=

    Pith/arXiv arXiv

  72. [72]

    arXiv preprint arXiv:2512.17909 , year=

    Both Semantics and Reconstruction Matter: Making Representation Encoders Ready for Text-to-Image Generation and Editing , author=. arXiv preprint arXiv:2512.17909 , year=

    arXiv

  73. [73]

    arXiv preprint arXiv:2204.06125 , volume=

    Hierarchical text-conditional image generation with clip latents , author=. arXiv preprint arXiv:2204.06125 , volume=

    Pith/arXiv arXiv

  74. [74]

    Advances in neural information processing systems , volume=

    Elucidating the design space of diffusion-based generative models , author=. Advances in neural information processing systems , volume=

  75. [75]

    Forty-first international conference on machine learning , year=

    Scaling rectified flow transformers for high-resolution image synthesis , author=. Forty-first international conference on machine learning , year=

  76. [76]

    arXiv preprint arXiv:2310.06114 , year=

    Learning interactive real-world simulators , author=. arXiv preprint arXiv:2310.06114 , year=

    Pith/arXiv arXiv

  77. [77]

    arXiv preprint arXiv:2101.02722 , year=

    The Distracting Control Suite--A Challenging Benchmark for Reinforcement Learning from Pixels , author=. arXiv preprint arXiv:2101.02722 , year=

    Pith/arXiv arXiv

  78. [78]

    2021 IEEE International Conference on Robotics and Automation (ICRA) , pages=

    Generalization in reinforcement learning by soft data augmentation , author=. 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages=. 2021 , organization=

    2021

  79. [79]

    International Conference on Learning Representations , volume=

    Latent action pretraining from videos , author=. International Conference on Learning Representations , volume=

  80. [80]

    arXiv preprint arXiv:2505.06111 , year=

    Univla: Learning to act anywhere with task-centric latent actions , author=. arXiv preprint arXiv:2505.06111 , year=

    Pith/arXiv arXiv

Showing first 80 references.