Pith. sign in

REVIEW 4 major objections 5 minor 53 references

FACT: Failure-Aware Causal Training for World-Action Models

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A failure-aware world model that predicts what each action causes turns failed rollouts into training data, lifting real-world task success from 82% to 89% (92% with optional candidate scoring).

desk verdict A solid method paper that turns failed rollouts into consequence supervision; the train/inference action-conditioning gap is real but not fatal. read the letter →

arxiv 2608.10232 v1 pith:Y5MGRJMQ submitted 2026-08-10 cs.RO cs.AIcs.LG

Quanquan Peng , Yutong Liang , Rui Yan , Nicklas Hansen , Xiaolong Wang This is my paper · ORCID
classification cs.ROcs.AIcs.LG
keywords World-ActionModelsRobotFoundationManipulationFailure-awaretrainingFuturepredictionTask-progressvalueCausalattentionmaskBimanual
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FACT aims to establish that failed robot rollouts can be turned into structured supervision for a world model — a model that predicts how the scene evolves under robot actions — instead of being discarded. The key move is causal ordering: the model first produces an action chunk, then predicts the future video and an action-conditioned task-progress value (a score estimating how much of the task would be completed after that action). During training, success demonstrations supervise action, future, and value, while failure rollouts keep only the future and value losses, with the progress target lowered for actions that enter failure. On five real-world bimanual tasks, the paper reports that failure-aware training raises average success from 82% to 89%, and optional best-of-4 candidate scoring (choosing the action with the highest predicted progress) reaches 92%; on held-out variants the corresponding numbers are 67% to 77% and 82%. This matters because prior world-action models train mostly on successful demonstrations and therefore tend to hallucinate successful futures under bad actions, while FACT gives the model a concrete reason to know what wrong behavior leads to.

What carries the argument

The load-bearing mechanism is the teacher-forced action-conditioned causal mask over the token sequence $[P, A, G, V, I]$ in a shared causal diffusion transformer. $P$ is the observation prefix, $A$ is the noisy predicted action, $G$ is the clean ground-truth action, $V$ is the value token, and $I$ is the future-video segment; attention is arranged so that $V$ and $I$ attend to $G$ while $A$ cannot see $G$. This separates what to imitate from what to predict. All three outputs are trained with the same flow-matching denoising objective (a generative loss that learns to map noise to clean targets), and a lightweight action adapter adds robot-specific capacity to the shared video backbone. The value target is normalized progress $p_t = G_t/G_T \in [0,1]$, with a clip-down penalty $\lambda_{\text{fail}}$ applied to windows whose executed action enters failure, so the value head encodes action quality. At inference, Stage 1 denoises an action chunk, that chunk fills the $G$ slot, and Stage 2 denoises value and optionally future video, making best-of-$N$ action scoring a direct use of the learned value.

What would settle it

Run two-stage inference with increasingly perturbed Stage-1 actions while scoring with the learned value: if value scores and success stay flat as task outcomes worsen, the progress value is not truly reading action quality; if they drop monotonically with action noise, the action-conditioned causal training is doing the claimed work.

Watch

Extended reading notes

Core claim

The paper's central claim is that an action-conditioned causal interface turns failure rollouts into valid future targets: the world branch predicts what the executed action actually leads to, while the action branch is protected from imitating failed behavior. Concretely, FACT supervises success data with the full loss $w_a\mathcal{L}_a + w_v\mathcal{L}_v + w_I\mathcal{L}_I$ and failure rollouts with $w_v\mathcal{L}_v + w_I\mathcal{L}_I$, using a teacher-forced clean action token as the condition for value and future-video prediction. The value target is normalized task progress clipped down by a failure penalty, so the value head learns to distinguish good from bad actions. Empirically the failure-aware model reaches 87.5% average success across 50 simulated bimanual tasks, 89% on seen real-world tasks, and 77% on unseen variants, with optional value-based candidate scoring raising the real-world numbers to 92% and 82%. On held-out future-prediction samples, failure-aware training raises failure-rollout PSNR from 19.51 to 25.92 while leaving successful-rollout PSNR nearly unchanged, which the paper reads as reduced success-biased future hallucination.

Load-bearing premise

The value and future branches are trained on the exact actions that were executed, but at inference they are fed the model's own predicted actions, and no training-time correction explicitly closes that gap.

Editorial extensions

If this is right

  • Failure rollouts become reusable supervision: real-world seen-task success rises from 82% to 89%, and unseen variants from 67% to 77%.
  • The progress value is a usable ranking signal only after consequence training: best-of-4 scoring reaches 92% on seen tasks, whereas scoring without failure data stays at 79%.
  • Failure-aware training makes futures under bad actions realistic, raising failure-rollout PSNR from 19.51 to 25.92 while leaving successful futures unchanged, which reduces success-biased hallucination.
  • The benefit scales with failure data: on a three-task simulation subset, success grows monotonically from 32.7% with no failure rollouts to 57.3% when failure data is fully included.
  • Video co-training itself is a strong regularizer: removing it drops real-world seen-task success from 82% to 58%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the value and future branches are trained on clean executed actions but deployed on Stage-1 predicted actions, the reported gains may depend on how well the policy's predicted actions stay on the demonstration action manifold; adding action noise during training would be a direct test and a likely fix.
  • The same action-conditioned consequence interface could support online correction loops that re-label bad rollouts or reinforcement learning from negative experience, using the predicted progress drop as a shaped reward; the paper names this direction but does not implement it.
  • In principle the failure-aware value could be learned in settings without action labels, such as human egocentric video, by first inferring actions and then predicting consequences, though the paper does not evaluate that transfer.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes FACT, a causal World-Action Model that first generates an action chunk and then predicts future video and task-progress value conditioned on the executed action. A teacher-forcing causal mask separates action imitation from world prediction, so failure rollouts can supervise the future and value branches while their action imitation loss is masked. The learned progress value can optionally rank sampled action candidates at inference. Experiments in RoboTwin simulation and five real-world bimanual tasks compare FACT with VLA and WAM baselines, ablate video co-training and the causal mask, measure future-prediction PSNR/SSIM on success and failure rollouts, and report failure-data scaling and value traces.

Significance. If the central claims hold, FACT makes a useful conceptual contribution: it converts failed rollouts into action-conditioned consequence supervision without corrupting imitation, and it provides a value interface with a clear compute/reliability tradeoff. The controlled ablations (video co-training, causal mask, failure masking) support the main design choices, and the held-out future-prediction metrics directly test the hallucination-reduction claim. However, the strongest deployment claims—optional candidate scoring and reduced hallucination under the model's own actions—are conditional on an unverified train/inference consistency assumption, and the success-rate comparisons lack statistical support. The paper would be a solid contribution after these gaps are addressed.

major comments (4)
  1. [Section 3.2, Eq. (4), and Sec. 3.3.] The value and future branches are trained conditioned on the clean ground-truth action slot G, but two-stage inference fills G with Stage-1 predicted actions. No action-noise augmentation, DAgger-style relabeling, or regularization is described to cover this shift, and the Limitations section does not mention it. This is load-bearing because optional candidate scoring (Eq. (10)) and the deployment-oriented hallucination claims rely on value and future predictions at Stage-1 sampled actions. Tables 4 and 8 and Figure 5 condition on executed dataset actions, not on Stage-1 samples, so they do not close the gap. Please provide evidence that predicted actions stay on the training action manifold, or add corrective training; without this, the reported 92% scored success and the reduced-hallucination claim are conditional on an unverified assumption.
  2. [Tables 1-3.] Success-rate tables report point estimates without confidence intervals or significance tests. Each real-world cell averages only 20 trials, and many comparisons (e.g., Ours 82% vs. Ours w/ failure 89% on seen tasks; Ours w/ failure 87.5% vs. Motus 87.8% in simulation) are separated by amounts that could easily arise from sampling noise. Please report per-task confidence intervals or trial-level data and run paired significance tests for the main comparisons and ablations, including the claim that scoring alone does not help (Ours + scoring 79% vs. Ours 82%).
  3. [Eq. (6) and Algorithm 2.] The failure-aware value target depends on the failure-onset indicator 1_fail(t+H), but the paper does not specify how failure onset is determined or what happens when it is 'not available' (Algorithm 2, line 6). Since lambda_fail=1 makes the target zero for every window that reaches the onset, the exact value target depends on this annotation protocol. Please describe the labeling procedure and report sensitivity of the results to it.
  4. [Section 4.3, Table 4.] The hallucination diagnostic compares a success-only checkpoint with a failure-co-trained checkpoint; these differ both in the presence of failure data and in training-data size. To attribute the PSNR/SSIM improvement to failure-aware causal supervision, please add a control trained on the same failure data without the causal mask (or with the action imitation loss unmasked) and report future-prediction metrics for that variant. The policy ablation 'Ours w/ failed-action loss' shows that such a control exists, but it is not measured on the Table 4 metric.
minor comments (5)
  1. [Section 3.1, Eq. (3).] The notation G_T is defined as cumulative progress up to time T, but T is also used as episode length; since p_t = t/T under the uniform-reward convention, please state that G_T > 0 and clarify the episode indexing.
  2. [Section 3.3, Eq. (6), and Appendix H.] With lambda_fail=1, the failure target is clipped to 0 for any window after failure onset, making the value target a hard floor rather than a gradual penalty; Appendix H says value scores are unclipped at inference, so please clarify how negative scores arise if the training target is clipped to [0,1].
  3. [Section 4.2, Table 2.] The ablation 'Ours + scoring' is not marked as optional; please clarify whether it also uses N=4 so the comparison with 'Ours w/ failure + scoring' is at matched compute.
  4. [Section 4.3, Figure 6.] The failure-data scaling plot reports only average success over three tasks; please add per-task results and error bars so the monotonic improvement can be assessed.
  5. [References.] Several cited sources are arXiv preprints dated 2026, including the main baselines; if final versions exist, the references should be updated.

Circularity Check

1 steps flagged · score 2.0 of 10

No significant circularity; the only mild issue is the value-trace diagnostic partly restating the Eq. (6) training target.

  1. fitted input called prediction [Section 4.3, 'Value traces reflect action outcomes' (Figure 8)]
    "Because the value is predicted after conditioning on the executed action, it can decrease at a later timestep when that action leads to a poor outcome."

    Eq. (6) defines the failure-aware value target as v_t = clip(p_{t+H} - lambda_fail * 1_fail(t+H), 0, 1), so a value drop at a failed grasp is built into the regression label by construction. The Figure 8 trace therefore mostly demonstrates that the model fits its own training target rather than independently confirming that failure-aware training teaches action consequences. The paper calls this a diagnostic and separately validates the value head through downstream success gains from optional scoring (Tables 2 and 3), so this is a minor, non-load-bearing circularity.

full rationale

The paper's main contributions—action-before-future conditioning with a teacher-forced clean action slot, masked action loss on failures, and optional value-based candidate scoring—are not circular derivations. The central empirical claims are tested against external baselines (pi0, pi0.5, Motus, Cosmos) on downstream task success, and future-prediction quality is measured on 512 held-out samples; neither reduces to the training objective by construction. Failure-rollout data are collected from the initial policy and then used as consequence supervision, which is self-generated data but not a definitional shortcut. No load-bearing self-citation is present: the pretrained WAN2.2-5B checkpoint is external, and cited prior WAMs are used as baselines rather than to justify the core mechanism. The only mild circularity is the value-trace diagnostic in Figure 8: Eq. (6) defines the value target to be clipped lower when a failure onset has occurred, so a value drop at a missed grasp is largely the model fitting its own label. The paper labels this a diagnostic and independently supports the value head through the downstream success gain of optional scoring, so this does not infect the main claims. The train/inference mismatch on the action-conditioning slot (ground-truth G at training versus Stage-1 predicted actions at deployment) is a robustness/correctness concern, not a circularity: no quantity in that gap is defined in terms of another quantity that is then claimed to predict it.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claims rest mainly on empirical training assumptions rather than on a mathematical derivation. The free parameters are hand-selected training constants, and the main novelty is a design pattern (teacher-forced action conditioning with masked failure losses) rather than a new physical entity, so no invented-entity entries are listed. The most fragile inputs are success/failure labels, the uniform-progress target, and the assumption that failure rollouts from an initial policy stay relevant after co-training.

free parameters (5)
  • lambda_fail = 1
    Hand-chosen penalty in Eq. (6) subtracted from the progress target for failed action-conditioned futures; no sensitivity analysis is reported.
  • loss weights w_a, w_v, w_I = 20, 1, 1
    Hand-chosen in Sec. 3.3 to balance action imitation against value and future-video losses; no ablation is shown.
  • action chunk length H = 48
    Design choice in Sec. 3.3 controlling the action horizon; no sweep is reported.
  • future-video offsets = [0, H/4, H/2, 3H/4, H]
    Hand-selected future supervision times in Sec. 3.3; alternative offsets are not tested.
  • candidate count N = 4
    Chosen from the latency-versus-completion sweep in Fig. 7 as a deployment tradeoff.
assumptions (5)
  • domain assumption Failure rollouts collected from the initial policy remain representative after the policy is co-trained on them.
    Algorithm 2 fixes the rollout policy before training on D_f; once the policy changes, the value and future targets may be stale.
  • domain assumption Normalized task progress p_t = t/T under a uniform reward is a meaningful, cross-episode comparable target.
    Eq. (3) in Sec. 3.1 assumes uniform progress; real tasks have nonuniform difficulty, which may misrank action consequences.
  • domain assumption The world branch trained on clean ground-truth actions G transfers to Stage-1 predicted actions at inference.
    Sec. 3.2 and 3.3 teacher-force G during training but fill G with sampled actions at inference; no action noise or DAgger correction is used.
  • domain assumption Accurate success/failure labels and failure-onset annotations are available for every rollout.
    Sec. 3.3 and Algorithm 2 depend on these labels; real-world failure onset is not always well-defined.
  • standard math The flow-matching denoising objective (Eq. 7) and the pretrained WAN2.2-5B backbone provide a sufficient world prior.
    Standard generative-modeling assumptions; the paper initializes from WAN2.2 and uses flow matching for all predicted modalities.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FACT: Failure-Aware Causal Training for World-Action Models." pith.science (2026). https://pith.science/paper/Y5MGRJMQ

@misc{pith2026260810232,
  author       = {Pith},
  title        = {Pith review of: FACT: Failure-Aware Causal Training for World-Action Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y5MGRJMQ}},
  note         = {Machine review of arXiv:2608.10232}
}
read the original abstract

Recent world-action models (WAMs) show that co-training policies with future prediction can provide physical priors for action generation. Building on the future-prediction ability of video models, many WAMs generate future videos and recover actions with inverse-dynamics models, or use these predicted videos as goal conditions for action generation. In both cases, the world model is trained mostly on successful demonstrations and has little reason to predict the consequences of bad actions. We introduce FACT, a causal World-Action Model that predicts future video and task progress conditioned on the executed action. This action-conditioned interface allows failure rollouts to supervise action consequences, turning bad actions into valid future targets rather than being discarded. Failure-aware training makes the progress predictor aware of both successful and failed action outcomes, which can optionally be used to score sampled action candidates at inference. Extensive experiments on simulation and real-world bimanual manipulation tasks show that FACT outperforms many existing baselines, improves as failure data are incorporated into training, and reduces success-biased future hallucination under bad actions. See more details at https://fact-wam.github.io/

Figures

Figures reproduced from arXiv: 2608.10232 by the authors.

Figure 1
Figure 1. FACT first generates an action and then rolls out the resulting future video and task￾progress value. Because each future unfolds from the action that produced it, failure rollouts directly supervise the future-prediction branch under their own bad actions, teaching the world model what wrong behavior actually leads to. 1 Introduction Building general-purpose robot policies is a central goal in robot learning. Visio… view at source ↗
Figure 2
Figure 2. FACT architecture. (a) A shared causal diffusion transformer denoises action, value, and future-video tokens; value and future video condition on the clean action slot G, not the noisy A. At inference, Stage 1 denoises an action chunk, which fills G for Stage 2 value and future-video prediction. (b) Success demonstrations supervise all three losses; failure rollouts mask Lact but keep value and future-video supervis… view at source ↗
Figure 3
Figure 3. Training and inference masks. Rows and columns denote prefix image (P), predicted action (A), ground-truth action (G), value (V ), and future video (I); cells mark allowed attention, colored by the attended seg￾ment. The dashed block is Stage 1 action denoising; the denoised action then fills G for Stage 2 (Sec. 3.3). where P is the observation prefix, A is the noisy predicted-action seg￾ment, G is the clean teacher… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Real-world task overview. We show intermediate frame results for five tasks. 4 Experiments In this section, we evaluate FACT through benchmark comparisons, controlled variants, and diag￾nostic analyses. We first describe the experimental setup and compare our model wit…
Figure 5
Figure 5. Figure 5: Effect of failure data on future prediction. Under the same bad-action condition, the success-only model hallucinates a successful grasp (marked by a white dotted box), while failure￾aware co-training predicts the observed failure outcome. Handover Block Open Microwave…
Figure 6
Figure 6. Figure 6: Failure-data scaling on RoboTwin. Success rate vs. failure-rollout fraction p. Failure-data scaling. We further probe how the amount of failure data shapes policy learning. On three RoboTwin clean tasks, we train FACT with p ∈ {0%, 50%, 100%} of the available failure r…
Figure 7
Figure 7. Figure 7: Choices of different N. Relative task completion and inference latency as the candidate count N varies on a long-horizon grasping task; N=4 balances the two.                    Reaching Grasp failed [PITH_FULL_IMAGE:figure…
Figure 9
Figure 9. Figure 9: and [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Unseen real-world tasks. Held-out task variants used for [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Real-world hardware setup. As shown in (a), we use two YAM arms, three RGB camera views, and a GELLO teleoperation device. The objects we used in real-world experiments are shown in (b). Real-world Task Prompts Stack Cubes Grasp the red cube and place it at the center…
Figure 12
Figure 12. Figure 12: provides additional value-trace visualizations for the Stack Bowls and Stack Cubes tasks. Together with [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Action-conditioned value heat map. On Stack Cubes, the learned value head scores the successful center placement higher than failed placements around it. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 6 canonical work pages

  1. [1]

    Zitkovich, T

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023

    2023

  2. [2]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al. Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024

    arXiv 2024

  3. [3]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al.π 0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024

    arXiv 2024

  4. [4]

    Black, N

    Physical Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, et al.π 0.5: A vision-language-action model with open-world generalization.arXiv preprint arXiv:2504.16054, 2025

    arXiv 2025

  5. [5]

    L. Li, Q. Zhang, Y . Luo, S. Yang, R. Wang, F. Han, M. Yu, Z. Gao, N. Xue, X. Zhu, et al. Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

    arXiv 2026

  6. [6]

    M. J. Kim, Y . Gao, T.-Y . Lin, Y .-C. Lin, Y . Ge, G. Lam, P. Liang, S. Song, M.-Y . Liu, C. Finn, et al. Cosmos policy: Fine-tuning video models for visuomotor control and planning.arXiv preprint arXiv:2601.16163, 2026

    arXiv 2026

  7. [7]

    H. Bi, H. Tan, S. Xie, Z. Wang, S. Huang, H. Liu, R. Zhao, Y . Feng, C. Xiang, Y . Rong, et al. Motus: A unified latent action world model.arXiv preprint arXiv:2512.13030, 2025

    arXiv 2025

  8. [8]

    T. Yuan, Z. Dong, Y . Liu, and H. Zhao. Fast-wam: Do world action models need test-time future imagination?arXiv preprint arXiv:2603.16666, 2026

    arXiv 2026

Show all 53 references
  1. [9]

    A. Ye, B. Wang, C. Ni, G. Huang, G. Zhao, H. Li, H. Li, J. Li, J. Lv, J. Liu, et al. Gigaworld- policy: An efficient action-centered world–action model.arXiv preprint arXiv:2603.17240, 2026

    2026

  2. [10]

    Y . Hu, J. Zhang, Y . Luo, Y . Guo, X. Chen, X. Sun, K. Feng, Q. Lu, S. Chen, Y . Zhang, et al. Bagelvla: Enhancing long-horizon manipulation via interleaved vision-language-action generation.arXiv preprint arXiv:2602.09849, 2026

    2026

  3. [11]

    S. Ye, Y . Ge, K. Zheng, S. Gao, S. Yu, G. Kurian, S. Indupuru, Y . L. Tan, C. Zhu, J. Xiang, et al. World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026

    2026 arXiv

  4. [12]

    P. Zhou, S. Chen, D. Chen, J. Wang, R. Jin, B. Zhu, Y . Pan, S. Gu, K. Wang, S. Nan, et al.τ 0-wm: A unified video-action world model for robotic manipulation.arXiv preprint arXiv:2606.01027, 2026

    2026 arXiv

  5. [13]

    Z. Bai, P. Wang, T. Xiao, T. He, Z. Han, Z. Zhang, and M. Z. Shou. Hallucination of multimodal large language models: A survey.arXiv preprint arXiv:2404.18930, 2024

    2024 arXiv

  6. [14]

    Rawte, A

    V . Rawte, A. Sheth, and A. Das. A survey of hallucination in large foundation models.arXiv preprint arXiv:2309.05922, 2023

    2023 arXiv

  7. [15]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    2022 arXiv

  8. [16]

    O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, et al. Octo: An open-source generalist robot policy.arXiv preprint arXiv:2405.12213, 2024. 10

    2024 arXiv

  9. [17]

    Zheng, J

    J. Zheng, J. Li, Z. Wang, D. Liu, X. Kang, Y . Feng, Y . Zheng, J. Zou, Y . Chen, J. Zeng, et al. X- vla: Soft-prompted transformer as scalable cross-embodiment vision-language-action model. arXiv preprint arXiv:2510.10274, 2025

    arXiv

  10. [18]

    W. Wu, F. Lu, Y . Wang, S. Yang, S. Liu, F. Wang, Q. Zhu, H. Sun, Y . Wang, S. Ma, et al. A pragmatic vla foundation model.arXiv preprint arXiv:2601.18692, 2026

    2026 arXiv

  11. [19]

    W. Wu, F. Wang, F. Lu, H. Sun, S. Liu, Y . Wang, Y . Yan, Y . Wang, S. Ma, X. Wang, Y . Liu, S. Yang, T. Zhou, K. Zhang, L. Zhou, C. Su, N. Xue, B. Tan, H. Zhang, Y . Zhang, F. Liao, X. Zhu, Y . Shen, and K. Zheng. From foundation to application: Improving vla models in practi...

    2026 arXiv

  12. [20]

    S. Gao, W. Liang, K. Zheng, A. Malik, S. Ye, S. Yu, W.-C. Tseng, Y . Dong, K. Mo, C.-H. Lin, et al. Dreamdojo: A generalist robot world model from large-scale human videos.arXiv preprint arXiv:2602.06949, 2026

    2026 arXiv

  13. [21]

    J. Lyu, K. Liu, X. Zhang, H. Liao, Y . Feng, W. Zhu, T. Shen, J. Chen, J. Zhang, Y . Dong, et al. Lda-1b: Scaling latent dynamics action model via universal embodied data ingestion.arXiv preprint arXiv:2602.12215, 2026

    2026 arXiv

  14. [22]

    Jiang, Y

    G. Jiang, Y . Liang, J. Ye, J.-Y . Huang, C. Jing, R. Duan, P. Abbeel, X. Wang, and X. Zou. Cross-hand latent representation for vision-language-action models.arXiv preprint arXiv:2603.10158, 2026

    2026

  15. [23]

    Liu, A.-C

    I. Liu, A.-C. Cheng, R. Yan, G. Chen, R.-Z. Qiu, X. Zou, S. Yi, H. Yin, X. Wang, and S. Liu. Long-horizon manipulation via trace-conditioned vla planning.arXiv preprint arXiv:2604.21924, 2026

    2026 arXiv

  16. [24]

    Dasari, F

    S. Dasari, F. Ebert, S. Tian, S. Nair, B. Bucher, K. Schmeckpeper, S. Singh, S. Levine, and C. Finn. Robonet: Large-scale multi-robot learning.arXiv preprint arXiv:1910.11215, 2019

    1910 arXiv

  17. [25]

    H. R. Walke, K. Black, T. Z. Zhao, Q. Vuong, C. Zheng, P. Hansen-Estruch, A. W. He, V . My- ers, M. J. Kim, M. Du, et al. Bridgedata v2: A dataset for robot learning at scale. InConference on Robot Learning, pages 1723–1736. PMLR, 2023

    2023

  18. [26]

    O’Neill, A

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 6892–

  19. [27]

    Khazatsky, K

    A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y . Chen, K. Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset.arXiv preprint arXiv:2403.12945, 2024

    2024 arXiv

  20. [28]

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware.arXiv preprint arXiv:2304.13705, 2023

    2023 arXiv

  21. [29]

    Z. Fu, T. Z. Zhao, and C. Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation.arXiv preprint arXiv:2401.02117, 2024

    2024 arXiv

  22. [30]

    C. Chi, Z. Xu, C. Pan, E. Cousineau, B. Burchfiel, S. Feng, R. Tedrake, and S. Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots.arXiv preprint arXiv:2402.10329, 2024

    2024 arXiv

  23. [31]

    M. Xu, H. Zhang, Y . Hou, Z. Xu, L. Fan, M. Veloso, and S. Song. Dexumi: Using hu- man hand as the universal manipulation interface for dexterous manipulation.arXiv preprint arXiv:2505.21864, 2025. 11

    2025

  24. [32]

    Cheng, K

    T. Cheng, K. Chen, L. Chen, L. Zhang, Y . Zhang, Y . Ling, M. Hamad, Z. Bing, F. Wu, K. Sharma, et al. Tacumi: A multi-modal universal manipulation interface for contact-rich tasks.arXiv preprint arXiv:2601.14550, 2026

    2026

  25. [33]

    R. Yang, Q. Yu, Y . Wu, R. Yan, B. Li, A.-C. Cheng, X. Zou, Y . Fang, X. Cheng, R.-Z. Qiu, et al. Egovla: Learning vision-language-action models from egocentric human videos.arXiv preprint arXiv:2507.12440, 2025

    2025 arXiv

  26. [34]

    Zheng, D

    R. Zheng, D. Niu, Y . Xie, J. Wang, M. Xu, Y . Jiang, F. Casta˜neda, F. Hu, Y . L. Tan, L. Fu, et al. Egoscale: Scaling dexterous manipulation with diverse egocentric human data.arXiv preprint arXiv:2602.16710, 2026

    2026

  27. [35]

    R.-Z. Qiu, S. Yang, X. Cheng, C. Chawla, J. Li, T. He, G. Yan, D. J. Yoon, R. Hoque, L. Paulsen, et al. Humanoid policy˜ human policy.arXiv preprint arXiv:2503.13441, 2025

    2025

  28. [36]

    Nasiriany, A

    S. Nasiriany, A. Maddukuri, L. Zhang, A. Parikh, A. Lo, A. Joshi, A. Mandlekar, and Y . Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots.arXiv preprint arXiv:2406.02523, 2024

    2024 arXiv

  29. [37]

    Nasiriany, S

    S. Nasiriany, S. Nasiriany, A. Maddukuri, and Y . Zhu. Robocasa365: A large-scale simulation framework for training and benchmarking generalist robots.arXiv preprint arXiv:2603.04356, 2026

    2026

  30. [38]

    Mandlekar, S

    A. Mandlekar, S. Nasiriany, B. Wen, I. Akinola, Y . Narang, L. Fan, Y . Zhu, and D. Fox. Mimicgen: A data generation system for scalable robot learning using human demonstrations. arXiv preprint arXiv:2310.17596, 2023

    2023 arXiv

  31. [39]

    B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone. Libero: Benchmarking knowl- edge transfer for lifelong robot learning.Advances in Neural Information Processing Systems, 36:44776–44791, 2023

    2023

  32. [40]

    O. Mees, L. Hermann, E. Rosete-Beas, and W. Burgard. Calvin: A benchmark for language- conditioned policy learning for long-horizon robot manipulation tasks.IEEE Robotics and Automation Letters, 7(3):7327–7334, 2022

    2022

  33. [41]

    T. Chen, Z. Chen, B. Chen, Z. Cai, Y . Liu, Z. Li, Q. Liang, X. Lin, Y . Ge, Z. Gu, et al. Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation.arXiv preprint arXiv:2506.18088, 2025

    2025 arXiv

  34. [42]

    H. Li, P. Ding, R. Suo, Y . Wang, Z. Ge, D. Zang, K. Yu, M. Sun, H. Zhang, D. Wang, et al. Vla-rft: Vision-language-action reinforcement fine-tuning with verified rewards in world sim- ulators.arXiv preprint arXiv:2510.00406, 2025

  35. [43]

    J. Lv, H. Li, J. Li, Y . Nie, F. Kong, Y . Wang, X. Wang, Z. Zhu, C. Ni, Q. Deng, et al. Viva: A video-generative value model for robot reinforcement learning.arXiv preprint arXiv:2604.08168, 2026

    2026 arXiv

  36. [44]

    J. Yang, K. Lin, J. Li, W. Zhang, T. Lin, L. Wu, Z. Su, H. Zhao, Y .-Q. Zhang, L. Chen, et al. Rise: Self-improving robot policy with compositional world model.arXiv preprint arXiv:2602.11075, 2026

    2026 arXiv

  37. [45]

    Z. Liu, A. Bahety, and S. Song. Reflect: Summarizing robot experiences for failure explanation and correction.arXiv preprint arXiv:2306.15724, 2023

    2023 arXiv

  38. [46]

    D. H. Grollman and A. G. Billard. Robot learning from failed demonstrations.International Journal of Social Robotics, 4(4):331–342, 2012

    2012

  39. [47]

    X. Wang, S. Cheng, S. Wang, X. Zhang, D. Zhang, and W. Zhang. Learning from demon- stration with failure awareness for safe robot navigation.arXiv preprint arXiv:2604.23360, 2026. 12

    2026 arXiv

  40. [48]

    B. Li, M. Mishra, Y . I. Liu, S. Tao, N. Kumar, A. G. Gray, R. Wickramarachchi, J. Francis, S. Scherer, and T. Silver. Recover, discover, plan: Learning skills and concepts from robot failures.arXiv preprint arXiv:2606.18328, 2026

    2026 arXiv

  41. [49]

    Lipman, R

    Y . Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling.arXiv preprint arXiv:2210.02747, 2022

    2022 arXiv

  42. [50]

    H. Qi, H. Yin, A. Zhu, Y . Du, and H. Yang. Inference-time enhancement of generative robot policies via predictive world modeling.IEEE Robotics and Automation Letters, 2026

    2026

  43. [51]

    T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C.-W. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    2025 arXiv

  44. [52]

    Loshchilov and F

    I. Loshchilov and F. Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  45. [53]

    P. Wu, Y . Shentu, Z. Yi, X. Lin, and P. Abbeel. Gello: A general, low-cost, and intuitive teleoperation framework for robot manipulators. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 12156–12163. IEEE, 2024. 13 Appendix A Training a...

    2024

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.