Pith. sign in

REVIEW 4 major objections 6 minor 23 cited by

Diffusion-VLA: Generalizable and Interpretable Robot Foundation Model via Self-Generated Reasoning

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A vision-language-action model that generates its own reasoning and injects it into a diffusion action decoder outperforms prior robot policies and becomes interpretable.

desk verdict Plausible architecture, promising real-robot results, but the central ablation doesn't isolate the reasoning-injection mechanism; deserves a referee who will demand a cleaner experiment. read the letter →

arxiv 2412.03293 v3 pith:SILTR7TU submitted 2024-12-04 cs.RO cs.CV

Junjie Wen , Minjie Zhu , Yichen Zhu , Zhibin Tang , Jinming Li , Zhongyi Zhou , Chengmeng Li , Xiaoyu Liu
show 3 more authors
Yaxin Peng Chaomin Shen Feifei Feng
This is my paper · ORCID
classification cs.ROcs.CV
keywords vision-language-actionmodelrobotfoundationdiffusionpolicyautoregressivereasoninginjectionreal-robotmanipulationgeneralizationinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Diffusion-VLA (DiVLA) is a robot policy that lets a vision-language model first write a short reasoning phrase about the scene and task, then feeds that phrase into a diffusion-based action generator that outputs continuous robot commands. The paper's central claim is that this two-part design, with the reasoning embedded directly into the policy, gives better real-world manipulation performance than both pure next-token vision-language-action models and pure diffusion policies, while making the robot's decisions readable. On a five-task multi-task benchmark on a single-arm robot, the 2B model averages 83.6% success versus 45.5% for the best baseline, and it picks 63.7% of 102 never-seen objects in a zero-shot bin-picking test versus 28.4% for the strongest baseline. The authors also report robustness to distractors, backgrounds, and camera shifts, fast adaptation to a bimanual robot, and 82 Hz inference on a single GPU. If these results hold under controlled comparison, they point to a practical recipe for combining language reasoning with high-frequency continuous action generation in robot foundation models.

What carries the argument

The load-bearing mechanism is the reasoning injection module. In a forward pass, the vision-language model emits a short sequence of reasoning tokens; the final embedding from that reasoning output is used as a Feature-wise Linear Modulation (FiLM) conditioning vector, scaling and shifting the activations of the diffusion policy's layers so that the explicit rationale directly modulates action generation. FiLM is a conditioning layer that applies an affine transformation to features based on a conditioning vector. Because the reasoning is generated inside the same pass, DiVLA avoids the recursive generate-then-feed-back loop used by chain-of-thought policies. The diffusion head follows the standard Diffusion Policy architecture, with a two-layer MLP projection aligning vision-language embeddings to the action decoder, and training combines a diffusion loss with a next-token-prediction loss at weight ratio alpha=10; the vision-language backbone is fine-tuned with LoRA while the vision encoder and base weights are frozen. For pretraining, an external language model rewrites raw robot trajectories into reasoning-augmented form.

What would settle it

Run the same real-robot tasks with all policies pretrained on the same data and matched for camera views, training epochs, and fine-tuning budget, and run an ablation that varies only the reasoning text while keeping the architecture fixed; if a diffusion policy without injected reasoning matches DiVLA on unseen-object bin picking, or if random reasoning phrases produce the same success as task-relevant ones, the paper's central claim is refuted.

Watch

Extended reading notes

Core claim

The authors' claim is that autoregressive reasoning and diffusion action generation solve each other's weaknesses: the vision-language head decomposes the user's query into a short rationale, and the diffusion head generates continuous joint-space commands, while a FiLM-based reasoning injection module makes the rationale directly modulate the policy. They report that this design reaches 83.6% average success on five real-robot multi-task skills, 66.2% average on a four-category factory-sorting task with seen and unseen objects, 63.7% zero-shot success on 102 unseen bin-picking objects, and 72.9% and 70.8% on bimanual table bussing for seen and mixed objects. Removing the reasoning injection drops the multi-task average from 83.6% to 50.3%. The same model retains visual-question-answering ability, follows multi-step instructions, and runs at 82 Hz for the smallest variant.

Load-bearing premise

The load-bearing premise is that the comparisons are fair: in the paper's Section 4.1, DiVLA is pretrained on a 39K-trajectory dataset while Octo and OpenVLA are pretrained on a 970K-trajectory dataset, and the zero-shot bin-picking evaluation in Figure 4 does not control for that difference.

Editorial extensions

If this is right

  • A policy can be both conversational and control a robot: DiVLA retains visual-question-answering ability even without co-training on vision-language chat data.
  • Generalization to new objects can arise by analogy: DiVLA labels a screwdriver as a hex key and a Sprite can as a green can, then sorts them correctly.
  • Adding a new robot embodiment reduces to training a new MLP action head, so pretrained knowledge is reused rather than rebuilt.
  • Scaling the same recipe from 2B to 72B improves sorting from 66.2% to 82.4% and bin picking from 63.7% to 75.9%.
  • The reasoning injection itself is essential: removing it lowers the five-task average from 83.6% to 50.3%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The FiLM injection path is not tied to diffusion: the paper does not test it on flow-matching or one-step distillation action heads, which could trade some reported robustness for higher control frequency.
  • A causal test the paper does not run is to sample random reasoning phrases at inference and check whether actions change; the mid-task hex-key swap suggests, but does not prove, that the rationale drives the action.
  • If reasoning quality is the active ingredient, then longer or multi-step rationales generated at inference could further improve novel-object and multi-step instruction following, since the architecture already supports autoregressive generation of arbitrary text.
  • The pretraining-data mismatch is the main threat to interpreting the reported gains as architectural; a matched-data replication might shrink the raw gaps while still leaving interpretability and speed as separable advantages.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Diffusion-VLA (DiVLA), a vision-language-action model that combines a pretrained autoregressive VLM (Qwen2-VL) with a diffusion policy head for action generation. The model is trained with a diffusion loss plus a next-token prediction loss so that, given an instruction and observation, it first emits free-form reasoning tokens and then generates actions. A reasoning injection module uses FiLM to condition the diffusion policy on the embedding of the generated reasoning phrases. The authors evaluate DiVLA on real robots across multi-task learning, factory sorting, zero-shot bin picking on 102 unseen objects, bimanual table bussing, novel instruction following, and model scaling from 2B to 72B parameters. The central claims are that self-generated reasoning improves generalization and interpretability, that the reasoning injection module is responsible for a 33.3-point performance gap, and that DiVLA outperforms OpenVLA, Octo, TinyVLA, and Diffusion Policy on the reported tasks while running at 82 Hz on an A6000 GPU.

Significance. If the empirical claims hold, the paper makes a useful architectural contribution: it shows a concrete way to give diffusion policies a chain-of-thought-like reasoning channel without requiring iterative re-prompting, and it provides real-robot evidence for data-efficient training and fast inference. The strengths include genuine real-robot evaluations across multiple embodiments, a large zero-shot bin-picking test set, explicit scaling experiments from 2B to 72B, and an honest discussion of limitations such as sensitivity to low-bit quantization and imperfect VQA behavior. The interpretability examples in Section 4.4 are visually compelling. However, the causal contribution of the named reasoning-injection mechanism is not isolated by the reported ablation, and the headline comparisons with OpenVLA and Octo are confounded by differences in pretraining data, model size, and backbone. The very small trial counts in several evaluations further weaken the quantitative claims. The paper is promising, but the central causal claim and the comparative superiority claims need additional experiments or re-analysis before they can be accepted as stated.

major comments (4)
  1. [Section 8.4, Table 8] The ablation labeled 'w/o reasoning injection' does not specify which components are removed. If removing the injection module also removes the autoregressive reasoning tokens and the alpha*L_ntp term, then the reported 33.3-point drop conflates (i) the FiLM-based conditioning, (ii) the extra language-supervision objective, and (iii) the presence of reasoning text as a training target. To support the paper's core claim that injecting self-generated reasoning into the policy is what drives the improvement, the authors should report at least two additional conditions: one that keeps the reasoning tokens and L_ntp loss but removes only the FiLM conditioning, and one that keeps FiLM conditioning but removes the reasoning content (e.g., injecting a generic text embedding). As written, the central causal claim is underdetermined by the reported experiment.
  2. [Section 4.2, Table 1, and Section 4.5, Figure 4] The main comparisons against OpenVLA and Octo do not control for pretraining data scale or model size. DiVLA-2B is pretrained on Droid (39K trajectories), while OpenVLA and Octo are pretrained on OXE (970K trajectories), and DiVLA-2B is 2B parameters while OpenVLA is 7B. The paper acknowledges this asymmetry (Section 4.2: 'OXE ... is 25 times larger than our pre-trained datasets') but still interprets the performance gaps, including the 63.7% versus 28.4% bin-picking result, as evidence of architectural superiority. The gaps could be due to the different pretraining data, the different backbones, or the fine-tuning protocol rather than the proposed reasoning mechanism. At minimum, the authors should provide an ablation in which DiVLA without the reasoning injection is trained and evaluated under the same protocol, and ideally a comparison in which OpenVLA or Octo is fine-tuned on the same data budget or DiVLA is pretrained on the same OXE data.
  3. [Section 6.1, Table 4] Several critical evaluations use very small trial counts. For the visual generalization setting, each task has only 9 trials, so a difference of one success changes the reported rate by 11.1 percentage points. For instance, in Task 1 of the visual generalization rows, DiVLA-2B succeeds 4/9 and OpenVLA succeeds 5/9, yet the text in Section 4.2 states that 'our method consistently maintains the highest average success rate' across tasks. No confidence intervals or significance tests are reported anywhere. The main multi-task per-task counts (11 trials for most tasks) also make the differences between DiVLA and baselines statistically fragile. The authors should provide confidence intervals or exact binomial tests, and ideally increase the trial counts for the claims that are central to the paper.
  4. [Section 4.4 and Figure 6] The interpretability and self-correction claims rely on a few selected examples rather than a systematic evaluation. The claim that 'the integrated reasoning module allows for a form of self-correction' is based on a single illustrative intervention (replacing a toy car with a hex key) with no quantification of how often the reasoning changes correctly, how often the action follows the reasoning, or whether a model without the reasoning injection would exhibit different behavior under the same intervention. To substantiate the interpretability claim, the authors should provide a protocol that measures the agreement between the generated reasoning and the executed action over multiple trials, and compare against a baseline that produces plausible-looking rationales after the fact.
minor comments (6)
  1. [Throughout] The paper uses both 'DiffusionVLA' and 'DiVLA' for the same method; please standardize the name.
  2. [Section 3.2, Pretraining Data] The sentence 'The original Droid data contains only robotic actions, paired partially with observations and language instructions' is repeated verbatim in consecutive sentences; one copy should be removed.
  3. [Section 4.1] The heading contains a typo: 'Implementation details and pretraiend data' should be 'pretrained data'.
  4. [Table 4] The column header 'Trails' should be 'Trials'.
  5. [Section 4.2] The sentence 'Notice that both Octo and OpenVLA is pre-trained on OXE' has a subject-verb agreement error; it should be 'are pre-trained'.
  6. [Section 8.2, Table 5] The inference-speed comparison omits hardware details for OpenVLA beyond 'A6000 GPU'; please specify whether the same batching, sequence length, and precision settings were used for all methods.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity by construction: the central success metrics are external robot task outcomes, and the reasoning-injection claim is tested empirically rather than defined into existence.

full rationale

Walking the paper's claimed derivation chain, I find no step in which a prediction or first-principles result reduces to its own inputs by construction. The central claims are empirical: task success is scored externally (e.g., "The task is considered successful only if the robot successfully grasps the object and places it in the correct sector"), and the reasoning-injection contribution is tested by an ablation (Table 8) rather than derived from the definition of the module. The 'self-generated reasoning' loop—the VLM generates reasoning tokens that are then injected via FiLM—is the proposed mechanism, not a hidden identity; the action targets (sorting, bin-picking, bussing) are independent of the reasoning text. The paper does cite the authors' own TinyVLA as a baseline and their scaling work, but these citations are not load-bearing: the comparisons are externally reproducible robot trials, and no uniqueness theorem or unverified premise is imported from those self-citations. The main caveats are experimental-design confounds (DiVLA-2B pretrains on Droid 39K while Octo/OpenVLA use OXE 970K; the Table 8 ablation does not state whether the 'w/o reasoning injection' model retains the L_ntp objective), but these are fairness and control concerns, not circularity. No equation in the paper is equivalent to another by construction, no fitted parameter is relabeled as a prediction, and the Appendix VQA section even flags a confound ("the pre-trained VLM may have encountered similar objects"), which further indicates the reasoning text is not being used to define the target. Therefore the derivation chain is self-contained with respect to circularity, and the appropriate score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 1 invented entities

The framework rests on standard deep-learning practice: a frozen SigLIP encoder and Qwen2-VL backbone, a LoRA fine-tune, and a Diffusion Policy decoder. Hand-chosen hyperparameters include the loss weighting alpha=10 and learning rate 2e-5. The main invented component is the reasoning injection module, whose utility is supported only by an in-paper ablation. The reasoning training signal is machine-generated by GPT-4o and never independently validated, which is a domain assumption rather than a free parameter.

free parameters (3)
  • alpha (loss weight for NTP term) = 10
    Chosen by hand in Section 3.2 to balance diffusion and next-token losses; the paper notes L_ntp is about ten times smaller than L_diff in magnitude.
  • Learning rate = 2e-5
    Fixed learning rate used for fine-tuning, stated in Section 4.1.
  • Number of action tokens
    The architecture uses a fixed number of action tokens but the number is not specified; likely a design choice that affects the projection layer and inference speed.
assumptions (3)
  • domain assumption Pre-trained VLM (Qwen2-VL) provides a useful visual-language representation for robot control.
    Used as the frozen backbone; if the VLM's representations are not suited to the control tasks, the model would not work. Invoked in Section 3.1.
  • domain assumption Diffusion policy can faithfully generate multi-modal robot actions from the VLM's action-token embeddings.
    The action decoder follows Diffusion Policy design; this is a standard assumption in the field but not proven in the paper.
  • domain assumption GPT-4o generated reasoning phrases are a useful training signal for the policy.
    The pretraining data is annotated with GPT-4o generated reasoning; if these annotations are poor, the reasoning injection could hurt. Stated in Section 3.2.
invented entities (1)
  • Reasoning injection module (FiLM-based)
    purpose: Embeds the final reasoning embedding into the diffusion action decoder to condition actions on self-generated rationales.
    The module is a new architectural component introduced in Section 3.1; its effectiveness is only supported by an in-paper ablation (Table 8), not by an external falsifiable prediction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Diffusion-VLA: Generalizable and Interpretable Robot Foundation Model via Self-Generated Reasoning." pith.science (2026). https://pith.science/paper/SILTR7TU

@misc{pith2026241203293,
  author       = {Pith},
  title        = {Pith review of: Diffusion-VLA: Generalizable and Interpretable Robot Foundation Model via Self-Generated Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SILTR7TU}},
  note         = {Machine review of arXiv:2412.03293}
}
read the original abstract

In this paper, we present DiffusionVLA, a novel framework that seamlessly combines the autoregression model with the diffusion model for learning visuomotor policy. Central to our approach is a next-token prediction objective, enabling the model to reason effectively over the user's query in the context of current observations. Subsequently, a diffusion model is attached to generate robust action outputs. To enhance policy learning through self-reasoning, we introduce a novel reasoning injection module that integrates reasoning phrases directly into the policy learning process. The whole framework is simple and flexible, making it easy to deploy and upgrade. We conduct extensive experiments using multiple real robots to validate the effectiveness of DiffusionVLA. Our tests include a challenging factory sorting task, where DiffusionVLA successfully categorizes objects, including those not seen during training. We observe that the reasoning module makes the model interpretable. It allows observers to understand the model thought process and identify potential causes of policy failures. Additionally, we test DiffusionVLA on a zero-shot bin-picking task, achieving 63.7\% accuracy on 102 previously unseen objects. Our method demonstrates robustness to visual changes, such as distractors and new backgrounds, and easily adapts to new embodiments. Furthermore, DiffusionVLA can follow novel instructions and retain conversational ability. Notably, DiffusionVLA is data-efficient and fast at inference; our smallest DiffusionVLA-2B runs 82Hz on a single A6000 GPU and can train from scratch on less than 50 demonstrations for a complex task. Finally, we scale the model from 2B to 72B parameters, showcasing improved generalization capabilities with increased model size.

Figures

Figures reproduced from arXiv: 2412.03293 by the authors.

Figure 1
Figure 1. Our proposed DiffusionVLA model unifies autoregressive and diffusion modeling to enable self-reasoning and robot policy [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Environmental Setup for the Franka Robot and Experimental Configuration for Factory Sorting. Left: For factor sorting tasks, (a) The target sorting box is divided into four distinct sectors, each designated for one of the following categories: stuffed toys, hex keys, knit gloves, and toy cars, (c) The seen objects in the train data, (d) mixing the seen and unseen object for evaluation, (e) cluttered scene for seen o… view at source ↗
Figure 3
Figure 3. Experimental Results for Factory Sorting. We compared our DiVLA with Diffusion Policy, Octo, TinyVLA, and OpenVLA. DiVLA achieves the highest average success rate, outperforming the runner-up OpenVLA by 20.9%. Reasoning injection module. The core of our approach lies in introducing explicit reasoning into Vision-Language￾Action (VLA) models. Unlike most autoregressive VLAs, which require a recursive setup — converti… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Zero-shot Bin Picking on 102 Unseen Objects. Our method outperforms the state-of-the-art robot foundation models by a large margin [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Examples of visual variations, including randomly placed [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: What internal processes guide a model’s actions? We illustrate this using an example of DiVLA’s reasoning, inferred from shifts in its behavior based on changes to the target object. The model initially intends to grasp a “toy blue car”, but when presented with a hex k…
Figure 7
Figure 7. Figure 7: Some of the unseen objects used for evaluation in the zero-shot bin-picking tasks. illustrated in [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Examples of various unseen objects in zero-shot bin [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: (a) Environmental setup for the bimanual robot, (b) [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Multi-task Learning and Visual Generalization. We evaluate each method on multi-task learning and visual generalizations, including adding additional distractors, changing the background, and implementing colorful lighting. Each set of three images represents the init…
Figure 11
Figure 11. Figure 11: Factory Sorting, Zero-Shot Bin-Picking and Table Bussing. We further evaluate all robot policies on additional challenging tasks, including factory sorting, zero-shot bin-picking, and bimanual table bussing. These tasks involve previously unseen objects with diverse t…
Figure 12
Figure 12. Figure 12: View Generalization. We evaluate DP, OpenVLA, and DiVLA-2B in the view shifting setting, where we use completely different camera positions to capture images. The blue part indi￾cates the original camera positions, while the red part indicates the new camera positions…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 23 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PriGo: Test-Time Primitive Guidance to Diffusion and Flow Policies for Adaptive Robotic Manipulation

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A lightweight primitive classifier and differentiable guidance mechanism improve pretrained diffusion and flow manipulation policies by 3–7 points at test time without retraining.

  2. UniSteer: Unified Noise Steering for Efficient Human-Guided VLA Adaptation

    cs.RO 2026-05 unverdicted novelty 6.0 of 10

    UniSteer unifies human corrective actions and noise-space RL for VLA adaptation by inverting actions to noise targets, raising success rates from 20% to 90% in 66 minutes across four real-world manipulation tasks.

  3. DreamNav: A Trajectory-Based Imaginative Framework for Zero-Shot Vision-and-Language Navigation

    cs.RO 2025-09 conditional novelty 6.0 of 10

    DreamNav achieves new zero-shot SOTA on VLN-CE with an egocentric-only pipeline that generates candidate trajectories, imagines their futures, and selects the best by language alignment.

  4. VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Models

    cs.CV 2025-06 conditional novelty 6.0 of 10

    In controlled experiments across 2D, 3D, simulated, and real robot tasks, VLA-OS shows visually grounded planning representations outperform language planning, and hierarchical planning-plus-action models generally ou...

  5. AntiGrounding: Lifting Robotic Actions into VLM Representation Space for Decision Making

    cs.RO 2025-06 conditional novelty 6.0 of 10

    AntiGrounding lifts candidate robot trajectories into the VLM's visual space via multi-view rendering and structured VQA, and reports 57.5% average success across eight manipulation tasks, beating three intermediate-r...

  6. Visual Embodied Brain: Let Multimodal Large Language Models See, Think, and Control in Spaces

    cs.CV 2025-05 reject novelty 6.0 of 10

    VeBrain unifies perception, spatial reasoning, and robot control in one MLLM by representing control as keypoint detection plus skill selection, with a robotic adapter for deployment.

  7. ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledge

    cs.RO 2025-05 conditional novelty 6.0 of 10

    ChatVLA-2 uses dynamic mixture-of-experts and a two-stage training recipe to let a vision-language-action model retain pretrained reasoning while following robot instructions.

  8. Conditioning Matters: Training Diffusion Policies is Faster Than You Think

    cs.RO 2025-05 conditional novelty 6.0 of 10

    A condition-dependent source distribution for flow-matching policies improves training speed and task success across simulated and real robot benchmarks.

  9. Lift3D-VLA: Lifting VLA Models to 3D Geometry and Dynamics-Aware Manipulation

    cs.RO 2026-07 conditional novelty 5.0 of 10

    Lift3D-VLA integrates 3D point cloud encoding and temporal action modeling into Vision-Language-Action models, achieving higher success rates on simulated and real-world robotic manipulation tasks.

  10. Source Component Shift Adaptation via Offline Decomposition and Online Mixing Approach

    cs.LG 2025-08 unverdicted novelty 5.0 of 10

    The claimed result is that source-component-shift adaptation splits cleanly into offline component learning via EM and online mixing-weight updates, cutting cumulative test loss by up to 67.4%.

  11. GraphCoT-VLA: A 3D Spatial-Aware Reasoning Vision-Language-Action Model for Robotic Manipulation with Ambiguous Instructions

    cs.RO 2025-08 unverdicted novelty 5.0 of 10

    GraphCoT-VLA couples structured chain-of-thought planning and a real-time 3D object-pose graph to improve robot manipulation under vague instructions.

  12. AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulation

    cs.RO 2025-07 conditional novelty 5.0 of 10

    AC-DiT adds mobility-to-body conditioning and perception-aware 2D/3D weighting to a diffusion transformer, improving success rates on simulated and real-world mobile manipulation tasks.

  13. EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Models

    cs.CV 2025-06 conditional novelty 5.0 of 10

    EfficientVLA combines LLM layer pruning, task-aware visual token selection, and diffusion-head feature caching to cut CogACT's inference cost to 28.9% of baseline FLOPs with a 0.6% SIMPLER success drop.

  14. Unveiling the Potential of Vision-Language-Action Models with Open-Ended Multimodal Instructions

    cs.RO 2025-05 conditional novelty 5.0 of 10

    OE-VLA extends vision-language-action models to follow open-ended instructions embedded in images, videos, and goal snapshots, matching text-only performance on the CALVIN benchmark.

  15. Rethinking Latent Redundancy in Behavior Cloning: An Information Bottleneck Approach for Robot Manipulation

    cs.RO 2025-02 conditional novelty 5.0 of 10

    Adding an information bottleneck regularizer that penalizes I(X,Z) between fused input features and the latent representation improves average success rates in behavior cloning benchmarks, though gains depend on a per...

  16. Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challenges

    cs.RO 2025-08 conditional novelty 4.0 of 10

    A survey that taxonomizes robotic manipulation policies trained by imitation learning, traces their evolution, and compiles benchmark comparisons.

  17. Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action Policy

    cs.RO 2025-08 unverdicted novelty 4.0 of 10

    OC-VLA re-labels robot action targets from the robot base frame to the camera frame using the camera's extrinsic calibration, improving cross-view generalization of VLA policies.

  18. Large Model Empowered Embodied AI: A Survey on Decision-Making and Embodied Learning

    cs.RO 2025-08 reject novelty 4.0 of 10

    A review that categorizes large-model-empowered embodied AI into hierarchical and end-to-end decision-making, imitation and reinforcement learning, and world models.

  19. Leveraging OS-Level Primitives for Robotic Action Management

    cs.OS 2025-08 conditional novelty 4.0 of 10

    Applying OS-style exception handling, context caching, and replay to robotic action slices raises success rates 7x to 24x and cuts execution steps up to 74% for repetitive manipulation tasks, without retraining the VLA model.

  20. A Survey on Vision-Language-Action Models for Autonomous Driving

    cs.CV 2025-06 conditional novelty 4.0 of 10

    A survey organizes vision-language-action models for autonomous driving into four stages, compares over 20 systems, and catalogs datasets, benchmarks, and open challenges.

  21. LoHoVLA: A Unified Vision-Language-Action Model for Long-Horizon Embodied Tasks

    cs.RO 2025-05 conditional novelty 4.0 of 10

    A unified vision-language-action model that emits a sub-task description followed by a discrete action token outperforms modular and action-only baselines on simulated long-horizon tabletop tasks.

  22. Think Twice, Act Once: Token-Aware Compression and Action Reuse for Efficient Inference in Vision-Language-Action Models

    cs.CV 2025-05 conditional novelty 4.0 of 10

    FlashVLA, a training-free plug-in, reuses stable actions and prunes visual tokens to cut VLA model inference FLOPs by 55.7% and latency by 36% with only a 0.7% success-rate drop on LIBERO.

  23. A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI

    cs.RO 2025-05 conditional novelty 2.0 of 10

    A review of navigation and manipulation simulators, datasets, and methods, framed around the sim-to-real gap.

Reference graph

Works this paper leans on

72 extracted references · 7 canonical work pages · cited by 23 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    Rt-h: Action hierarchies using language

    Belkhale, S., Ding, T., Xiao, T., Sermanet, P., Vuong, Q., Tompson, J., Chebotar, Y., Dwibedi, D., and Sadigh, D. Rt-h: Action hierarchies using language. arXiv preprint arXiv:2403.01823, 2024

    arXiv 2024

  4. [4]

    Deepseek llm: Scaling open-source language models with longtermism

    Bi, X., Chen, D., Chen, G., Chen, S., Dai, D., Deng, C., Ding, H., Dong, K., Du, Q., Fu, Z., et al. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954, 2024

    arXiv 2024

  5. [5]

    Training diffusion models with reinforcement learning

    Black, K., Janner, M., Du, Y., Kostrikov, I., and Levine, S. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305.13301, 2023 a

    arXiv 2023

  6. [6]

    Zero-shot robotic manipulation with pretrained image-editing diffusion models

    Black, K., Nakamoto, M., Atreya, P., Walke, H., Finn, C., Kumar, A., and Levine, S. Zero-shot robotic manipulation with pretrained image-editing diffusion models. arXiv preprint arXiv:2310.10639, 2023 b

    arXiv 2023

  7. [7]

    X., Tanner, J., Vuong, Q., Walling, A., Wang, H., and Zhilinsky, U

    Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch, K., Shi, L. X., Tanner, J., Vuong, Q., Walling, A., Wang, H., and Zhilinsky, U. _0 : A vision-language-action flow model for general robot control, 202...

    arXiv 2024

  8. [8]

    Rt-1: Robotics transformer for real-world control at scale

    Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Dabis, J., Finn, C., Gopalakrishnan, K., Hausman, K., Herzog, A., Hsu, J., et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817, 2022

    arXiv 2022

Show all 72 references
  1. [9]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Chen, X., Choromanski, K., Ding, T., Driess, D., Dubey, A., Finn, C., et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818, 2023

    2023 arXiv

  2. [10]

    Towards synergistic, generalized, and efficient dual-system for robotic manipulation

    Bu, Q., Li, H., Chen, L., Cai, J., Zeng, J., Cui, H., Yao, M., and Qiao, Y. Towards synergistic, generalized, and efficient dual-system for robotic manipulation. arXiv preprint arXiv:2410.08001, 2024

    2024 arXiv

  3. [11]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Chi, C., Feng, S., Du, Y., Xu, Z., Cousineau, E., Burchfiel, B., and Song, S. Diffusion policy: Visuomotor policy learning via action diffusion. arXiv preprint arXiv:2303.04137, 2023

    2023 arXiv

  4. [12]

    K., and Levine, S

    Dasari, S., Mees, O., Zhao, S., Srirama, M. K., and Levine, S. The ingredients for robotic diffusion transformers. arXiv preprint arXiv:2410.10088, 2024

    2024 arXiv

  5. [13]

    Z., and Finn, C

    Fu, Z., Zhao, T. Z., and Finn, C. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. arXiv preprint arXiv:2401.02117, 2024

    2024 arXiv

  6. [14]

    Seed-x: Multimodal models with unified multi-granularity comprehension and generation

    Ge, Y., Zhao, S., Zhu, J., Ge, Y., Yi, K., Song, L., Li, C., Ding, X., and Shan, Y. Seed-x: Multimodal models with unified multi-granularity comprehension and generation. arXiv preprint arXiv:2404.14396, 2024

    2024 arXiv

  7. [15]

    G., Rao, K., Yu, W., Fu, C., Gopalakrishnan, K., Xu, Z., et al

    Gu, J., Kirmani, S., Wohlhart, P., Lu, Y., Arenas, M. G., Rao, K., Yu, W., Fu, C., Gopalakrishnan, K., Xu, Z., et al. Rt-trajectory: Robotic task generalization via hindsight trajectory sketches. arXiv preprint arXiv:2311.01977, 2023

    2023 arXiv

  8. [16]

    Denoising diffusion probabilistic models

    Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 0 6840--6851, 2020

    2020

  9. [17]

    Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., and Fleet, D. J. Video diffusion models. Advances in Neural Information Processing Systems, 35: 0 8633--8646, 2022

    2022

  10. [18]

    J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  11. [19]

    An embodied generalist agent in 3d world

    Huang, J., Yong, S., Ma, X., Linghu, X., Li, P., Wang, Y., Li, Q., Zhu, S.-C., Jia, B., and Huang, S. An embodied generalist agent in 3d world. In ICLR 2024 Workshop: How Far Are We From AGI

    2024

  12. [20]

    Bc-z: Zero-shot task generalization with robotic imitation learning

    Jang, E., Irpan, A., Khansari, M., Kappler, D., Ebert, F., Lynch, C., Levine, S., and Finn, C. Bc-z: Zero-shot task generalization with robotic imitation learning. In Conference on Robot Learning, pp.\ 991--1002. PMLR, 2022

    2022

  13. [21]

    Mail: Improving imitation learning with selective state space models

    Jia, X., Wang, Q., Donat, A., Xing, B., Li, G., Zhou, H., Celik, O., Blessing, D., Lioutikov, R., and Neumann, G. Mail: Improving imitation learning with selective state space models. In 8th Annual Conference on Robot Learning

  14. [22]

    Vima: General robot manipulation with multimodal prompts

    Jiang, Y., Gupta, A., Zhang, Z., Wang, G., Dou, Y., Chen, Y., Fei-Fei, L., Anandkumar, A., Zhu, Y., and Fan, L. Vima: General robot manipulation with multimodal prompts. arXiv preprint arXiv:2210.03094, 2 0 (3): 0 6, 2022

    2022 arXiv

  15. [23]

    3d diffuser actor: Policy diffusion with 3d scene representations

    Ke, T.-W., Gkanatsios, N., and Fragkiadaki, K. 3d diffuser actor: Policy diffusion with 3d scene representations. arXiv preprint arXiv:2402.10885, 2024

    2024 arXiv

  16. [24]

    K., Chen, L

    Khazatsky, A., Pertsch, K., Nair, S., Balakrishna, A., Dasari, S., Karamcheti, S., Nasiriany, S., Srirama, M. K., Chen, L. Y., Ellis, K., et al. Droid: A large-scale in-the-wild robot manipulation dataset. arXiv preprint arXiv:2403.12945, 2024

    2024 arXiv

  17. [25]

    J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., Sanketi, P., et al

    Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., Sanketi, P., et al. Openvla: An open-source vision-language-action model

  18. [26]

    H., Gonzalez, J

    Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023

    2023

  19. [27]

    Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation

    Li, Q., Liang, Y., Wang, Z., Luo, L., Chen, X., Liao, M., Wei, F., Deng, Y., Xu, S., Zhang, Y., et al. Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:2411.19650, 2024 a

    2024 arXiv

  20. [28]

    Autoregressive image generation without vector quantization

    Li, T., Tian, Y., Li, H., Deng, M., and He, K. Autoregressive image generation without vector quantization. arXiv preprint arXiv:2406.11838, 2024 b

    2024 arXiv

  21. [29]

    Towards generalist robot policies: What matters in building vision-language-action models

    Li, X., Li, P., Liu, M., Wang, D., Liu, J., Kang, B., Ma, X., Kong, T., Zhang, H., and Liu, H. Towards generalist robot policies: What matters in building vision-language-action models. arXiv preprint arXiv:2412.14058, 2024 c

    2024 arXiv

  22. [30]

    Code as policies: Language model programs for embodied control

    Liang, J., Huang, W., Xia, F., Xu, P., Hausman, K., Ichter, B., Florence, P., and Zeng, A. Code as policies: Language model programs for embodied control. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp.\ 9493--9500. IEEE, 2023

    2023

  23. [31]

    Data scaling laws in imitation learning for robotic manipulation, 2024

    Lin, F., Hu, Y., Sheng, P., Wen, C., You, J., and Gao, Y. Data scaling laws in imitation learning for robotic manipulation, 2024. URL https://arxiv.org/abs/2410.18647

    2024 arXiv

  24. [32]

    Liu, H., Li, C., Li, Y., and Lee, Y. J. Improved baselines with visual instruction tuning. arXiv preprint arXiv:2310.03744, 2023 a

    2023 arXiv

  25. [33]

    Liu, H., Li, C., Wu, Q., and Lee, Y. J. Visual instruction tuning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023 b . URL https://openreview.net/forum?id=w0H2xGHlkw

    2023

  26. [34]

    Rdt-1b: a diffusion foundation model for bimanual manipulation

    Liu, S., Wu, L., Li, B., Tan, H., Chen, H., Wang, Z., Xu, K., Su, H., and Zhu, J. Rdt-1b: a diffusion foundation model for bimanual manipulation. arXiv preprint arXiv:2410.07864, 2024

    2024 arXiv

  27. [35]

    N., Zhu, S.-C., and Gao, J

    Lu, P., Peng, B., Cheng, H., Galley, M., Chang, K.-W., Wu, Y. N., Zhu, S.-C., and Gao, J. Chameleon: Plug-and-play compositional reasoning with large language models. Advances in Neural Information Processing Systems, 36, 2024

    2024

  28. [36]

    Y., Sanketi, P., Vuong, Q., Xiao, T., Sadigh, D., Finn, C., and Levine, S

    Octo Model Team , Ghosh, D., Walke, H., Pertsch, K., Black, K., Mees, O., Dasari, S., Hejna, J., Xu, C., Luo, J., Kreiman, T., Tan, Y., Chen, L. Y., Sanketi, P., Vuong, Q., Xiao, T., Sadigh, D., Finn, C., and Levine, S. Octo: An open-source generalist robot policy. In Proceedi...

    2024

  29. [37]

    Open x-embodiment: Robotic learning datasets and rt-x models

    O'Neill, A., Rehman, A., Gupta, A., Maddukuri, A., Gupta, A., Padalkar, A., Lee, A., Pooley, A., Gupta, A., Mandlekar, A., et al. Open x-embodiment: Robotic learning datasets and rt-x models. arXiv preprint arXiv:2310.08864, 2023

    arXiv

  30. [38]

    and Xie, S

    Peebles, W. and Xie, S. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 4195--4205, 2023

    2023

  31. [39]

    Film: Visual reasoning with a general conditioning layer

    Perez, E., Strub, F., De Vries, H., Dumoulin, V., and Courville, A. Film: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018

    2018

  32. [40]

    Fast: Efficient action tokenization for vision-language-action models

    Pertsch, K., Stachowicz, K., Ichter, B., Driess, D., Nair, S., Vuong, Q., Mees, O., Finn, C., and Levine, S. Fast: Efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747, 2025

    2025 arXiv

  33. [41]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., M \"u ller, J., Penna, J., and Rombach, R. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  34. [42]

    Consistency policy: Accelerated visuomotor policies via consistency distillation

    Prasad, A., Lin, K., Wu, J., Zhou, L., and Bohg, J. Consistency policy: Accelerated visuomotor policies via consistency distillation. arXiv preprint arXiv:2405.07503, 2024

    2024 arXiv

  35. [43]

    E., Wenzel, F., and Lioutikov, R

    Reuss, M., Ya g murlu, \"O . E., Wenzel, F., and Lioutikov, R. Multimodal diffusion transformer: Learning versatile behavior from multimodal goals. 2024

    2024

  36. [44]

    High-resolution image synthesis with latent diffusion models

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 10684--10695, 2022

    2022

  37. [45]

    X., Hu, Z., Zhao, T

    Shi, L. X., Hu, Z., Zhao, T. Z., Sharma, A., Pertsch, K., Luo, J., Levine, S., and Finn, C. Yell at your robot: Improving on-the-fly from language corrections. arXiv preprint arXiv:2403.12910, 2024

    2024 arXiv

  38. [46]

    Autoregressive model beats diffusion: Llama for scalable image generation

    Sun, P., Jiang, Y., Chen, S., Zhang, S., Peng, B., Luo, P., and Yuan, Z. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024

    2024 arXiv

  39. [47]

    Chameleon: Mixed-modal early-fusion foundation models

    Team, C. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818, 2024

    2024 arXiv

  40. [48]

    Llama: Open and efficient foundation language models

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi \`e re, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023 a

    2023 arXiv

  41. [49]

    Llama 2: Open foundation and fine-tuned chat models

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023 b

    2023 arXiv

  42. [50]

    L., Tseng, A

    Uehara, M., Zhao, Y., Black, K., Hajiramezanali, E., Scalia, G., Diamant, N. L., Tseng, A. M., Biancalani, T., and Levine, S. Fine-tuning of continuous-time diffusion models as entropy-regularized control. arXiv preprint arXiv:2402.15194, 2024 a

    2024 arXiv

  43. [51]

    L., Tseng, A

    Uehara, M., Zhao, Y., Black, K., Hajiramezanali, E., Scalia, G., Diamant, N. L., Tseng, A. M., Levine, S., and Biancalani, T. Feedback efficient online fine-tuning of diffusion models. arXiv preprint arXiv:2402.16359, 2024 b

    2024 arXiv

  44. [52]

    Equivariant diffusion policy

    Wang, D., Hart, S., Surovik, D., Kelestemur, T., Huang, H., Zhao, H., Yeatman, M., Wang, J., Walters, R., and Platt, R. Equivariant diffusion policy. arXiv preprint arXiv:2407.01812, 2024 a

    2024 arXiv

  45. [53]

    Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution

    Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., et al. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024 b

    2024 arXiv

  46. [54]

    Emu3: Next-token prediction is all you need

    Wang, X., Zhang, X., Luo, Z., Sun, Q., Cui, Y., Wang, J., Zhang, F., Wang, Y., Li, Z., Yu, Q., et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024 c

    2024 arXiv

  47. [55]

    Sparse diffusion policy: A sparse, reusable, and flexible policy for robot learning

    Wang, Y., Zhang, Y., Huo, M., Tian, R., Zhang, X., Xie, Y., Xu, C., Ji, P., Zhan, W., Ding, M., et al. Sparse diffusion policy: A sparse, reusable, and flexible policy for robot learning. arXiv preprint arXiv:2407.01531, 2024 d

    2024 arXiv

  48. [56]

    One-step diffusion policy: Fast visuomotor policies via diffusion distillation

    Wang, Z., Li, Z., Mandlekar, A., Xu, Z., Fan, J., Narang, Y., Fan, L., Zhu, Y., Balaji, Y., Zhou, M., et al. One-step diffusion policy: Fast visuomotor policies via diffusion distillation. arXiv preprint arXiv:2410.21257, 2024 e

    arXiv

  49. [57]

    Tinyvla: Towards fast, data-efficient vision-language-action models for robotic manipulation

    Wen, J., Zhu, Y., Li, J., Zhu, M., Wu, K., Xu, Z., Cheng, R., Shen, C., Peng, Y., Feng, F., et al. Tinyvla: Towards fast, data-efficient vision-language-action models for robotic manipulation. arXiv preprint arXiv:2409.12514, 2024

    2024 arXiv

  50. [58]

    Janus: Decoupling visual encoding for unified multimodal understanding and generation

    Wu, C., Chen, X., Wu, Z., Ma, Y., Liu, X., Pan, Z., Liu, W., Xie, Z., Yu, X., Ruan, C., et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410.13848, 2024 a

    2024 arXiv

  51. [59]

    Vila-u: a unified foundation model integrating visual understanding and generation

    Wu, Y., Zhang, Z., Chen, J., Tang, H., Li, D., Fang, Y., Zhu, L., Xie, E., Yin, H., Yi, L., et al. Vila-u: a unified foundation model integrating visual understanding and generation. arXiv preprint arXiv:2409.04429, 2024 b

    2024 arXiv

  52. [60]

    J., Wang, W., Lin, K

    Xie, J., Mao, W., Bai, Z., Zhang, D. J., Wang, W., Lin, K. Q., Gu, Y., Chen, Z., Yang, Z., and Shou, M. Z. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528, 2024

    2024 arXiv

  53. [61]

    Dnact: Diffusion guided multi-task 3d policy learning

    Yan, G., Wu, Y.-H., and Wang, X. Dnact: Diffusion guided multi-task 3d policy learning. arXiv preprint arXiv:2403.04115, 2024

    2024 arXiv

  54. [62]

    Learning to manipulate anywhere: A visual generalizable framework for reinforcement learning

    Yuan, Z., Wei, T., Cheng, S., Zhang, G., Chen, Y., and Xu, H. Learning to manipulate anywhere: A visual generalizable framework for reinforcement learning. arXiv preprint arXiv:2407.15815, 2024

    2024 arXiv

  55. [63]

    Robotic control via embodied chain-of-thought reasoning

    Zawalski, M., Chen, W., Pertsch, K., Mees, O., Finn, C., and Levine, S. Robotic control via embodied chain-of-thought reasoning. arXiv preprint arXiv:2407.08693, 2024

    2024 arXiv

  56. [64]

    B., and Wu, J

    Ze, Y., Chen, Z., Wang, W., Chen, T., He, X., Yuan, Y., Peng, X. B., and Wu, J. Generalizable humanoid manipulation with improved 3d diffusion policies. arXiv preprint arXiv:2410.10803, 2024 a

    2024 arXiv

  57. [65]

    3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations

    Ze, Y., Zhang, G., Zhang, K., Hu, C., Wang, M., and Xu, H. 3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations. In ICRA 2024 Workshop on 3D Visual Representations for Robot Manipulation, 2024 b

    2024

  58. [66]

    Sigmoid loss for language image pre-training

    Zhai, X., Mustafa, B., Kolesnikov, A., and Beyer, L. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 11975--11986, 2023

    2023

  59. [67]

    Grape: Generalizing robot policy via preference alignment

    Zhang, Z., Zheng, K., Chen, Z., Jang, J., Li, Y., Wang, C., Ding, M., Fox, D., and Yao, H. Grape: Generalizing robot policy via preference alignment. arXiv preprint arXiv:2411.19309, 2024

    2024 arXiv

  60. [68]

    Monoformer: One transformer for both diffusion and autoregression

    Zhao, C., Song, Y., Wang, W., Feng, H., Ding, E., Sun, Y., Xiao, X., and Wang, J. Monoformer: One transformer for both diffusion and autoregression. arXiv preprint arXiv:2409.16280, 2024

    2024 arXiv

  61. [69]

    Z., Tompson, J., Driess, D., Florence, P., Ghasemipour, S

    Zhao, T. Z., Tompson, J., Driess, D., Florence, P., Ghasemipour, S. K. S., Finn, C., and Wahid, A. Aloha unleashed: A simple recipe for robot dexterity. In 8th Annual Conference on Robot Learning

  62. [70]

    Universal actions for enhanced embodied foundation models

    Zheng, J., Li, J., Liu, D., Zheng, Y., Wang, Z., Ou, Z., Liu, Y., Liu, J., Zhang, Y.-Q., and Zhan, X. Universal actions for enhanced embodied foundation models. arXiv preprint arXiv:2501.10105, 2025

    2025 arXiv

  63. [71]

    Transfusion: Predict the next token and diffuse images with one multi-modal model

    Zhou, C., Yu, L., Babu, A., Tirumala, K., Yasunaga, M., Shamis, L., Kahn, J., Ma, X., Zettlemoyer, L., and Levy, O. Transfusion: Predict the next token and diffuse images with one multi-modal model. arXiv preprint arXiv:2408.11039, 2024

    2024 arXiv

  64. [72]

    Scaling diffusion policy in transformer to 1 billion parameters for robotic manipulation

    Zhu, M., Zhu, Y., Li, J., Wen, J., Xu, Z., Liu, N., Cheng, R., Shen, C., Peng, Y., Feng, F., et al. Scaling diffusion policy in transformer to 1 billion parameters for robotic manipulation. arXiv preprint arXiv:2409.14411, 2024

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.