Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

A full robot-learning stack—from motion-capture data to reward-free post-training—lets one policy transfer across unseen robot bodies.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 11:27 UTC pith:J7S522H6

load-bearing objection A credible full-stack VLA system with real hardware results, but the RoboTwin SOTA claim is statistically thin and rests on uncontrolled baselines and a data filter. the 4 major comments →

arxiv 2606.14409 v2 pith:J7S522H6 submitted 2026-06-12 cs.RO cs.AI

Hy-Embodied-0.5-VLA: From Vision-Language-Action Models to a Real-World Robot Learning Stack

He Zhang , Lingzhu Xiang , Haitao Lin , Zeyu Huang , Minghui Wang , Dingyan Zhong , Yubo Dong , Yihao Wu
show 18 more authors
Yongming Rao Dongsheng Zhang Wanjia He Ling Chen Kai Huang Jiahao Chen Sichang Su Xumin Yu Ziyi Wang Chengwei Zhu Xiao Teng Yuchun Guo Yufeng Zhang Yuandong Liu Rui Wang Zisheng Lu Han Hu Zhengyou Zhang
This is my paper
classification cs.RO cs.AI
keywords vision-language-action modelsrobot learning stackcross-embodiment transferflow matchingpreference optimizationmotion capturebimanual manipulationdeployment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that deployable generalist robots will not come from a single model alone, but from a co-designed stack spanning data, architecture, post-training, and deployment. It presents HyVLA-0.5, built on a 10,000-hour corpus of egocentric hand-held demonstrations tracked by an external motion-capture system at sub-millimetre precision. The model pairs a vision-language backbone with a flow-matching action expert and a delta-chunk action representation that is meant to be independent of any specific robot's kinematics. A reward-free preference-optimization stage, FlowPRO, turns failure rollouts into contrastive training signals without learning a reward model. The claimed payoff: top results on a 50-task simulated benchmark, and cross-embodiment transfer to two real robots using only UMI demonstrations.

Core claim

On its own terms, the paper's central discovery is that the bottleneck for generalist robots is not the policy alone but the interface between data, model, and hardware. HyVLA-0.5 shows that a policy trained on end-effector delta chunks, annotated by a motion-capture cage instead of SLAM, can be post-trained on only human hand-held demonstrations and then deployed on morphologically different robots. On a 50-task simulated benchmark it reports the best success rates in both clean (90.9%) and randomized (90.1%) settings, and after FlowPRO post-training four real-robot bimanual tasks reach 94-99% success with shorter completion times than the baselines. The authors interpret this as evidence t

What carries the argument

The central mechanism is the delta-chunk action representation in the end-effector frame: actions are predicted as incremental 3-D position and 6-D rotation changes (plus gripper) relative to the current gripper pose, which decouples the learned policy from embodiment-specific kinematics and lets the same policy be composed with any robot's inverse kinematics at deployment. Around it, the stack uses a compact memory encoder that compresses multi-frame history with interleaved temporal-spatial attention, a flow-matching action expert that regresses continuous velocity fields instead of discretized action tokens, and FlowPRO, a proximalized preference-optimization loss that contrasts paired su

Load-bearing premise

The load-bearing premise is that the external motion-capture system labels every 6-DoF trajectory at sub-millimetre precision in a single globally consistent frame; no calibration or error analysis is provided, so if the true label precision is worse or drifts, the 'high-fidelity' advantage and all cross-embodiment transfer results would need to be reinterpreted.

What would settle it

Measure the motion-capture cage's label error directly against an independent ground-truth tracker in a static and moving gripper test across the full workspace; if the mean pose error is millimetre-scale or shows drift across episodes, the claimed sub-millimetre precision and the transfer benefit attributed to it are falsified. Alternatively, retrain the same pipeline with action labels downsampled or noised to SLAM-level precision; if success rates on Track-B do not drop, the precision claim is not load-bearing.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • On a 50-task simulated benchmark, the full model reaches 90.9% success in clean and 90.1% in randomized settings, the highest among compared methods.
  • Track-B shows that fine-tuning only on task-specific UMI demonstrations—no target-robot teleoperation—yields effective policies on a fixed-base arm and a humanoid platform.
  • FlowPRO post-training lifts four real-robot bimanual tasks to 94-99% success rates and lowers mean completion times, compared to dataset aggregation and an advantage-conditioned regression baseline.
  • Removing the memory encoder or the UMI pre-training stage degrades simulated benchmark performance by about 2 points each, indicating both components carry some of the gain.
  • The same checkpoint can be executed on heterogeneous robots through the embodiment-agnostic platform mapper and asynchronous Bézier smoothing.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the motion-capture label precision holds, a natural test is to inject controlled noise into the UMI action labels and measure the drop in Track-B success; the paper itself proposes this, and it would isolate how much of the benefit comes from label accuracy versus corpus scale and diversity.
  • The intervention-and-rollback preference pipeline is generic: it could be applied to other flow-matching policies, and the proximal regularizer promisingly addresses the reward-hacking failure mode common to preference-optimization methods.
  • The paper admits the stack does not yet demonstrate zero-shot generalization; a fair next step is to apply the same UMI-only transfer recipe to more robot morphologies and to larger aggregation corpora to see whether the result scales.
  • The delta-chunk representation plus the deployment-time mapper effectively turns cross-embodiment transfer into a finite data problem; if it holds, it suggests that target-robot teleoperation can be replaced by cheaper human hand-held collection.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript presents HyVLA-0.5, an end-to-end vision-language-action robot learning stack covering custom UMI data collection with motion capture, a 4B-parameter MoT-based backbone with a flow-matching action expert and a compact memory encoder, continued pre-training on a 10K-hour UMI corpus, supervised fine-tuning in two tracks (intra-embodiment and cross-embodiment), an offline preference-based RL post-training stage (FlowPRO/RPRO), and an asynchronous deployment pipeline with Bézier chunk stitching. The main empirical claims are: state-of-the-art success rates on RoboTwin2.0 (90.9% Clean, 90.1% Randomized), successful cross-embodiment transfer to JAKA K1 and Astribot S1 using only UMI demonstrations, and FlowPRO raising real-robot bimanual success rates to 94–99% while reducing completion time. The paper positions these results as evidence for a co-designed, deployable generalist-robot stack rather than a single model improvement.

Significance. If the reported results hold, the paper is a useful systems contribution: it integrates data collection, architecture, RL post-training, and deployment in one pipeline and reports real-robot experiments with per-seed variance in the FlowPRO study. Strengths include the full-scale release of model, dataset, and code; a critic-free, reward-free RL post-training objective with a proximal regularizer; a parameter-free memory encoder that reduces exactly to the single-frame encoder at K=1; and real-hardware Track-B experiments that avoid target-robot teleoperation. These elements are credible and likely to be reproduced and built upon. The main scientific risk is in the RoboTwin2.0 headline comparison, which uses externally reported baseline numbers and a data-filtering procedure applied only to the proposed method; the statistical and protocol weaknesses there are load-bearing and need to be fixed before the SOTA claim is accepted.

major comments (4)
  1. [§6.1, Table 1; Appendix A] The headline claim of best RoboTwin2.0 success rate is fragile. The comparison uses 'each method's officially reported success rates' rather than re-running baselines under one protocol. The margin over the strongest baseline, JoyAI-RA, is 0.4 points Clean and 0.8 points Randomized; with 100 rollouts × 50 tasks ≈ 5,000 rollouts per setting, the standard error of a 90% success rate is roughly 0.42 percentage points, placing the Clean lead within one SE and the Randomized lead under two SEs. No confidence intervals or per-task variances are reported for the baselines. In addition, Appendix A describes an HDBSCAN episode-length filter applied to HyVLA-0.5's training data; the removal rate and the effect of the filter on downstream success rate are not quantified, and the same filter is not applied to the baselines. If the filter preferentially removes difficult or noisy episodes, it can inf
  2. [§3.1] The paper repeatedly attributes real-robot and cross-embodiment gains to 'sub-millimetre precision' motion-capture labels. However, no metrological validation is provided: there is no calibration error analysis, no RMSE of the mocap system against a held-out reference, and no quantitative comparison against the SLAM baseline that the paper rejects. The claimed advantage of 'high-fidelity' UMI pre-training is load-bearing for the precision-critical-task evidence in §6.2 and for the cross-embodiment transfer story. Please provide a direct measurement of pose-label accuracy (e.g., static and dynamic repeatability, per-axis errors, and drift bounds over a recording session), or explicitly state this precision as an untested assumption and temper the data-quality narrative.
  3. [§6.2, Track B; Fig. 9] The cross-embodiment transfer claim rests on exactly one task per target robot (Put Away the Accessory on JAKA K1; Clean Up the Table on Astribot S1). The reported per-task success rates in Fig. 9 have no number of rollouts, no confidence intervals, and no per-seed variance. Since the UMI data contain no target-robot reachability guarantee (acknowledged in Appendix B.2) and the Astribot deployment relies on a heuristic torso/chassis mapping (Algorithm 1), the evidence for 'UMI-only cross-embodiment transfer' is thin. Please report rollout counts and uncertainty for these two tasks, and consider framing the claim as a feasibility demonstration rather than a general capability unless more tasks and robots are added.
  4. [§6.3, Table 2] The conclusion that RPRO 'consistently dominates' DAgger and π0.6* is supported by only three training seeds per method per task. While reporting per-seed SR is commendable, the differences between RPRO and π0.6* on Cap (99 vs 95) and USB (98 vs 95) are not accompanied by a significance test or raw per-seed values, and the completion times are only cross-rollout means without variance. Please provide per-seed raw numbers or a simple test (e.g., paired bootstrap over the 100 rollouts or over the three seeds), or temper the 'consistently dominates' wording to 'outperforms on these runs'.
minor comments (6)
  1. [Throughout] The manuscript has many missing spaces in the opening paragraphs (e.g., 'Thepastyearhaswitnesseda'), which makes the text hard to read. Please run a typographic pass.
  2. [Fig. 4] The labels contain typos: 'Contianer Filling' should be 'Container Filling', and 'Kitchen' is split across lines in the task-family panel.
  3. [§2.3 / Eq. (3)] The use of 'a' for both the action chunk and individual actions is confusing. Define the chunk dimension explicitly, e.g., A_t ∈ R^{H×10} for two arms, and ensure no clash with Eq. (6)'s at notation.
  4. [Appendix B.1, Algorithm 1] The heuristic mapping depends on parameters α, Δz_C, δ, and θ_0, but their values are not reported. Provide the numerical settings used in the Astribot S1 experiments for reproducibility.
  5. [§6.2, Fig. 9] Please add the number of rollouts per task and per-seed or confidence intervals to the figure; the text cites only aggregate success-rate numbers.
  6. [References] Reference [19] is marked 'Under review' and is also a companion paper by the same group; please clarify its status or avoid relying on it for a core algorithmic claim without a stable citation.

Circularity Check

0 steps flagged

No significant circularity: benchmark claims are externally evaluated; self-citations to Hy-Embodied-0.5 and FlowPRO are transparent lineage, not load-bearing evidence.

full rationale

This is an empirical systems report rather than a formal derivation, and its headline quantities are measured rather than derived. The RoboTwin2.0 success rates (Table 1) and real-robot results (Figs. 9-12, Table 2) are tested against external benchmarks and independent baselines (π0, π0.5, DAgger, π0.6*, etc.), not obtained by fitting a parameter and then renaming it as a prediction. The two main self-citations are [15] (Hy-Embodied-0.5 backbone) and [19] (FlowPRO recipe). Both are explicitly identified as the authors' prior work, and the paper's own ablations (w/o memory encoder, w/o UMI pre-training, RPRO vs DAgger vs π0.6*) provide direct empirical support that does not reduce to those citations. The core equations are standard or transparently borrowed: Eq. 3 is flow matching, Eq. 7 is an explicit Flow-DPO surrogate, Eq. 8 is the resulting loss, and Eqs. 10-11 are kinematic identities for deploying a relative end-effector action representation. The unvalidated motion-capture precision claim (Sec. 3.1) and the RoboTwin SOTA margin/filtering asymmetry (Table 1 vs Appendix A) are evaluation-validity risks, not circularity. No equation or claimed prediction is equivalent to its input by construction.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The report is a system demonstration rather than a derivation; most commitments are empirical design choices and asserted hardware precision rather than mathematical axioms. The list captures the premises a reader must accept for the central claims to transfer to other platforms.

free parameters (4)
  • Bézier/trajectory stitching hyperparameters α, γ, σ = not fully specified; chosen per hardware
    Eqs. 12–17 define the chunk connector; γ is described as a 'deployment hyperparameter chosen according to hardware response' and α, σ are scaling/truncation constants. The paper provides no grid or sensitivity analysis, so the reported smoothness and success rates implicitly depend on these hand-set values.
  • Smooth-interpolation distance weights = 0.5 (geodesic rotation), 0.2 (gripper width), 1.0 (position)
    Appendix C defines d(M,M') = ||pM − pM'||² + 0.5·d_geo + 0.2·|gM − gM'| with 'weights are set empirically'. These weights determine which synthetic preference pairs are generated for the RPRO loss.
  • Heuristic torso/head mapping parameters in Algorithm 1 = α, Δz_C, θ0, δ, R_align, T_T^H
    The humanoid deployment relies on a rule-based estimator with empirically established reach-shell constants. These parameters determine all non-arm DoFs and thus directly affect the Astribot S1 Track-B results.
  • HDBSCAN cluster-selection radius and filtering thresholds = cluster-selection radius 5; mode size <100 episodes; top 5% length tail
    Appendix A uses these choices to flag RoboTwin episodes as dirty and trains on the cleaned subset. The thresholds are not varied and no unfiltered comparison is reported, so the RoboTwin scores depend on this post-hoc filter.
axioms (5)
  • domain assumption External optical motion capture resolves every 6-DoF trajectory at sub-millimetre precision in a single globally consistent world frame.
    Section 3.1 asserts this property without metrological validation. The pre-training data quality claim rests on it.
  • domain assumption The flow-matching regression loss is a tractable surrogate for the negative log-likelihood, hence valid as an implicit reward in Eq. 7.
    Section 4.2, Eq. 6–7 follows Flow-DPO [29], treating ℓ_θ as an NLL surrogate. If this surrogate is unreliable for flow-matching policies, the RPRO loss has no theoretical grounding.
  • domain assumption UMI world frame and Astribot S1 chassis frame are related by a pure translation (identical orientation).
    Appendix B.1 states this 'holds on Astribot S1' and uses it to construct the chassis-frame mapping. If the assumption fails, the humanoid deployment targets are mispositioned.
  • ad hoc to paper HDBSCAN-flagged episodes are genuinely invalid inverse-kinematics solutions and excluding them does not bias evaluation.
    Appendix A assumes that abnormal episode lengths identify implausible IK solutions. This is a post-hoc data-quality assumption specific to this paper; it is not independently verified.
  • domain assumption The delta-chunk relative-EEF representation sufficiently decouples policy learning from embodiment-specific kinematics.
    Sections 1 and 2.1 assume this decoupling; Track-B experiments are designed to test it. The assumption is not derived, and the heuristic torso mapping already injects embodiment-specific structure.

pith-pipeline@v1.3.0-alltime-deepseek · 22934 in / 11886 out tokens · 128468 ms · 2026-08-02T11:27:05.013199+00:00 · methodology

0 comments
read the original abstract

In this report, we present Hy-Embodied-0.5-VLA, abbreviated as HyVLA-0.5, an end-to-end system that spans the full robot learning stack: data collection, model design, continued pre-training and supervised fine-tuning, RL post-training, and real-world deployment. Each component serves a distinct role in this stack.

Figures

Figures reproduced from arXiv: 2606.14409 by Chengwei Zhu, Dingyan Zhong, Dongsheng Zhang, Haitao Lin, Han Hu, He Zhang, Jiahao Chen, Kai Huang, Ling Chen, Lingzhu Xiang, Minghui Wang, Rui Wang, Sichang Su, Wanjia He, Xiao Teng, Xumin Yu, Yihao Wu, Yongming Rao, Yuandong Liu, Yubo Dong, Yuchun Guo, Yufeng Zhang, Zeyu Huang, Zhengyou Zhang, Zisheng Lu, Ziyi Wang.

Figure 1
Figure 1. Figure 1: Overview of Hy-Embodied-0.5-VLA. An end-to-end VLA system that pairs the Hy-Embodied￾0.5-MoT backbone with a flow-matching action expert under a delta-chunk action representation, pre-trained on a 10K-hour egocentric UMI corpus and refined with a reward-free, Proximalized Preference Optimization (PRO)-based offline RL stage (FlowPRO). A single pre-trained checkpoint specializes along two parallel post-trai… view at source ↗
Figure 2
Figure 2. Figure 2: Architectural overview of HyVLA-0.5. The framework adopts a MoT architecture to facilitate cross-modal interactions via a shared joint-attention mechanism. To effectively process K-frame multi￾view RGB sequences, the image encoder is extended into a compact memory encoder. Specifically, temporal attention blocks are interleaved every four layers to enforce causal masking across the temporal dimension and s… view at source ↗
Figure 3
Figure 3. Figure 3: UMI custom data collection workstation. The in-house designed hardware setup features an external optical motion-capture system delivering sub-millimeter high-precision tracking, an ego-centric visual perspective camera with native depth capture, a 6-dimensional force-sensing gripper on each hand. head RGB-D camera to avoid interference of IR emissions. This optical tracking replaces the on-board visual SL… view at source ↗
Figure 4
Figure 4. Figure 4: UMI dataset distribution. Detailed characterization of our diverse, in-house collected 10K-hour UMI demonstration corpus. The distribution outlines broad scale, diverse skill categories, environmental conditions, and manipulated objects, ensuring generalist-level manipulation capacity. Data and Pre-training recipe. We use the full 10K-hour UMI corpus for pre-training. The dataloader samples the dataset wit… view at source ↗
Figure 5
Figure 5. Figure 5: FlowPRO data pipeline for collecting real-robot preference trajectories and converting them into dense per-state preference tuples. During policy rollouts, an operator triggers an intervention-and￾rollback: the system rewinds to a prior state, logs the executed segment as a negative trajectory, and records a corrective teleoperation segment as the paired positive trajectory. A smooth-interpolation procedur… view at source ↗
Figure 6
Figure 6. Figure 6: RPRO optimization. The learnable policy πθ and frozen reference πref predict actions a θ and a ref for the same state. The objective pulls a θ toward the preferred action a w (r w ↑) and pushes it from the dispreferred a l (r l ↓). A proximal regularizer (blue dashed) anchors both reward branches to πref, preventing reward hacking. Batches mix Dk pref, D <k pref, and DSFT. Gaussian noise ϵ ∼ N (0, I) to an… view at source ↗
Figure 7
Figure 7. Figure 7: Asynchronous execution timeline. Policy inference, Bézier smoothing, buffer overwrite, and servo-rate action execution are overlapped; executed actions are recorded in H to estimate tangents for the next chunk stitch. 5.2 Asynchronous Execution for Real-Time Control A high-capacity VLA policy runs slower than the robot servo loop, so synchronous execution would leave the robot idle between forward passes. … view at source ↗
Figure 8
Figure 8. Figure 8: Trajectory comparison between raw action chunks (orange) and the asynchronous Bézier￾smoothed trajectory (blue). Smoothing reduces visible discontinuities at chunk boundaries for both arms across x, y, and z dimensions. with tangents dˆ hist = h0 − h−1 ∥h0 − h−1∥ , dˆ fut = fc+1 − fc−1 ∥fc+1 − fc−1∥ , (14) when the corresponding norm is non-zero. The endpoint control points anchor the transition at the cur… view at source ↗
Figure 9
Figure 9. Figure 9: Real-robot evaluation on six bimanual manipulation tasks. Left panel: Snapshots of representative task executions captured during rollout. Right panel: Per-task success rates (%) after supervised fine-tuning on tele-operated or UMI demonstrations. Track A — Intra-Embodiment Fine-Tuning (Dobot X-Trainer). Data are collected via tele￾operation on a Dobot X-Trainer and the same platform is used for evaluation… view at source ↗
Figure 10
Figure 10. Figure 10: Force-guided object discrimination on a Unitree G1. The robot sequentially grasps two boxes of differing mass and places the lighter one into the front basket, confirming that the in-house UMI workstation captures actionable tactile information. 6.3 Real-World Reinforcement USB Insertion Pen-Cap Assembly [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Additional fine-grained real-robot tasks for FlowPRO post-training. Beyond Insert Bottles (Bottle, sub-cm insertion) and Zip Up the Pen Case, which are shown in [PITH_FULL_IMAGE:figures/full_fig_p016_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Per-iteration success rate on the four real-robot tasks with HyVLA-0.5 as the base policy. Iteration 0 corresponds to the shared SFT checkpoint; iterations 1–3 correspond to successive rounds of post-training. RPRO consistently dominates DAgger and π0.6* throughout the iterative process. 7 Related Work Generalist VLA Models Early VLAs abstracted robotic control into discrete tokens processed by autoregres… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories

    cs.RO 2026-07 conditional novelty 6.0

    Pre-training a VLA model on 100k hours of auto-labeled UMI trajectories, then post-training on robot data, yields SOTA simulated manipulation and data-efficient fine-tuning.

  2. RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imagination

    cs.AI 2026-07 conditional novelty 6.0

    A unified transformer model generates language-and-image planning sequences for embodied tasks, and shows real-robot manipulation without large-scale action pretraining.

  3. ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining

    cs.RO 2026-06 unverdicted novelty 6.0

    ACE-Ego-0 is a VLA pretraining framework that turns egocentric human videos into robot-format pseudo-actions via a video-to-action pipeline and trains jointly with robot data under a reliability-aware objective.

Reference graph

Works this paper leans on

59 extracted references · 30 linked inside Pith · cited by 3 Pith papers

  1. [1]

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky.π0: A visio...

    Pith/arXiv arXiv 2024

  2. [2]

    π0.5: A VLA with open-world generalization.arXiv preprint arXiv:2504.16054, 2025

    Physical Intelligence. π0.5: A VLA with open-world generalization.arXiv preprint arXiv:2504.16054, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montser- rat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, et al. Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Universal pose pretraining for generalizable vision-language-action policies

    Haitao Lin, Hanyang Yu, Jingshun Huang, He Zhang, Yonggen Ling, Ping Tan, Xiangyang Xue, and Yanwei Fu. Universal pose pretraining for generalizable vision-language-action policies. RSS 2026, 2026

    2026

  6. [6]

    Rdt-1b: a diffusion foundation model for bimanual manipulation

    Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1b: a diffusion foundation model for bimanual manipulation. In International Conference on Learning Representations, volume 2025, pages 29982–30009, 2025

    2025

  7. [7]

    Learningfine-grainedbimanual manipulation with low-cost hardware

    TonyZ.Zhao,VikashKumar,SergeyLevine,andChelseaFinn. Learningfine-grainedbimanual manipulation with low-cost hardware. InRobotics: Science and Systems (RSS), 2023

    2023

  8. [8]

    Learning fine-grained bimanual manipulation with low-cost hardware.arXiv preprint arXiv:2304.13705, 2023

    Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware.arXiv preprint arXiv:2304.13705, 2023

    Pith/arXiv arXiv 2023

  9. [9]

    Egovla: Learning vision-language-action models from egocentric human videos.arXiv preprint arXiv:2507.12440, 2025

    RuihanYang,QinxiYu,YechengWu,RuiYan,BoruiLi,An-ChiehCheng,XueyanZou,Yunhao Fang, Xuxin Cheng, Ri-Zhao Qiu, et al. Egovla: Learning vision-language-action models from egocentric human videos.arXiv preprint arXiv:2507.12440, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    Scalable vision-language-action model pretraining for robotic manipulation with real-life human activity videos.arXiv preprint arXiv:2510.21571, 2025

    Qixiu Li, Yu Deng, Yaobo Liang, Lin Luo, Lei Zhou, Chengtang Yao, Lingqi Zeng, Zhiyuan Feng, Huizhi Liang, Sicheng Xu, et al. Scalable vision-language-action model pretraining for robotic manipulation with real-life human activity videos.arXiv preprint arXiv:2510.21571, 2025

    arXiv 2025

  11. [11]

    Universalmanipulationinterface:In-the-wildrobot teachingwithout in-the-wild robots

    Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake,andShuranSong. Universalmanipulationinterface:In-the-wildrobot teachingwithout in-the-wild robots. InRobotics: Science and Systems (RSS), 2024

    2024

  12. [12]

    RT-2:Vision-language-action models transfer web knowledge to robotic control

    AnthonyBrohan,NoahBrown,JusticeCarbajal,YevgenChebotar,XiChen,KrzysztofChoroman- ski,TianliDing,DannyDriess,AvinavaDubey,ChelseaFinn,etal. RT-2:Vision-language-action models transfer web knowledge to robotic control. InConference on Robot Learning (CoRL), 2023

    2023

  13. [13]

    OpenVLA: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. OpenVLA: An open-source vision-language-action model. InConference on Robot Learning (CoRL), 2024

    2024

  14. [14]

    π∗ 0.6:AVLAthatlearnsfromexperience.arXivpreprintarXiv:2511.14759, 2025

    PhysicalIntelligence. π∗ 0.6:AVLAthatlearnsfromexperience.arXivpreprintarXiv:2511.14759, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    Hy-Embodied-0.5: Embodied foundation models for real-world agents

    Tencent Robotics X and Tencent HY Vision Team. Hy-Embodied-0.5: Embodied foundation models for real-world agents. Tencent hy technical report, Tencent, 2025. URLhttps: //github.com/Tencent-Hunyuan/HY-Embodied

    2025

  16. [16]

    PaliGemma: A versatile 3B VLM for transfer.arXiv preprint arXiv:2407.07726, 2024

    Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. PaliGemma: A versatile 3B VLM for transfer.arXiv preprint arXiv:2407.07726, 2024

    Pith/arXiv arXiv 2024

  17. [17]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Eagle 2: Building post-training data strategies from scratch for frontier vision-language models.arXiv preprint arXiv:2501.14818, 2025

    Zhiqi Li, Guo Chen, Shilong Liu, Shihao Wang, Vibashan VS, Yishen Ji, Shiyi Lan, Hao Zhang, Yilin Zhao, Subhashree Radhakrishnan, et al. Eagle 2: Building post-training data strategies from scratch for frontier vision-language models.arXiv preprint arXiv:2501.14818, 2025. 20

    Pith/arXiv arXiv 2025

  19. [19]

    Flowpro: Reward-free reinforced fine-tuning of flow-matching vlas via proximalized preference optimization

    Yihao Wu, He Zhang, Junbo Tan, Xueqian Wang, and Zhengyou Zhang. Flowpro: Reward-free reinforced fine-tuning of flow-matching vlas via proximalized preference optimization. InarXiv preprint arXiv:2606.05468, 2026. Under review

    Pith/arXiv arXiv 2026

  20. [20]

    Mixture-of- transformers: A sparse and scalable architecture for multi-modal foundation models.arXiv preprint arXiv:2411.04996, 2024

    Weixin Liang, Lili Yu, Liang Luo, Srinivasan Iyer, Ning Dong, Chunting Zhou, Gargi Ghosh, Mike Lewis, Wen-tau Yih, Luke Zettlemoyer, and Xi Victoria Lin. Mixture-of- transformers: A sparse and scalable architecture for multi-modal foundation models.arXiv preprint arXiv:2411.04996, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. InInternational Conference on Learning Representations (ICLR), 2023

    2023

  22. [22]

    Multi-scale embodied memory for vision-language-action models.arXiv preprint arXiv:2603.03596, 2026

    Physical Intelligence. Multi-scale embodied memory for vision-language-action models.arXiv preprint arXiv:2603.03596, 2026

    arXiv 2026

  23. [23]

    Mode-adaptive neural networks for quadruped motion control.ACM Transactions on Graphics (ToG), 37(4):1–11, 2018

    He Zhang, Sebastian Starke, Taku Komura, and Jun Saito. Mode-adaptive neural networks for quadruped motion control.ACM Transactions on Graphics (ToG), 37(4):1–11, 2018

    2018

  24. [24]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Pith/arXiv arXiv 2010

  25. [25]

    Patchn’pack:Navit,avisiontransformerforanyaspectratioandresolution.AdvancesinNeural Information Processing Systems, 36:2252–2274, 2023

    Mostafa Dehghani, Basil Mustafa, Josip Djolonga, Jonathan Heek, Matthias Minderer, Mathilde Caron, Andreas Steiner, Joan Puigcerver, Robert Geirhos, Ibrahim M Alabdulmohsin, et al. Patchn’pack:Navit,avisiontransformerforanyaspectratioandresolution.AdvancesinNeural Information Processing Systems, 36:2252–2274, 2023

    2023

  26. [26]

    Decoupled weight decay regularization, 2019

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019. URL https://arxiv.org/abs/1711.05101

    Pith/arXiv arXiv 2019

  27. [27]

    RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation.arXiv preprint arXiv:2506.18088, 2025

    Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Qiwei Liang, Zixuan Li, Xianliang Lin, Yiheng Ge, Zhenyu Gu, Weiliang Deng, and Yubin Guo. RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation.arXiv preprint arXiv:2506.18088, 2025

    Pith/arXiv arXiv 2025

  28. [28]

    Flow straight and fast: Learning to generate and transfer data with rectified flow.International Conference on Learning Representations (ICLR), 2023

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow.International Conference on Learning Representations (ICLR), 2023

    2023

  29. [29]

    Improving video generation with human feedback.arXiv preprint arXiv:2501.13918, 2025

    Jie Liu, Gongye Liu, Jiajun Liang, Zhihao Yuan, Xingchao Liu, Mengnan Zheng, Xuewei Wu, Qian Wang, Wei Qin, Min Xia, et al. Improving video generation with human feedback.arXiv preprint arXiv:2501.13918, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Proximalized preference optimization for diverse feedback types: A decomposed perspective on DPO

    Kuan Guo, Yifan Li, and Zhengyang Chen. Proximalized preference optimization for diverse feedback types: A decomposed perspective on DPO. InAdvances in Neural Information Processing Systems (NeurIPS), volume 38, pages 94533–94576, 2026

    2026

  31. [31]

    ABot-M0:VLAfoundationmodelforroboticmanipulationwithactionmanifold learning.arXiv preprint arXiv:2602.11236, 2026

    FanYangetal. ABot-M0:VLAfoundationmodelforroboticmanipulationwithactionmanifold learning.arXiv preprint arXiv:2602.11236, 2026

    Pith/arXiv arXiv 2026

  32. [32]

    Qwen-vla: Unifying vision-language-action modeling across tasks, environments, and robot embodiments

    Qwen Team. Qwen-vla: Unifying vision-language-action modeling across tasks, environments, and robot embodiments. 2026. URLhttps://arxiv.org/abs/2605.30280

    Pith/arXiv arXiv 2026

  33. [33]

    LingBot-VLA: A pragmatic VLA foundation model.arXiv preprint arXiv:2601.18692, 2026

    Wei Wu et al. LingBot-VLA: A pragmatic VLA foundation model.arXiv preprint arXiv:2601.18692, 2026

    Pith/arXiv arXiv 2026

  34. [34]

    starVLA: A lego-like codebase for vision-language-action model developing.arXiv preprint arXiv:2604.05014, 2026

    StarVLA Community. starVLA: A lego-like codebase for vision-language-action model developing.arXiv preprint arXiv:2604.05014, 2026

    Pith/arXiv arXiv 2026

  35. [35]

    Motus: A unified latent action world model.arXiv preprint arXiv:2512.13030, 2025

    Dong Bi et al. Motus: A unified latent action world model.arXiv preprint arXiv:2512.13030, 2025

    Pith/arXiv arXiv 2025

  36. [36]

    JoyAI-RA 0.1: A foundation model for robotic autonomy.arXiv preprint arXiv:2604.20100, 2026

    Tianle Zhang et al. JoyAI-RA 0.1: A foundation model for robotic autonomy.arXiv preprint arXiv:2604.20100, 2026

    Pith/arXiv arXiv 2026

  37. [37]

    Colin Lea, René Vidal, Austin Reiter, and Gregory D. Hager. Temporal convolutional networks: Aunifiedapproachtoactionsegmentation. InComputerVision–ECCV2016Workshops,pages 47–54. Springer, 2016

    2016

  38. [38]

    Gordon, and J

    Stéphane Ross, Geoffrey J. Gordon, and J. Andrew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. InInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2011

    2011

  39. [39]

    Gemini robotics 1.5: Pushing thefrontierofgeneralistrobotswithadvancedembodiedreasoning,thinking,andmotiontransfer

    Abbas Abdolmaleki, Saminda Abeyruwan, Joshua Ainslie, et al. Gemini robotics 1.5: Pushing thefrontierofgeneralistrobotswithadvancedembodiedreasoning,thinking,andmotiontransfer. arXiv preprint arXiv:2510.03342, 2025

    Pith/arXiv arXiv 2025

  40. [40]

    Qwen3-VL technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuanbo Cai, et al. Qwen3-VL technical report.arXiv preprint arXiv:2511.21631, 2025. 21

    Pith/arXiv arXiv 2025

  41. [41]

    RoboBrain 2.5: Depth insight, time in mind.arXiv preprint arXiv:2601.14352, 2026

    HuajieTan et al. RoboBrain 2.5: Depth insight, time in mind.arXiv preprint arXiv:2601.14352, 2026

    arXiv 2026

  42. [42]

    RynnBrain: Open embodied foundation models, 2026

    Ronghao Dang, Jiayan Guo, Zixuan Zeng, Kang Yan, Jinpeng Wu, Chenrui Shi, Haifeng Wang, Le Liu, Shiyang Chen, Jin Huang, Ziming Huang, and Deli Zhao. RynnBrain: Open embodied foundation models, 2026. URLhttps://arxiv.org/abs/2602.14979

    arXiv 2026

  43. [43]

    A careful examination of large behavior models for multitask robot manipulation.arXiv preprint arXiv:2507.05331, 2025

    Jose Barreiros, Aditya Bhat, Eric Cousineau, et al. A careful examination of large behavior models for multitask robot manipulation.arXiv preprint arXiv:2507.05331, 2025

    Pith/arXiv arXiv 2025

  44. [44]

    Open X-Embodiment: Robotic learning datasets and RT-X models

    Open X-Embodiment Collaboration. Open X-Embodiment: Robotic learning datasets and RT-X models. InIEEE International Conference on Robotics and Automation (ICRA), 2024

    2024

  45. [45]

    DROID: A large-scale in-the-wild robot manipulation dataset

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. DROID: A large-scale in-the-wild robot manipulation dataset. InRobotics: Science and Systems (RSS), 2024

    2024

  46. [46]

    DexUMI:Usinghumanhandastheuniversalmanipulationinterfacefordexterous manipulation.arXiv preprint arXiv:2505.21864, 2025

    Mengda Xu, Han Zhang, Yifan Hou, Zhenjia Goldie Xu, Linxi Fan, Manuela Veloso, and ShuranSong. DexUMI:Usinghumanhandastheuniversalmanipulationinterfacefordexterous manipulation.arXiv preprint arXiv:2505.21864, 2025

    arXiv 2025

  47. [47]

    EgoMI: Learning active vision and whole-body manipulation from egocentric human demonstrations.arXiv preprint arXiv:2511.00153, 2025

    Justin Yu, Yide Shentu, Di Wu, Pieter Abbeel, and Ken Goldberg. EgoMI: Learning active vision and whole-body manipulation from egocentric human demonstrations.arXiv preprint arXiv:2511.00153, 2025

    arXiv 2025

  48. [48]

    HoMMI: Learning whole-body mobile manipulation from human demonstrations.arXiv preprint arXiv:2603.03243, 2026

    Xiaomeng Xu, Jisang Park, Han Zhang, Eric Cousineau, Aditya Bhat, Jose Barreiros, Dian Wang, Shuran Song, and Cheng Chi. HoMMI: Learning whole-body mobile manipulation from human demonstrations.arXiv preprint arXiv:2603.03243, 2026

    Pith/arXiv arXiv 2026

  49. [49]

    Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. InAdvances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  50. [50]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  51. [51]

    Precise and dexterous robotic manipulationviahuman-in-the-loopreinforcementlearning.ScienceRobotics,10(105):eads5033, 2025

    Jianlan Luo, Charles Xu, Jeffrey Wu, and Sergey Levine. Precise and dexterous robotic manipulationviahuman-in-the-loopreinforcementlearning.ScienceRobotics,10(105):eads5033, 2025

    2025

  52. [52]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  53. [53]

    GRAPE: Generalizing robot policy via preference alignment.arXiv preprint arXiv:2411.19309, 2024

    Zijian Zhang, Kaiyuan Zheng, Zhaorun Chen, Joel Jang, Yi Li, and Siwei Lyu. GRAPE: Generalizing robot policy via preference alignment.arXiv preprint arXiv:2411.19309, 2024

    Pith/arXiv arXiv 2024

  54. [54]

    Real-time action chunking with large models.arXiv preprint arXiv:2503.07206, 2025

    Physical Intelligence. Real-time action chunking with large models.arXiv preprint arXiv:2503.07206, 2025

    arXiv 2025

  55. [55]

    Training-time real-time chunking: Co-training high-frequency action refinement with policies, 2025

    Physical Intelligence. Training-time real-time chunking: Co-training high-frequency action refinement with policies, 2025. Physical Intelligence Blog Post

    2025

  56. [56]

    VLASH: Real-time VLAs via future-state-aware asynchronous inference.arXiv preprint arXiv:2512.01031, 2025

    Jiaming Tang, Yufei Sun, Yilong Zhao, et al. VLASH: Real-time VLAs via future-state-aware asynchronous inference.arXiv preprint arXiv:2512.01031, 2025

    Pith/arXiv arXiv 2025

  57. [57]

    22 Appendix A RoboTwin2.0 Evaluation Details Per-taskresults.Table3reportstheper-tasksuccessratesof HyVLA-0.5onthe 50-taskRoboTwin2.0 suite

    Physical Intelligence, Bo Ai, Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Greg Balke, Kevin Black, George Bokinsky, Shihao Cao, Thomas Charbonnier, et al.π0.7: a steerable generalistroboticfoundationmodelwithemergentcapabilities.arXivpreprintarXiv:2604.15483, 2026. 22 Appendix A RoboTwin2.0 Evaluation Details Per-taskresults.Table3reportstheper-tasks...

    Pith/arXiv arXiv 2026

  58. [58]

    It assumes that the UMI world frame and the robot chassis frame are related by a pure translation (identical orientation), which holds on Astribot S1

    Heuristic torso/head pose inference (used in our experiments).A lightweight rule-based estimator consumes bimanual gripper poses{WTGL t ,WTGR t } and infers the world-to-chassis transform, the torso pose, and the head pose such that (i) the torso forward axis aligns with the centroid of the two gripper positions and (ii) the torso height places both gripp...

  59. [59]

    We document this compatibility for completeness; our AstribotS1 results in §6.2 use the heuristic exclusively

    Whole-body IK solvers (alternative).HoMMI-style whole-body IK [48] jointly resolves torso and arm configurations from EE targets and could replace the heuristic above. We document this compatibility for completeness; our AstribotS1 results in §6.2 use the heuristic exclusively. B.2 Track-B Reachability and Data Hygiene Because UMI demonstrations are captu...