Pith. sign in

REVIEW 3 major objections 5 minor 67 references

HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alone

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read High-fidelity handheld demonstrations can replace the real-robot teleoperation anchor at post-training time.

desk verdict Solid system paper with a genuinely new UMI-only post-training comparison, but the headline 'matches teleoperation' overstates what 40-rollout conditions can support; the data show parity cannot be rejected, not that parity holds. read the letter →

arxiv 2607.25895 v1 pith:4IWSNZ3G submitted 2026-07-28 cs.RO cs.CVcs.LG

Simple AI: Yuteng Wei , Jinming Ma , Jiawei Wang , Weitao Zhou , Yushen Zuo , Ke Rui , Minglei Li , Jinhao Zhang
show 9 more authors
Zhikang Pan Xiang Wang Haoran Jia Huan Du Zicheng Zeng Jun Ma Guiyu Qin Di Zhang Xiaofei Li
This is my paper · ORCID
classification cs.ROcs.CVcs.LG MSC 68T40
keywords robotlearningimitationUMIuniversalmanipulationinterfacedatacollectionvision-language-actionmodelsworldactionbimanual
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Robot-free demonstrations have mostly been treated as pre-training material, with a small real-robot teleoperation set added later to make a policy deployable. This paper argues that the real bottleneck is the fidelity of the robot-free data, and it builds a handheld capture system—offline stereo SLAM on the head, hardware-synchronized cameras, ultra-wide views, and a full-palm glove—that records trajectories accurate to about 3 mm. Policies post-trained only on these demonstrations deploy directly on a real robot and match in-domain teleoperation across three backbones from the vision-language-action and world-action-model families, with success-rate differences of -2.5, +3.1, and -0.6 percentage points on four bimanual tasks. Pre-training on the same robot-free corpus further reduces action error on unseen tasks and raises final real-robot success, and a curated 2,000-hour subset is released. The claim, if true, means robot-free UMI data can finish a policy, not just start it.

What carries the argument

The mechanism is the HiFi-UMI capture-and-processing pipeline combined with a policy-agnostic action interface. Pose fidelity comes from head-mounted offline stereo-inertial SLAM with per-hand marker cubes observed in the head frame, which measures the two grippers' relative pose natively rather than reconstructing it; a shared GPIO hardware trigger synchronizes all sensors to under 40 microseconds; and two non-parallel fisheye cameras per hand give roughly 200 degrees of coverage. The action interface is a chunk-anchored, 20-channel bimanual representation—relative end-effector pose increments plus absolute gripper opening—that each backbone's native action tensor absorbs. Every capture the

What would settle it

Run the same UMI-versus-teleoperation protocol on a force-critical task, such as maintaining steady wipe pressure on a sensor-equipped surface or torque-limited insertion, with matched trajectory counts and the same 20-channel position-only action format; if the teleoperation-trained policy beats the UMI-trained one by a margin that cannot be explained by the 40-rollout noise band, then the position-only action interface (or the fidelity of force-sensitive kinematics in handheld capture) is the failing component.

Watch

Extended reading notes

Core claim

The paper's central claim is that data fidelity, not the robot-free setting, is what has kept handheld UMI (Universal Manipulation Interface) demonstrations in a pre-training-only role. With a portable capture device engineered for trajectory accuracy, native inter-gripper relative pose, microsecond synchronization, and roughly 200 degrees of camera coverage per hand, the authors post-train policies solely on such demonstrations and deploy them directly on a real bimanual robot. Across four tasks—stain wiping, shirt folding, remote insertion, and produce sorting—and three backbone policies spanning the VLA and WAM families, UMI-only post-training matches teleoperation post-training, with suc

Load-bearing premise

The results stand or fall on the premise that a 20-channel, position-only description of each demonstration—relative end-effector pose increments plus absolute gripper opening—carries everything the policy needs, including for contact-rich tasks such as wiping, folding, and insertion; no force, torque, or compliance information is recorded or trained.

Editorial extensions

If this is right

  • If the claim holds, robot-free UMI data can serve as post-training supervision, so deployment-ready policies no longer require a teleoperated real-robot anchor for the target task.
  • Collection can be parallelized across sites and operators without the target robot, making it substantially cheaper to grow deployment-grade task data.
  • Pre-training on the same handheld corpus improves both data efficiency and final real-robot success, shifting the efficiency-performance frontier rather than just adding scale.
  • Since parity holds across reactive vision-language-action models and a predictive world-action model, the data source—not the architecture family—appears to be the relevant variable.
  • The practical comparison is between pipelines, not per-trajectory efficiency: UMI used roughly ten times more demonstrations per task than teleoperation, so the advantage is cost and scalability, not sample efficiency.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A reader should not trust per-task orderings at 40 rollouts per task, where one success swing is 2.5 points; the paper's own aggregate framing is the level at which parity is claimed.
  • The paper validates fidelity as a joint design rather than isolating it; deliberately degrading synchronization, field of view, or inter-gripper pose while holding sample count and scene coverage fixed would identify which fidelity property actually carries the result.
  • The action interface is position-only, with no force or torque channel; contact-rich tasks where force information is essential, such as torque-limited screwing or compliant insertion, are the natural place to test whether UMI parity survives.
  • The released corpus is replay-validated for one target arm; transferring to a different robot would require new retargeting and replay validation, so the deployment-grade status is embodiment-specific until re-validated.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents HiFi-UMI, a robot-free data-capture system and pipeline that combines head-mounted offline stereo-inertial SLAM, native inter-gripper relative pose, microsecond-level GPIO synchronization, and ultra-wide six-view sensing. The authors report 3 mm end-effector accuracy, a 96% cumulative replay-validity yield, and the release of a 2,000-hour curated dataset. The central empirical claim is "zero-robot post-training": policies post-trained solely on HiFi-UMI demonstrations match policies post-trained on in-domain real-robot teleoperation, on four tabletop tasks and three backbones (StarVLA-QwenPI, OpenPI-π0.5, LingBot-VA), with aggregate success-rate differences of −2.5, +3.1, and −0.6 percentage points. A second result shows that 4,000 hours of HiFi-UMI pre-training reduces held-out action error by 41% on ten unseen tasks and improves downstream real-robot success by 18.1 points on one backbone. The paper argues that fidelity, not the robot-free setting, is the limiting factor for UMI-style data.

Significance. The hardware and data contributions are substantial. The capture system addresses well-known UMI fidelity bottlenecks (SLAM drift, reconstructed inter-gripper pose, software time alignment, narrow field of view), and the automated reconstruction/replay validation pipeline is a practical step toward scalable deployment-grade data. The evaluation protocol is unusually careful in several respects: a frozen benchmark, separated evaluator roles, randomized policy order, recorded termination reasons, and an asymmetric setup that evaluates UMI policies under scene shift while teleoperation is collected in the evaluation scene. The 2,000-hour open dataset is a valuable community resource. If the parity claim were statistically supported, the result would be significant for the field: robot-free UMI data could replace the real-robot teleoperation anchor at post-training time. However, the headline claim is currently supported only by aggregate point estimates without confidence intervals or equivalence tests, and the comparison is not sample-matched (3,200 UMI vs. ~300 teleoperation trajectories). The system contribution is strong, but the central evidential claim needs substantially more w

major comments (3)
  1. [§6.2, Fig. 9, and §7 (Limitations)] The headline claim "matches in-domain teleoperation" is an equivalence claim, but the design does not support it at the stated precision. Each task–policy pair uses 40 rollouts (one success = 2.5 pp), and aggregate comparisons rest on 160 rollouts per condition. For success rates near 64%, the standard error of the difference is about 5.4 pp; observed differences of −2.5, +3.1, and −0.6 pp are all within ±1 SE, but so is a true gap of 10–15 pp. The paper's own Limitations section concedes the task-level resolution problem, yet the aggregate parity claim inherits the same problem: "within sampling noise" is an absence-of-evidence statement, not evidence for parity. Report confidence intervals on the aggregate differences and ideally a pre-specified equivalence test with a margin (e.g., ±5 or ±10 pp); otherwise the abstract and conclusion should say "no detectable difference in this protoc
  2. [§6.1.3 and §6.2] The comparison is not sample-matched: UMI post-training uses 3,200 demonstrations per task vs. ~300 teleoperation trajectories. The paper explicitly frames this as a comparison of practical pipelines, but the abstract and conclusion state that "UMI data alone" matches teleoperation, which readers will reasonably take as a claim about data source equivalence. With a 10× sample-size advantage, the point estimates could reflect scale effects rather than equivalence of the two data sources. To support the causal "remove the real-robot anchor" claim, add at least one matched-sample condition (e.g., UMI subsampled to ~300 trajectories or teleoperation scaled up) or clearly restrict all conclusions to the practical-pipeline comparison and remove the stronger causal wording. This is the second load-bearing weakness.
  3. [§5, Eq. (2); §6.1.2] The action representation is purely kinematic: 3 + 6 + 1 = 10 channels per arm (relative pose increment + absolute gripper opening), with no force, torque, or compliance channel. The robot hardware is force-controlled, but policies emit pose targets only. For contact-rich tasks such as Stain Wiping, Shirt Folding, and Remote Insertion, force interaction information is not represented in either the UMI or teleoperation condition, so this does not invalidate the UMI-vs-teleop parity comparison (both use the same interface). However, the general claim that HiFi-UMI yields "deployable manipulation policies" is thereby limited to tasks solvable by kinematic trajectory tracking. The paper should add force/torque channels to the interface or explicitly scope the claim. This is a substantive limitation of the deployment claim as stated.
minor comments (5)
  1. [§6.2.2, Fig. 12] The ground-truth-video diagnostic reports overlapping bootstrap confidence intervals between UMI→Real and Real→Real. The text says the decoders achieve "comparable" accuracy; this should be phrased as "we cannot rule out a difference" unless a formal equivalence or non-inferiority analysis is added.
  2. [§6.3, Eq. (9), Fig. 13] The power-law fit to a single learning curve with correlated checkpoints is descriptive, not a dataset-scaling law. The caption already notes "exposure scaling," but the abstract's "power-law trend" may overstate it; consider "exposure-driven trend" consistently.
  3. [§3.1.2] The full-palm glove is said to preserve "natural force and contact," but no force measurement is made or released. This wording should be softened to avoid implying force channels are captured.
  4. [Tab. 1] Positional errors from different systems use different references and motion profiles; the caption already cautions against direct comparison, but the table's visual alignment may still invite over-reading. Consider adding a prominent footnote on the first use of the table.
  5. [§5.4] The description of training variants is clear, but the relationship between C1 (scratch action head) and C7 (UMI-pretrained init.) could be stated even more explicitly in the main text, since the "matched post-training data" comparison in Fig. 15 depends on it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central parity claim rests on an empirical benchmark comparison, not on a derivation from its own inputs.

full rationale

The paper's main claims are empirical: UMI-only post-training is compared with teleoperation post-training on three backbones under a frozen benchmark, and the reported differences are observed success rates, not quantities forced by construction. The action representation in Eq. (2) is a shared physical convention applied identically to both UMI and teleoperation data, so it does not encode the outcome. The only fitted model is the power-law scaling relation in Eq. (9), which is explicitly fit to the held-out learning curve and reported as a fit ('The fitted exponent is α=0.268'); it is not used to generate the parity claim. The corpus is used for both pre-training and post-training, but this is an experimental design choice rather than a circular step, and the paper separately controls the initialization comparison. Citations to public checkpoints (OpenPI-π0.5, LingBot-VA, StarVLA) are external baselines, not uniqueness theorems or ansatze imported to force the result. The paper also discloses the sample-count asymmetry and the lack of a fidelity ablation as limitations, further indicating that the conclusions are not tautological. Any concerns about statistical power or force-channel omission are external-validity/correctness risks, not circularity.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

No new physical entities are postulated. The central claim rests on a set of engineering and measurement assumptions about SLAM accuracy, simulation-replay validity, the sufficiency of position-only action labels, and embodiment matching. The only fitted quantities are the scaling-law exponents, which are descriptive curve fits rather than independent predictions.

free parameters (2)
  • Power-law scaling parameters (L∞, A, α) = L∞ and A not reported; α=0.268 (pre-training), α=0.095 (OOD); family exponents 0.134/0.105/0.072; R²=0.993/0.836/0.520/0
    Eq. (9) fits held-out action MSE as L(S)=L∞+A S^{-α}; all three parameters are fitted to the learning curve. The abstract calls this a scaling trend, but no parameter-free derivation or out-of-fit prediction is provided.
  • Flow-matching sampling hyperparameters = Beta(1.5,1.0); s=0.999; 8 Euler steps; H=20, H_exec=10
    Hand-chosen recipe in Sec. 5.1; not fitted to the central parity claim but part of the training configuration.
assumptions (5)
  • domain assumption Head-mounted offline stereo-inertial SLAM plus fiducial markers yields ~3 mm workspace-local end-effector error over a ~2 m workspace without external tracking.
    Reported in Sec. 3.4/Tab. 2 as a single number; the measurement protocol (motion profile, number of trials, marker geometry) is not given. The parity claim requires this accuracy to hold during normal capture.
  • domain assumption Simulation replay with the whole-body controller certifies a reconstructed trajectory as a valid robot-executable demonstration.
    Sec. 3.3.3 applies replay validation to all data and reports 98% pass; replay feasibility is treated as equivalent to deployability, but the real-robot validity of replay-pass trajectories is not separately measured.
  • domain assumption The 20-channel pose-plus-opening action interface (Eq. 2) carries all information needed to learn contact-rich tasks; no force/torque channel is used.
    Sec. 5 defines 10 active channels per arm (3 translation, 6 rotation, 1 gripper). Force/torque is never recorded; force-controlled arms receive pose targets. Contact-rich parity depends on this sufficiency.
  • domain assumption The deployment robot's gripper and wrist cameras exactly match the HiFi-UMI capture rig, so the embodiment gap reduces to arm kinematics.
    Sec. 6.1.1 states that the platform 'carries the same gripper and four wrist cameras'; physical identity is asserted, not calibrated or measured.
  • domain assumption Native inter-gripper relative pose from a single head frame is as accurate as per-gripper pose.
    Sec. 3.1.1: both marker cubes are observed by the head cameras, but occlusion, marker loss, and wrist motion are not quantified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alone." pith.science (2026). https://pith.science/paper/4IWSNZ3G

@misc{pith2026260725895,
  author       = {Pith},
  title        = {Pith review of: HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alone},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4IWSNZ3G}},
  note         = {Machine review of arXiv:2607.25895}
}
read the original abstract

Learning deployable manipulation policies is bottlenecked by the scarcity of data that is both high-fidelity and scalable. Real-robot teleoperation is accurate but costly to scale; robot-free UMI capture scales readily, and current practice uses the resulting data mainly for pre-training, adding a small real-robot "anchor" at post-training. We ask whether raising the fidelity of robot-free UMI data, rather than shrinking the real-robot fraction, can remove that anchor. We present HiFi-UMI, a portable UMI data-production system co-designed for trajectory accuracy, inter-gripper relative pose, synchronization, and field of view: head-mounted offline stereo-inertial SLAM, native rather than reconstructed relative pose, a shared microsecond GPIO trigger, and two wide-angle cameras per hand covering ~200 degrees. It reaches 3 mm workspace-local end-effector accuracy without external tracking infrastructure. Using this corpus, we demonstrate zero-robot post-training: a policy post-trained solely on HiFi-UMI demonstrations deploys directly on a real robot and matches in-domain teleoperation across three backbones spanning the vision-language-action and world-action-model families, with success-rate differences of -2.5, +3.1, and -0.6 percentage points on StarVLA-QwenPI, OpenPI-pi_0.5, and LingBot-VA; the strongest policy reaches 85% on a precision insertion task, even though the teleoperation baseline is collected in the evaluation scene and no HiFi-UMI trajectory is. Pre-training on 4,000 hours from the same corpus lowers action error on ten unseen tasks by 41% and, on StarVLA-QwenPI, raises real-robot success by a further 18.1 percentage points. We open-source HiFi-UMI-2K, 2,000 hours of microsecond-synchronized, ultra-wide-FoV demonstrations, each automatically reconstructed and validated through simulation replay, as a large-scale, high-fidelity resource for the robot-learning community.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 53 linked inside Pith

  1. [1]

    RT-1: Robotics transformer for real-world control at scale

    Anthony Brohan, Noah Brown, Justice Carbajal, et al. RT-1: Robotics transformer for real-world control at scale. InRobotics: Science and Systems (RSS), 2023

    2023

  2. [2]

    BridgeData V2: A dataset for robot learning at scale

    Homer Walke, Kevin Black, Abraham Lee, et al. BridgeData V2: A dataset for robot learning at scale. In Conference on Robot Learning (CoRL), 2023. arXiv:2308.12952

    arXiv 2023

  3. [3]

    DROID: A large-scale in-the-wild robot manipulation dataset

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, et al. DROID: A large-scale in-the-wild robot manipulation dataset. InRobotics: Science and Systems (RSS), 2024. arXiv:2403.12945

    arXiv 2024

  4. [4]

    AgiBot World Colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems.arXiv preprint arXiv:2503.06669, 2025

    AgiBot-World-Contributors, Qingwen Bu, Jisong Cai, et al. AgiBot World Colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems.arXiv preprint arXiv:2503.06669, 2025

    arXiv 2025

  5. [5]

    RoboMIND: Benchmark on multi-embodiment intelligence normative data for robot manipulation.arXiv preprint arXiv:2412.13877, 2024

    Kun Wu, Chengkai Hou, Jiaming Liu, et al. RoboMIND: Benchmark on multi-embodiment intelligence normative data for robot manipulation.arXiv preprint arXiv:2412.13877, 2024

    arXiv 2024

  6. [6]

    Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots

    Cheng Chi, Zhenjia Xu, Chuer Pan, et al. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots. InRobotics: Science and Systems (RSS), 2024. arXiv:2402.10329

    arXiv 2024

  7. [7]

    FastUMI: A scalable and hardware-independent universal manip- ulation interface with dataset.arXiv preprint arXiv:2409.19499, 2024

    Zhaxizhuoma, Kehui Liu, Chuyue Guan, et al. FastUMI: A scalable and hardware-independent universal manip- ulation interface with dataset.arXiv preprint arXiv:2409.19499, 2024

    arXiv 2024

  8. [8]

    Data scaling laws in imitation learning for robotic manipulation

    Fanqi Lin, Yingdong Hu, Pingyue Sheng, et al. Data scaling laws in imitation learning for robotic manipulation. InInternational Conference on Learning Representations (ICLR), 2025. arXiv:2410.18647

    arXiv 2025

Show all 67 references
  1. [9]

    AirExo-2: Scaling up generalizable robotic imitation learning with low-cost exoskeletons.arXiv preprint arXiv:2503.03081, 2025

    Hongjie Fang, Chenxi Wang, Yiming Wang, et al. AirExo-2: Scaling up generalizable robotic imitation learning with low-cost exoskeletons.arXiv preprint arXiv:2503.03081, 2025

    2025 arXiv

  2. [10]

    GR00T N1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, et al. GR00T N1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    2025 arXiv

  3. [11]

    H-RDT: Human manipulation enhanced bimanual robotic manip- ulation.arXiv preprint arXiv:2507.23523, 2025

    Hongzhe Bi, Lingxuan Wu, Tianwei Lin, et al. H-RDT: Human manipulation enhanced bimanual robotic manip- ulation.arXiv preprint arXiv:2507.23523, 2025

    2025 arXiv

  4. [12]

    John, et al

    Qiyuan Zeng, Chengmeng Li, Jude St. John, et al. ActiveUMI: Robotic manipulation with active perception from robot-free human demonstrations.arXiv preprint arXiv:2510.01607, 2025

    2025

  5. [13]

    XRZero-G0: Pushing the frontier of dexterous robotic manipulation with interfaces, quality and ratios.arXiv preprint arXiv:2604.13001, 2026

    James Wang, Primo Pu, Zephyr Fung, et al. XRZero-G0: Pushing the frontier of dexterous robotic manipulation with interfaces, quality and ratios.arXiv preprint arXiv:2604.13001, 2026

    2026 arXiv

  6. [14]

    RDT2: Exploring the scaling limit of UMI data towards zero-shot cross-embodiment generalization.arXiv preprint arXiv:2602.03310, 2026

    Songming Liu, Bangguo Li, Kai Ma, et al. RDT2: Exploring the scaling limit of UMI data towards zero-shot cross-embodiment generalization.arXiv preprint arXiv:2602.03310, 2026

    2026

  7. [15]

    RH20T: A comprehensive robotic dataset for learning diverse skills in one-shot.arXiv preprint arXiv:2307.00595, 2023

    Hao-Shu Fang, Hongjie Fang, Zhenyu Tang, et al. RH20T: A comprehensive robotic dataset for learning diverse skills in one-shot.arXiv preprint arXiv:2307.00595, 2023

    2023 arXiv

  8. [16]

    Open X-embodiment: Robotic learning datasets and RT-X models

    Open X-Embodiment Collaboration, Abby O’Neill, Abdul Rehman, et al. Open X-embodiment: Robotic learning datasets and RT-X models. InIEEE International Conference on Robotics and Automation (ICRA), 2024. arXiv:2310.08864. 30

    2024 arXiv

  9. [17]

    RoboMIND 2.0: A multimodal, bimanual mobile manipulation dataset for generalizable embodied intelligence.arXiv preprint arXiv:2512.24653, 2025

    Chengkai Hou, Kun Wu, Jiaming Liu, et al. RoboMIND 2.0: A multimodal, bimanual mobile manipulation dataset for generalizable embodied intelligence.arXiv preprint arXiv:2512.24653, 2025

    2025

  10. [18]

    Ego4D: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, et al. Ego4D: Around the world in 3,000 hours of egocentric video. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. arXiv:2110.07058

    2022 arXiv

  11. [19]

    Ego-Exo4D: Understanding skilled human activity from first- and third-person perspectives

    Kristen Grauman, Andrew Westbury, Lorenzo Torresani, et al. Ego-Exo4D: Understanding skilled human activity from first- and third-person perspectives. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. arXiv:2311.18259

    2024 arXiv

  12. [20]

    Yoon, et al

    Ryan Hoque, Peide Huang, David J. Yoon, et al. EgoDex: Learning dexterous manipulation from large-scale egocentric video.arXiv preprint arXiv:2505.11709, 2025

    2025 arXiv

  13. [21]

    FastUMI-100K: Advancing data-driven robotic manipulation with a large-scale UMI-style dataset.arXiv preprint arXiv:2510.08022, 2025

    Kehui Liu, Zhongjie Jia, Yang Li, et al. FastUMI-100K: Advancing data-driven robotic manipulation with a large-scale UMI-style dataset.arXiv preprint arXiv:2510.08022, 2025

    2025

  14. [22]

    VISTA: Vision-grounded and physics-validated adaptation of UMI data for VLA training.arXiv preprint arXiv:2606.04708, 2026

    Siyuan Yang, Linzheng Guo, Ouyang Lu, et al. VISTA: Vision-grounded and physics-validated adaptation of UMI data for VLA training.arXiv preprint arXiv:2606.04708, 2026

    2026 arXiv

  15. [23]

    DexCap: Scalable and portable mocap data collection system for dexterous manipulation

    Chen Wang, Haochen Shi, Weizhuo Wang, et al. DexCap: Scalable and portable mocap data collection system for dexterous manipulation. InRobotics: Science and Systems (RSS), 2024. arXiv:2403.07788

    2024 arXiv

  16. [24]

    DexUMI: Using human hand as the universal manipulation interface for dexterous manipulation

    Mengda Xu, Han Zhang, Yifan Hou, et al. DexUMI: Using human hand as the universal manipulation interface for dexterous manipulation. InConference on Robot Learning (CoRL), 2025. arXiv:2505.21864

    2025

  17. [25]

    DexWild: Dexterous human interactions for in-the-wild robot policies.Robotics: Science and Systems (RSS), 2025

    Tony Tao, Mohan Kumar Srirama, Jason Jingzhou Liu, et al. DexWild: Dexterous human interactions for in-the-wild robot policies.Robotics: Science and Systems (RSS), 2025

    2025

  18. [26]

    ARCap: Collecting high-quality human demonstrations for robot learning with augmented reality feedback

    Sirui Chen, Chen Wang, Kaden Nguyen, et al. ARCap: Collecting high-quality human demonstrations for robot learning with augmented reality feedback. InIEEE International Conference on Robotics and Automation (ICRA), 2025. arXiv:2410.08464

    2025 arXiv

  19. [27]

    AirExo: Low-cost exoskeletons for learning whole-arm manipulation in the wild

    Hongjie Fang, Hao-Shu Fang, Yiming Wang, et al. AirExo: Low-cost exoskeletons for learning whole-arm manipulation in the wild. InIEEE International Conference on Robotics and Automation (ICRA), 2024. arXiv:2309.14975

    2024 arXiv

  20. [28]

    EgoMimic: Scaling imitation learning via egocentric video

    Simar Kareer, Dhruv Patel, Ryan Punamiya, et al. EgoMimic: Scaling imitation learning via egocentric video. InIEEE International Conference on Robotics and Automation (ICRA), 2025. arXiv:2410.24221

    2025 arXiv

  21. [29]

    RT-2: Vision-language-action models transfer web knowledge to robotic control

    Anthony Brohan, Noah Brown, Justice Carbajal, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. InConference on Robot Learning (CoRL), 2023. arXiv:2307.15818

    2023 arXiv

  22. [30]

    OpenVLA: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, et al. OpenVLA: An open-source vision-language-action model. InConference on Robot Learning (CoRL), 2024. arXiv:2406.09246

    2024 arXiv

  23. [31]

    Octo: An open-source generalist robot policy

    Octo Model Team, Dibya Ghosh, Homer Walke, et al. Octo: An open-source generalist robot policy. InRobotics: Science and Systems (RSS), 2024. arXiv:2405.12213

    2024 arXiv

  24. [32]

    arXiv preprint arXiv:2410.24164, 2024

    Kevin Black, Noah Brown, Danny Driess, et al.π0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024

    2024 arXiv

  25. [33]

    Physical Intelligence, Kevin Black, Noah Brown, et al.π0.5: A vision-language-action model with open-world generalization.arXiv preprint arXiv:2504.16054, 2025

    2025 arXiv

  26. [34]

    GR-3 technical report.arXiv preprint arXiv:2507.15493, 2025

    ByteDance Seed. GR-3 technical report.arXiv preprint arXiv:2507.15493, 2025

    2025 arXiv

  27. [35]

    Gemini robotics: Bringing AI into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Google DeepMind. Gemini robotics: Bringing AI into the physical world.arXiv preprint arXiv:2503.20020, 2025

    2025 arXiv

  28. [36]

    SmolVLA: A vision-language-action model for affordable and efficient robotics.arXiv preprint arXiv:2506.01844, 2025

    Mustafa Shukor, Dana Aubakirova, Francesco Capuano, et al. SmolVLA: A vision-language-action model for affordable and efficient robotics.arXiv preprint arXiv:2506.01844, 2025

    2025 arXiv

  29. [37]

    FAST: Efficient action tokenization for vision-language-action models.arXiv preprint arXiv:2501.09747, 2025

    Karl Pertsch, Kyle Stachowicz, Brian Ichter, et al. FAST: Efficient action tokenization for vision-language-action models.arXiv preprint arXiv:2501.09747, 2025

    2025 arXiv

  30. [38]

    Fine-tuning vision-language-action models: Optimizing speed and success.Robotics: Science and Systems (RSS), 2025

    Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and success.Robotics: Science and Systems (RSS), 2025. arXiv:2502.19645

    2025 arXiv

  31. [39]

    A pragmatic VLA foundation model.arXiv preprint arXiv:2601.18692, 2026

    Wei Wu, Fan Lu, Yunnan Wang, et al. A pragmatic VLA foundation model.arXiv preprint arXiv:2601.18692, 2026. 31

    2026 arXiv

  32. [40]

    Qwen-VLA: Unifying vision-language-action modeling across tasks, environments, and robot embodiments.arXiv preprint arXiv:2605.30280, 2026

    Qiuyue Wang, Mingsheng Li, Jian Guan, et al. Qwen-VLA: Unifying vision-language-action modeling across tasks, environments, and robot embodiments.arXiv preprint arXiv:2605.30280, 2026

    2026 arXiv

  33. [41]

    Qwen-RobotManip technical report: Alignment unlocks scale for robotic manipulation foundation models.arXiv preprint arXiv:2606.17846, 2026

    Haoqi Yuan, Zhixuan Liang, Anzhe Chen, et al. Qwen-RobotManip technical report: Alignment unlocks scale for robotic manipulation foundation models.arXiv preprint arXiv:2606.17846, 2026

    2026 arXiv

  34. [42]

    Unleashing large-scale video generative pre-training for visual robot manipulation

    Hongtao Wu, Ya Jing, Chilam Cheang, et al. Unleashing large-scale video generative pre-training for visual robot manipulation. InInternational Conference on Learning Representations (ICLR), 2024. arXiv:2312.13139

    2024 arXiv

  35. [43]

    GR-2: A generative video-language-action model with web-scale knowledge for robot manipulation.arXiv preprint arXiv:2410.06158, 2024

    Chi-Lam Cheang, Guangzeng Chen, Ya Jing, et al. GR-2: A generative video-language-action model with web-scale knowledge for robot manipulation.arXiv preprint arXiv:2410.06158, 2024

    2024 arXiv

  36. [44]

    Learning universal policies via text-guided video generation

    Yilun Du, Mengjiao Yang, Bo Dai, et al. Learning universal policies via text-guided video generation. InAdvances in Neural Information Processing Systems (NeurIPS), 2023. arXiv:2302.00111

    2023 arXiv

  37. [45]

    Video prediction policy: A generalist robot policy with predic- tive visual representations

    Yucheng Hu, Yanjiang Guo, Pengchao Wang, et al. Video prediction policy: A generalist robot policy with predic- tive visual representations. InInternational Conference on Machine Learning (ICML), 2025. arXiv:2412.14803

    2025 arXiv

  38. [46]

    World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026

    Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, et al. World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026

    2026 arXiv

  39. [47]

    WorldVLA: Towards autoregressive action world model.arXiv preprint arXiv:2506.21539, 2025

    Jun Cen, Chaohui Yu, Hangjie Yuan, et al. WorldVLA: Towards autoregressive action world model.arXiv preprint arXiv:2506.21539, 2025

    2025 arXiv

  40. [48]

    Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

    Lin Li, Qihang Zhang, Yiming Luo, et al. Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

    2026 arXiv

  41. [49]

    StarVLA: A lego-like codebase for vision-language-action model developing.arXiv preprint arXiv:2604.05014, 2026

    StarVLA Community. StarVLA: A lego-like codebase for vision-language-action model developing.arXiv preprint arXiv:2604.05014, 2026. Code available at: https://github.com/starVLA/starVLA

    2026 arXiv

  42. [50]

    VINS-Mono: A robust and versatile monocular visual-inertial state estimator.IEEE Transactions on Robotics, 34(4):1004–1020, 2018

    Tong Qin, Peiliang Li, and Shaojie Shen. VINS-Mono: A robust and versatile monocular visual-inertial state estimator.IEEE Transactions on Robotics, 34(4):1004–1020, 2018

    2018

  43. [51]

    Gómez Rodríguez, et al

    Carlos Campos, Richard Elvira, Juan J. Gómez Rodríguez, et al. ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multimap SLAM.IEEE Transactions on Robotics, 37(6):1874–1890, 2021. arXiv:2007.11898

    2021 arXiv

  44. [52]

    AprilTag: A robust and flexible visual fiducial system

    Edwin Olson. AprilTag: A robust and flexible visual fiducial system. InIEEE International Conference on Robotics and Automation (ICRA), pp. 3400–3407, 2011

    2011

  45. [53]

    VersaVIS—an open versatile multi-camera visual-inertial sensor suite.Sensors, 20(5):1439, 2020

    Florian Tschopp, Michael Riner, Marius Fehr, et al. VersaVIS—an open versatile multi-camera visual-inertial sensor suite.Sensors, 20(5):1439, 2020. arXiv:1912.02469

    2020 arXiv

  46. [54]

    DAS Fingers: A high-precision multimodal data-collection device for embodied AI.https: //www.genrobot.ai/products/finger, 2025

    GenRobot AI. DAS Fingers: A high-precision multimodal data-collection device for embodied AI.https: //www.genrobot.ai/products/finger, 2025. Product page. Accessed: 2026-07-08

    2025

  47. [55]

    TacUMI: A multi-modal universal manipulation interface for contact-rich tasks.arXiv preprint arXiv:2601.14550, 2026

    Tailai Cheng, Kejia Chen, Lingyun Chen, et al. TacUMI: A multi-modal universal manipulation interface for contact-rich tasks.arXiv preprint arXiv:2601.14550, 2026

    2026

  48. [56]

    Fácil, Javier Civera, et al

    Berta Bescos, José M. Fácil, Javier Civera, et al. DynaSLAM: Tracking, mapping, and inpainting in dynamic scenes.IEEE Robotics and Automation Letters, 3(4):4076–4083, 2018. arXiv:1806.05620

    2018 arXiv

  49. [57]

    UMI on legs: Making manipulation policies mobile with manipulation- centric whole-body controllers.arXiv preprint arXiv:2407.10353, 2024

    Huy Ha, Yihuai Gao, Zipeng Fu, et al. UMI on legs: Making manipulation policies mobile with manipulation- centric whole-body controllers.arXiv preprint arXiv:2407.10353, 2024

    2024 arXiv

  50. [58]

    RoboVQA: Multimodal long-horizon reasoning for robotics

    Pierre Sermanet, Tianli Ding, Jeffrey Zhao, et al. RoboVQA: Multimodal long-horizon reasoning for robotics. In IEEE International Conference on Robotics and Automation (ICRA), pp. 645–652, 2024. arXiv:2311.00899

    2024 arXiv

  51. [59]

    On the continuity of rotation representations in neural networks

    Yi Zhou, Connelly Barnes, Jingwan Lu, et al. On the continuity of rotation representations in neural networks. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. arXiv:1812.07035

    2019 arXiv

  52. [60]

    Qwen3 Technical Report.arXiv preprint arXiv:2505.09388, 2025

    Qwen Team. Qwen3 Technical Report.arXiv preprint arXiv:2505.09388, 2025

    2025 arXiv

  53. [61]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, et al. Flow matching for generative modeling. InInternational Conference on Learning Representations (ICLR), 2023. arXiv:2210.02747

    2023 arXiv

  54. [62]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InIEEE/CVF International Conference on Computer Vision (ICCV), 2023. arXiv:2212.09748. 32

    2023 arXiv

  55. [63]

    Zhao, Vikash Kumar, Sergey Levine, et al

    Tony Z. Zhao, Vikash Kumar, Sergey Levine, et al. Learning fine-grained bimanual manipulation with low-cost hardware. InRobotics: Science and Systems (RSS), 2023

    2023

  56. [64]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Siyuan Feng, Yilun Du, et al. Diffusion policy: Visuomotor policy learning via action diffusion. In Robotics: Science and Systems (RSS), 2023

    2023

  57. [65]

    PaliGemma: A versatile 3b VLM for transfer.arXiv preprint arXiv:2407.07726, 2024

    Lucas Beyer, Andreas Steiner, André Susano Pinto, et al. PaliGemma: A versatile 3b VLM for transfer.arXiv preprint arXiv:2407.07726, 2024

    2024 arXiv

  58. [66]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations (ICLR), 2019. arXiv:1711.05101

    2019 arXiv

  59. [67]

    A careful examination of large behavior models for multitask dexterous manipulation.Science Robotics, 11(113):eaea6201, 2026

    Jose Barreiros, Andrew Beaulieu, Aditya Bhat, et al. A careful examination of large behavior models for multitask dexterous manipulation.Science Robotics, 11(113):eaea6201, 2026. 33

    2026

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.