Pith. sign in

REVIEW 3 major objections 5 minor 67 references

High-fidelity handheld demonstrations can replace the real-robot teleoperation anchor at post-training time.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 01:10 UTC pith:4IWSNZ3G

load-bearing objection Solid system paper with a genuinely new UMI-only post-training comparison, but the headline 'matches teleoperation' overstates what 40-rollout conditions can support; the data show parity cannot be rejected, not that parity holds. the 3 major comments →

arxiv 2607.25895 v1 pith:4IWSNZ3G submitted 2026-07-28 cs.RO cs.CVcs.LG

HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alone

Simple AI: Yuteng Wei , Jinming Ma , Jiawei Wang , Weitao Zhou , Yushen Zuo , Ke Rui , Minglei Li , Jinhao Zhang
show 9 more authors
Zhikang Pan Xiang Wang Haoran Jia Huan Du Zicheng Zeng Jun Ma Guiyu Qin Di Zhang Xiaofei Li
This is my paper
classification cs.RO cs.CVcs.LG MSC 68T40
keywords robot learningimitation learningUMIuniversal manipulation interfacedata collectionvision-language-action modelsworld action modelsbimanual manipulation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Robot-free demonstrations have mostly been treated as pre-training material, with a small real-robot teleoperation set added later to make a policy deployable. This paper argues that the real bottleneck is the fidelity of the robot-free data, and it builds a handheld capture system—offline stereo SLAM on the head, hardware-synchronized cameras, ultra-wide views, and a full-palm glove—that records trajectories accurate to about 3 mm. Policies post-trained only on these demonstrations deploy directly on a real robot and match in-domain teleoperation across three backbones from the vision-language-action and world-action-model families, with success-rate differences of -2.5, +3.1, and -0.6 percentage points on four bimanual tasks. Pre-training on the same robot-free corpus further reduces action error on unseen tasks and raises final real-robot success, and a curated 2,000-hour subset is released. The claim, if true, means robot-free UMI data can finish a policy, not just start it.

Core claim

The paper's central claim is that data fidelity, not the robot-free setting, is what has kept handheld UMI (Universal Manipulation Interface) demonstrations in a pre-training-only role. With a portable capture device engineered for trajectory accuracy, native inter-gripper relative pose, microsecond synchronization, and roughly 200 degrees of camera coverage per hand, the authors post-train policies solely on such demonstrations and deploy them directly on a real bimanual robot. Across four tasks—stain wiping, shirt folding, remote insertion, and produce sorting—and three backbone policies spanning the VLA and WAM families, UMI-only post-training matches teleoperation post-training, with suc

What carries the argument

The mechanism is the HiFi-UMI capture-and-processing pipeline combined with a policy-agnostic action interface. Pose fidelity comes from head-mounted offline stereo-inertial SLAM with per-hand marker cubes observed in the head frame, which measures the two grippers' relative pose natively rather than reconstructing it; a shared GPIO hardware trigger synchronizes all sensors to under 40 microseconds; and two non-parallel fisheye cameras per hand give roughly 200 degrees of coverage. The action interface is a chunk-anchored, 20-channel bimanual representation—relative end-effector pose increments plus absolute gripper opening—that each backbone's native action tensor absorbs. Every capture the

Load-bearing premise

The results stand or fall on the premise that a 20-channel, position-only description of each demonstration—relative end-effector pose increments plus absolute gripper opening—carries everything the policy needs, including for contact-rich tasks such as wiping, folding, and insertion; no force, torque, or compliance information is recorded or trained.

What would settle it

Run the same UMI-versus-teleoperation protocol on a force-critical task, such as maintaining steady wipe pressure on a sensor-equipped surface or torque-limited insertion, with matched trajectory counts and the same 20-channel position-only action format; if the teleoperation-trained policy beats the UMI-trained one by a margin that cannot be explained by the 40-rollout noise band, then the position-only action interface (or the fidelity of force-sensitive kinematics in handheld capture) is the failing component.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the claim holds, robot-free UMI data can serve as post-training supervision, so deployment-ready policies no longer require a teleoperated real-robot anchor for the target task.
  • Collection can be parallelized across sites and operators without the target robot, making it substantially cheaper to grow deployment-grade task data.
  • Pre-training on the same handheld corpus improves both data efficiency and final real-robot success, shifting the efficiency-performance frontier rather than just adding scale.
  • Since parity holds across reactive vision-language-action models and a predictive world-action model, the data source—not the architecture family—appears to be the relevant variable.
  • The practical comparison is between pipelines, not per-trajectory efficiency: UMI used roughly ten times more demonstrations per task than teleoperation, so the advantage is cost and scalability, not sample efficiency.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A reader should not trust per-task orderings at 40 rollouts per task, where one success swing is 2.5 points; the paper's own aggregate framing is the level at which parity is claimed.
  • The paper validates fidelity as a joint design rather than isolating it; deliberately degrading synchronization, field of view, or inter-gripper pose while holding sample count and scene coverage fixed would identify which fidelity property actually carries the result.
  • The action interface is position-only, with no force or torque channel; contact-rich tasks where force information is essential, such as torque-limited screwing or compliant insertion, are the natural place to test whether UMI parity survives.
  • The released corpus is replay-validated for one target arm; transferring to a different robot would require new retargeting and replay validation, so the deployment-grade status is embodiment-specific until re-validated.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents HiFi-UMI, a robot-free data-capture system and pipeline that combines head-mounted offline stereo-inertial SLAM, native inter-gripper relative pose, microsecond-level GPIO synchronization, and ultra-wide six-view sensing. The authors report 3 mm end-effector accuracy, a 96% cumulative replay-validity yield, and the release of a 2,000-hour curated dataset. The central empirical claim is "zero-robot post-training": policies post-trained solely on HiFi-UMI demonstrations match policies post-trained on in-domain real-robot teleoperation, on four tabletop tasks and three backbones (StarVLA-QwenPI, OpenPI-π0.5, LingBot-VA), with aggregate success-rate differences of −2.5, +3.1, and −0.6 percentage points. A second result shows that 4,000 hours of HiFi-UMI pre-training reduces held-out action error by 41% on ten unseen tasks and improves downstream real-robot success by 18.1 points on one backbone. The paper argues that fidelity, not the robot-free setting, is the limiting factor for UMI-style data.

Significance. The hardware and data contributions are substantial. The capture system addresses well-known UMI fidelity bottlenecks (SLAM drift, reconstructed inter-gripper pose, software time alignment, narrow field of view), and the automated reconstruction/replay validation pipeline is a practical step toward scalable deployment-grade data. The evaluation protocol is unusually careful in several respects: a frozen benchmark, separated evaluator roles, randomized policy order, recorded termination reasons, and an asymmetric setup that evaluates UMI policies under scene shift while teleoperation is collected in the evaluation scene. The 2,000-hour open dataset is a valuable community resource. If the parity claim were statistically supported, the result would be significant for the field: robot-free UMI data could replace the real-robot teleoperation anchor at post-training time. However, the headline claim is currently supported only by aggregate point estimates without confidence intervals or equivalence tests, and the comparison is not sample-matched (3,200 UMI vs. ~300 teleoperation trajectories). The system contribution is strong, but the central evidential claim needs substantially more w

major comments (3)
  1. [§6.2, Fig. 9, and §7 (Limitations)] The headline claim "matches in-domain teleoperation" is an equivalence claim, but the design does not support it at the stated precision. Each task–policy pair uses 40 rollouts (one success = 2.5 pp), and aggregate comparisons rest on 160 rollouts per condition. For success rates near 64%, the standard error of the difference is about 5.4 pp; observed differences of −2.5, +3.1, and −0.6 pp are all within ±1 SE, but so is a true gap of 10–15 pp. The paper's own Limitations section concedes the task-level resolution problem, yet the aggregate parity claim inherits the same problem: "within sampling noise" is an absence-of-evidence statement, not evidence for parity. Report confidence intervals on the aggregate differences and ideally a pre-specified equivalence test with a margin (e.g., ±5 or ±10 pp); otherwise the abstract and conclusion should say "no detectable difference in this protoc
  2. [§6.1.3 and §6.2] The comparison is not sample-matched: UMI post-training uses 3,200 demonstrations per task vs. ~300 teleoperation trajectories. The paper explicitly frames this as a comparison of practical pipelines, but the abstract and conclusion state that "UMI data alone" matches teleoperation, which readers will reasonably take as a claim about data source equivalence. With a 10× sample-size advantage, the point estimates could reflect scale effects rather than equivalence of the two data sources. To support the causal "remove the real-robot anchor" claim, add at least one matched-sample condition (e.g., UMI subsampled to ~300 trajectories or teleoperation scaled up) or clearly restrict all conclusions to the practical-pipeline comparison and remove the stronger causal wording. This is the second load-bearing weakness.
  3. [§5, Eq. (2); §6.1.2] The action representation is purely kinematic: 3 + 6 + 1 = 10 channels per arm (relative pose increment + absolute gripper opening), with no force, torque, or compliance channel. The robot hardware is force-controlled, but policies emit pose targets only. For contact-rich tasks such as Stain Wiping, Shirt Folding, and Remote Insertion, force interaction information is not represented in either the UMI or teleoperation condition, so this does not invalidate the UMI-vs-teleop parity comparison (both use the same interface). However, the general claim that HiFi-UMI yields "deployable manipulation policies" is thereby limited to tasks solvable by kinematic trajectory tracking. The paper should add force/torque channels to the interface or explicitly scope the claim. This is a substantive limitation of the deployment claim as stated.
minor comments (5)
  1. [§6.2.2, Fig. 12] The ground-truth-video diagnostic reports overlapping bootstrap confidence intervals between UMI→Real and Real→Real. The text says the decoders achieve "comparable" accuracy; this should be phrased as "we cannot rule out a difference" unless a formal equivalence or non-inferiority analysis is added.
  2. [§6.3, Eq. (9), Fig. 13] The power-law fit to a single learning curve with correlated checkpoints is descriptive, not a dataset-scaling law. The caption already notes "exposure scaling," but the abstract's "power-law trend" may overstate it; consider "exposure-driven trend" consistently.
  3. [§3.1.2] The full-palm glove is said to preserve "natural force and contact," but no force measurement is made or released. This wording should be softened to avoid implying force channels are captured.
  4. [Tab. 1] Positional errors from different systems use different references and motion profiles; the caption already cautions against direct comparison, but the table's visual alignment may still invite over-reading. Consider adding a prominent footnote on the first use of the table.
  5. [§5.4] The description of training variants is clear, but the relationship between C1 (scratch action head) and C7 (UMI-pretrained init.) could be stated even more explicitly in the main text, since the "matched post-training data" comparison in Fig. 15 depends on it.

Circularity Check

0 steps flagged

No significant circularity: the central parity claim rests on an empirical benchmark comparison, not on a derivation from its own inputs.

full rationale

The paper's main claims are empirical: UMI-only post-training is compared with teleoperation post-training on three backbones under a frozen benchmark, and the reported differences are observed success rates, not quantities forced by construction. The action representation in Eq. (2) is a shared physical convention applied identically to both UMI and teleoperation data, so it does not encode the outcome. The only fitted model is the power-law scaling relation in Eq. (9), which is explicitly fit to the held-out learning curve and reported as a fit ('The fitted exponent is α=0.268'); it is not used to generate the parity claim. The corpus is used for both pre-training and post-training, but this is an experimental design choice rather than a circular step, and the paper separately controls the initialization comparison. Citations to public checkpoints (OpenPI-π0.5, LingBot-VA, StarVLA) are external baselines, not uniqueness theorems or ansatze imported to force the result. The paper also discloses the sample-count asymmetry and the lack of a fidelity ablation as limitations, further indicating that the conclusions are not tautological. Any concerns about statistical power or force-channel omission are external-validity/correctness risks, not circularity.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 0 invented entities

No new physical entities are postulated. The central claim rests on a set of engineering and measurement assumptions about SLAM accuracy, simulation-replay validity, the sufficiency of position-only action labels, and embodiment matching. The only fitted quantities are the scaling-law exponents, which are descriptive curve fits rather than independent predictions.

free parameters (2)
  • Power-law scaling parameters (L∞, A, α) = L∞ and A not reported; α=0.268 (pre-training), α=0.095 (OOD); family exponents 0.134/0.105/0.072; R²=0.993/0.836/0.520/0
    Eq. (9) fits held-out action MSE as L(S)=L∞+A S^{-α}; all three parameters are fitted to the learning curve. The abstract calls this a scaling trend, but no parameter-free derivation or out-of-fit prediction is provided.
  • Flow-matching sampling hyperparameters = Beta(1.5,1.0); s=0.999; 8 Euler steps; H=20, H_exec=10
    Hand-chosen recipe in Sec. 5.1; not fitted to the central parity claim but part of the training configuration.
axioms (5)
  • domain assumption Head-mounted offline stereo-inertial SLAM plus fiducial markers yields ~3 mm workspace-local end-effector error over a ~2 m workspace without external tracking.
    Reported in Sec. 3.4/Tab. 2 as a single number; the measurement protocol (motion profile, number of trials, marker geometry) is not given. The parity claim requires this accuracy to hold during normal capture.
  • domain assumption Simulation replay with the whole-body controller certifies a reconstructed trajectory as a valid robot-executable demonstration.
    Sec. 3.3.3 applies replay validation to all data and reports 98% pass; replay feasibility is treated as equivalent to deployability, but the real-robot validity of replay-pass trajectories is not separately measured.
  • domain assumption The 20-channel pose-plus-opening action interface (Eq. 2) carries all information needed to learn contact-rich tasks; no force/torque channel is used.
    Sec. 5 defines 10 active channels per arm (3 translation, 6 rotation, 1 gripper). Force/torque is never recorded; force-controlled arms receive pose targets. Contact-rich parity depends on this sufficiency.
  • domain assumption The deployment robot's gripper and wrist cameras exactly match the HiFi-UMI capture rig, so the embodiment gap reduces to arm kinematics.
    Sec. 6.1.1 states that the platform 'carries the same gripper and four wrist cameras'; physical identity is asserted, not calibrated or measured.
  • domain assumption Native inter-gripper relative pose from a single head frame is as accurate as per-gripper pose.
    Sec. 3.1.1: both marker cubes are observed by the head cameras, but occlusion, marker loss, and wrist motion are not quantified.

pith-pipeline@v1.3.0-alltime-deepseek · 28193 in / 14567 out tokens · 123166 ms · 2026-08-01T01:10:11.065283+00:00 · methodology

0 comments
read the original abstract

Learning deployable manipulation policies is bottlenecked by the scarcity of data that is both high-fidelity and scalable. Real-robot teleoperation is accurate but costly to scale; robot-free UMI capture scales readily, and current practice uses the resulting data mainly for pre-training, adding a small real-robot "anchor" at post-training. We ask whether raising the fidelity of robot-free UMI data, rather than shrinking the real-robot fraction, can remove that anchor. We present HiFi-UMI, a portable UMI data-production system co-designed for trajectory accuracy, inter-gripper relative pose, synchronization, and field of view: head-mounted offline stereo-inertial SLAM, native rather than reconstructed relative pose, a shared microsecond GPIO trigger, and two wide-angle cameras per hand covering ~200 degrees. It reaches 3 mm workspace-local end-effector accuracy without external tracking infrastructure. Using this corpus, we demonstrate zero-robot post-training: a policy post-trained solely on HiFi-UMI demonstrations deploys directly on a real robot and matches in-domain teleoperation across three backbones spanning the vision-language-action and world-action-model families, with success-rate differences of -2.5, +3.1, and -0.6 percentage points on StarVLA-QwenPI, OpenPI-pi_0.5, and LingBot-VA; the strongest policy reaches 85% on a precision insertion task, even though the teleoperation baseline is collected in the evaluation scene and no HiFi-UMI trajectory is. Pre-training on 4,000 hours from the same corpus lowers action error on ten unseen tasks by 41% and, on StarVLA-QwenPI, raises real-robot success by a further 18.1 percentage points. We open-source HiFi-UMI-2K, 2,000 hours of microsecond-synchronized, ultra-wide-FoV demonstrations, each automatically reconstructed and validated through simulation replay, as a large-scale, high-fidelity resource for the robot-learning community.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 53 linked inside Pith

  1. [1]

    RT-1: Robotics transformer for real-world control at scale

    Anthony Brohan, Noah Brown, Justice Carbajal, et al. RT-1: Robotics transformer for real-world control at scale. InRobotics: Science and Systems (RSS), 2023

    2023

  2. [2]

    BridgeData V2: A dataset for robot learning at scale

    Homer Walke, Kevin Black, Abraham Lee, et al. BridgeData V2: A dataset for robot learning at scale. In Conference on Robot Learning (CoRL), 2023. arXiv:2308.12952

    Pith/arXiv arXiv 2023

  3. [3]

    DROID: A large-scale in-the-wild robot manipulation dataset

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, et al. DROID: A large-scale in-the-wild robot manipulation dataset. InRobotics: Science and Systems (RSS), 2024. arXiv:2403.12945

    Pith/arXiv arXiv 2024

  4. [4]

    AgiBot World Colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems.arXiv preprint arXiv:2503.06669, 2025

    AgiBot-World-Contributors, Qingwen Bu, Jisong Cai, et al. AgiBot World Colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems.arXiv preprint arXiv:2503.06669, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    RoboMIND: Benchmark on multi-embodiment intelligence normative data for robot manipulation.arXiv preprint arXiv:2412.13877, 2024

    Kun Wu, Chengkai Hou, Jiaming Liu, et al. RoboMIND: Benchmark on multi-embodiment intelligence normative data for robot manipulation.arXiv preprint arXiv:2412.13877, 2024

    Pith/arXiv arXiv 2024

  6. [6]

    Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots

    Cheng Chi, Zhenjia Xu, Chuer Pan, et al. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots. InRobotics: Science and Systems (RSS), 2024. arXiv:2402.10329

    Pith/arXiv arXiv 2024

  7. [7]

    FastUMI: A scalable and hardware-independent universal manip- ulation interface with dataset.arXiv preprint arXiv:2409.19499, 2024

    Zhaxizhuoma, Kehui Liu, Chuyue Guan, et al. FastUMI: A scalable and hardware-independent universal manip- ulation interface with dataset.arXiv preprint arXiv:2409.19499, 2024

    Pith/arXiv arXiv 2024

  8. [8]

    Data scaling laws in imitation learning for robotic manipulation

    Fanqi Lin, Yingdong Hu, Pingyue Sheng, et al. Data scaling laws in imitation learning for robotic manipulation. InInternational Conference on Learning Representations (ICLR), 2025. arXiv:2410.18647

    Pith/arXiv arXiv 2025

  9. [9]

    AirExo-2: Scaling up generalizable robotic imitation learning with low-cost exoskeletons.arXiv preprint arXiv:2503.03081, 2025

    Hongjie Fang, Chenxi Wang, Yiming Wang, et al. AirExo-2: Scaling up generalizable robotic imitation learning with low-cost exoskeletons.arXiv preprint arXiv:2503.03081, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    GR00T N1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, et al. GR00T N1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    H-RDT: Human manipulation enhanced bimanual robotic manip- ulation.arXiv preprint arXiv:2507.23523, 2025

    Hongzhe Bi, Lingxuan Wu, Tianwei Lin, et al. H-RDT: Human manipulation enhanced bimanual robotic manip- ulation.arXiv preprint arXiv:2507.23523, 2025

    Pith/arXiv arXiv 2025

  12. [12]

    John, et al

    Qiyuan Zeng, Chengmeng Li, Jude St. John, et al. ActiveUMI: Robotic manipulation with active perception from robot-free human demonstrations.arXiv preprint arXiv:2510.01607, 2025

    arXiv 2025

  13. [13]

    XRZero-G0: Pushing the frontier of dexterous robotic manipulation with interfaces, quality and ratios.arXiv preprint arXiv:2604.13001, 2026

    James Wang, Primo Pu, Zephyr Fung, et al. XRZero-G0: Pushing the frontier of dexterous robotic manipulation with interfaces, quality and ratios.arXiv preprint arXiv:2604.13001, 2026

    Pith/arXiv arXiv 2026

  14. [14]

    RDT2: Exploring the scaling limit of UMI data towards zero-shot cross-embodiment generalization.arXiv preprint arXiv:2602.03310, 2026

    Songming Liu, Bangguo Li, Kai Ma, et al. RDT2: Exploring the scaling limit of UMI data towards zero-shot cross-embodiment generalization.arXiv preprint arXiv:2602.03310, 2026

    arXiv 2026

  15. [15]

    RH20T: A comprehensive robotic dataset for learning diverse skills in one-shot.arXiv preprint arXiv:2307.00595, 2023

    Hao-Shu Fang, Hongjie Fang, Zhenyu Tang, et al. RH20T: A comprehensive robotic dataset for learning diverse skills in one-shot.arXiv preprint arXiv:2307.00595, 2023

    Pith/arXiv arXiv 2023

  16. [16]

    Open X-embodiment: Robotic learning datasets and RT-X models

    Open X-Embodiment Collaboration, Abby O’Neill, Abdul Rehman, et al. Open X-embodiment: Robotic learning datasets and RT-X models. InIEEE International Conference on Robotics and Automation (ICRA), 2024. arXiv:2310.08864. 30

    Pith/arXiv arXiv 2024

  17. [17]

    RoboMIND 2.0: A multimodal, bimanual mobile manipulation dataset for generalizable embodied intelligence.arXiv preprint arXiv:2512.24653, 2025

    Chengkai Hou, Kun Wu, Jiaming Liu, et al. RoboMIND 2.0: A multimodal, bimanual mobile manipulation dataset for generalizable embodied intelligence.arXiv preprint arXiv:2512.24653, 2025

    arXiv 2025

  18. [18]

    Ego4D: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, et al. Ego4D: Around the world in 3,000 hours of egocentric video. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. arXiv:2110.07058

    Pith/arXiv arXiv 2022

  19. [19]

    Ego-Exo4D: Understanding skilled human activity from first- and third-person perspectives

    Kristen Grauman, Andrew Westbury, Lorenzo Torresani, et al. Ego-Exo4D: Understanding skilled human activity from first- and third-person perspectives. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. arXiv:2311.18259

    Pith/arXiv arXiv 2024

  20. [20]

    Yoon, et al

    Ryan Hoque, Peide Huang, David J. Yoon, et al. EgoDex: Learning dexterous manipulation from large-scale egocentric video.arXiv preprint arXiv:2505.11709, 2025

    Pith/arXiv arXiv 2025

  21. [21]

    FastUMI-100K: Advancing data-driven robotic manipulation with a large-scale UMI-style dataset.arXiv preprint arXiv:2510.08022, 2025

    Kehui Liu, Zhongjie Jia, Yang Li, et al. FastUMI-100K: Advancing data-driven robotic manipulation with a large-scale UMI-style dataset.arXiv preprint arXiv:2510.08022, 2025

    arXiv 2025

  22. [22]

    VISTA: Vision-grounded and physics-validated adaptation of UMI data for VLA training.arXiv preprint arXiv:2606.04708, 2026

    Siyuan Yang, Linzheng Guo, Ouyang Lu, et al. VISTA: Vision-grounded and physics-validated adaptation of UMI data for VLA training.arXiv preprint arXiv:2606.04708, 2026

    Pith/arXiv arXiv 2026

  23. [23]

    DexCap: Scalable and portable mocap data collection system for dexterous manipulation

    Chen Wang, Haochen Shi, Weizhuo Wang, et al. DexCap: Scalable and portable mocap data collection system for dexterous manipulation. InRobotics: Science and Systems (RSS), 2024. arXiv:2403.07788

    Pith/arXiv arXiv 2024

  24. [24]

    DexUMI: Using human hand as the universal manipulation interface for dexterous manipulation

    Mengda Xu, Han Zhang, Yifan Hou, et al. DexUMI: Using human hand as the universal manipulation interface for dexterous manipulation. InConference on Robot Learning (CoRL), 2025. arXiv:2505.21864

    arXiv 2025

  25. [25]

    DexWild: Dexterous human interactions for in-the-wild robot policies.Robotics: Science and Systems (RSS), 2025

    Tony Tao, Mohan Kumar Srirama, Jason Jingzhou Liu, et al. DexWild: Dexterous human interactions for in-the-wild robot policies.Robotics: Science and Systems (RSS), 2025

    2025

  26. [26]

    ARCap: Collecting high-quality human demonstrations for robot learning with augmented reality feedback

    Sirui Chen, Chen Wang, Kaden Nguyen, et al. ARCap: Collecting high-quality human demonstrations for robot learning with augmented reality feedback. InIEEE International Conference on Robotics and Automation (ICRA), 2025. arXiv:2410.08464

    Pith/arXiv arXiv 2025

  27. [27]

    AirExo: Low-cost exoskeletons for learning whole-arm manipulation in the wild

    Hongjie Fang, Hao-Shu Fang, Yiming Wang, et al. AirExo: Low-cost exoskeletons for learning whole-arm manipulation in the wild. InIEEE International Conference on Robotics and Automation (ICRA), 2024. arXiv:2309.14975

    Pith/arXiv arXiv 2024

  28. [28]

    EgoMimic: Scaling imitation learning via egocentric video

    Simar Kareer, Dhruv Patel, Ryan Punamiya, et al. EgoMimic: Scaling imitation learning via egocentric video. InIEEE International Conference on Robotics and Automation (ICRA), 2025. arXiv:2410.24221

    Pith/arXiv arXiv 2025

  29. [29]

    RT-2: Vision-language-action models transfer web knowledge to robotic control

    Anthony Brohan, Noah Brown, Justice Carbajal, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. InConference on Robot Learning (CoRL), 2023. arXiv:2307.15818

    Pith/arXiv arXiv 2023

  30. [30]

    OpenVLA: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, et al. OpenVLA: An open-source vision-language-action model. InConference on Robot Learning (CoRL), 2024. arXiv:2406.09246

    Pith/arXiv arXiv 2024

  31. [31]

    Octo: An open-source generalist robot policy

    Octo Model Team, Dibya Ghosh, Homer Walke, et al. Octo: An open-source generalist robot policy. InRobotics: Science and Systems (RSS), 2024. arXiv:2405.12213

    Pith/arXiv arXiv 2024

  32. [32]

    arXiv preprint arXiv:2410.24164, 2024

    Kevin Black, Noah Brown, Danny Driess, et al.π0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    Physical Intelligence, Kevin Black, Noah Brown, et al.π0.5: A vision-language-action model with open-world generalization.arXiv preprint arXiv:2504.16054, 2025

    Pith/arXiv arXiv 2025

  34. [34]

    GR-3 technical report.arXiv preprint arXiv:2507.15493, 2025

    ByteDance Seed. GR-3 technical report.arXiv preprint arXiv:2507.15493, 2025

    Pith/arXiv arXiv 2025

  35. [35]

    Gemini robotics: Bringing AI into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Google DeepMind. Gemini robotics: Bringing AI into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Pith/arXiv arXiv 2025

  36. [36]

    SmolVLA: A vision-language-action model for affordable and efficient robotics.arXiv preprint arXiv:2506.01844, 2025

    Mustafa Shukor, Dana Aubakirova, Francesco Capuano, et al. SmolVLA: A vision-language-action model for affordable and efficient robotics.arXiv preprint arXiv:2506.01844, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    FAST: Efficient action tokenization for vision-language-action models.arXiv preprint arXiv:2501.09747, 2025

    Karl Pertsch, Kyle Stachowicz, Brian Ichter, et al. FAST: Efficient action tokenization for vision-language-action models.arXiv preprint arXiv:2501.09747, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    Fine-tuning vision-language-action models: Optimizing speed and success.Robotics: Science and Systems (RSS), 2025

    Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and success.Robotics: Science and Systems (RSS), 2025. arXiv:2502.19645

    Pith/arXiv arXiv 2025

  39. [39]

    A pragmatic VLA foundation model.arXiv preprint arXiv:2601.18692, 2026

    Wei Wu, Fan Lu, Yunnan Wang, et al. A pragmatic VLA foundation model.arXiv preprint arXiv:2601.18692, 2026. 31

    Pith/arXiv arXiv 2026

  40. [40]

    Qwen-VLA: Unifying vision-language-action modeling across tasks, environments, and robot embodiments.arXiv preprint arXiv:2605.30280, 2026

    Qiuyue Wang, Mingsheng Li, Jian Guan, et al. Qwen-VLA: Unifying vision-language-action modeling across tasks, environments, and robot embodiments.arXiv preprint arXiv:2605.30280, 2026

    Pith/arXiv arXiv 2026

  41. [41]

    Qwen-RobotManip technical report: Alignment unlocks scale for robotic manipulation foundation models.arXiv preprint arXiv:2606.17846, 2026

    Haoqi Yuan, Zhixuan Liang, Anzhe Chen, et al. Qwen-RobotManip technical report: Alignment unlocks scale for robotic manipulation foundation models.arXiv preprint arXiv:2606.17846, 2026

    Pith/arXiv arXiv 2026

  42. [42]

    Unleashing large-scale video generative pre-training for visual robot manipulation

    Hongtao Wu, Ya Jing, Chilam Cheang, et al. Unleashing large-scale video generative pre-training for visual robot manipulation. InInternational Conference on Learning Representations (ICLR), 2024. arXiv:2312.13139

    Pith/arXiv arXiv 2024

  43. [43]

    GR-2: A generative video-language-action model with web-scale knowledge for robot manipulation.arXiv preprint arXiv:2410.06158, 2024

    Chi-Lam Cheang, Guangzeng Chen, Ya Jing, et al. GR-2: A generative video-language-action model with web-scale knowledge for robot manipulation.arXiv preprint arXiv:2410.06158, 2024

    Pith/arXiv arXiv 2024

  44. [44]

    Learning universal policies via text-guided video generation

    Yilun Du, Mengjiao Yang, Bo Dai, et al. Learning universal policies via text-guided video generation. InAdvances in Neural Information Processing Systems (NeurIPS), 2023. arXiv:2302.00111

    Pith/arXiv arXiv 2023

  45. [45]

    Video prediction policy: A generalist robot policy with predic- tive visual representations

    Yucheng Hu, Yanjiang Guo, Pengchao Wang, et al. Video prediction policy: A generalist robot policy with predic- tive visual representations. InInternational Conference on Machine Learning (ICML), 2025. arXiv:2412.14803

    Pith/arXiv arXiv 2025

  46. [46]

    World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026

    Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, et al. World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026

    Pith/arXiv arXiv 2026

  47. [47]

    WorldVLA: Towards autoregressive action world model.arXiv preprint arXiv:2506.21539, 2025

    Jun Cen, Chaohui Yu, Hangjie Yuan, et al. WorldVLA: Towards autoregressive action world model.arXiv preprint arXiv:2506.21539, 2025

    Pith/arXiv arXiv 2025

  48. [48]

    Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

    Lin Li, Qihang Zhang, Yiming Luo, et al. Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

    Pith/arXiv arXiv 2026

  49. [49]

    StarVLA: A lego-like codebase for vision-language-action model developing.arXiv preprint arXiv:2604.05014, 2026

    StarVLA Community. StarVLA: A lego-like codebase for vision-language-action model developing.arXiv preprint arXiv:2604.05014, 2026. Code available at: https://github.com/starVLA/starVLA

    Pith/arXiv arXiv 2026

  50. [50]

    VINS-Mono: A robust and versatile monocular visual-inertial state estimator.IEEE Transactions on Robotics, 34(4):1004–1020, 2018

    Tong Qin, Peiliang Li, and Shaojie Shen. VINS-Mono: A robust and versatile monocular visual-inertial state estimator.IEEE Transactions on Robotics, 34(4):1004–1020, 2018

    2018

  51. [51]

    Gómez Rodríguez, et al

    Carlos Campos, Richard Elvira, Juan J. Gómez Rodríguez, et al. ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multimap SLAM.IEEE Transactions on Robotics, 37(6):1874–1890, 2021. arXiv:2007.11898

    Pith/arXiv arXiv 2021

  52. [52]

    AprilTag: A robust and flexible visual fiducial system

    Edwin Olson. AprilTag: A robust and flexible visual fiducial system. InIEEE International Conference on Robotics and Automation (ICRA), pp. 3400–3407, 2011

    2011

  53. [53]

    VersaVIS—an open versatile multi-camera visual-inertial sensor suite.Sensors, 20(5):1439, 2020

    Florian Tschopp, Michael Riner, Marius Fehr, et al. VersaVIS—an open versatile multi-camera visual-inertial sensor suite.Sensors, 20(5):1439, 2020. arXiv:1912.02469

    Pith/arXiv arXiv 2020

  54. [54]

    DAS Fingers: A high-precision multimodal data-collection device for embodied AI.https: //www.genrobot.ai/products/finger, 2025

    GenRobot AI. DAS Fingers: A high-precision multimodal data-collection device for embodied AI.https: //www.genrobot.ai/products/finger, 2025. Product page. Accessed: 2026-07-08

    2025

  55. [55]

    TacUMI: A multi-modal universal manipulation interface for contact-rich tasks.arXiv preprint arXiv:2601.14550, 2026

    Tailai Cheng, Kejia Chen, Lingyun Chen, et al. TacUMI: A multi-modal universal manipulation interface for contact-rich tasks.arXiv preprint arXiv:2601.14550, 2026

    arXiv 2026

  56. [56]

    Fácil, Javier Civera, et al

    Berta Bescos, José M. Fácil, Javier Civera, et al. DynaSLAM: Tracking, mapping, and inpainting in dynamic scenes.IEEE Robotics and Automation Letters, 3(4):4076–4083, 2018. arXiv:1806.05620

    Pith/arXiv arXiv 2018

  57. [57]

    UMI on legs: Making manipulation policies mobile with manipulation- centric whole-body controllers.arXiv preprint arXiv:2407.10353, 2024

    Huy Ha, Yihuai Gao, Zipeng Fu, et al. UMI on legs: Making manipulation policies mobile with manipulation- centric whole-body controllers.arXiv preprint arXiv:2407.10353, 2024

    Pith/arXiv arXiv 2024

  58. [58]

    RoboVQA: Multimodal long-horizon reasoning for robotics

    Pierre Sermanet, Tianli Ding, Jeffrey Zhao, et al. RoboVQA: Multimodal long-horizon reasoning for robotics. In IEEE International Conference on Robotics and Automation (ICRA), pp. 645–652, 2024. arXiv:2311.00899

    Pith/arXiv arXiv 2024

  59. [59]

    On the continuity of rotation representations in neural networks

    Yi Zhou, Connelly Barnes, Jingwan Lu, et al. On the continuity of rotation representations in neural networks. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. arXiv:1812.07035

    Pith/arXiv arXiv 2019

  60. [60]

    Qwen3 Technical Report.arXiv preprint arXiv:2505.09388, 2025

    Qwen Team. Qwen3 Technical Report.arXiv preprint arXiv:2505.09388, 2025

    Pith/arXiv arXiv 2025

  61. [61]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, et al. Flow matching for generative modeling. InInternational Conference on Learning Representations (ICLR), 2023. arXiv:2210.02747

    Pith/arXiv arXiv 2023

  62. [62]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InIEEE/CVF International Conference on Computer Vision (ICCV), 2023. arXiv:2212.09748. 32

    Pith/arXiv arXiv 2023

  63. [63]

    Zhao, Vikash Kumar, Sergey Levine, et al

    Tony Z. Zhao, Vikash Kumar, Sergey Levine, et al. Learning fine-grained bimanual manipulation with low-cost hardware. InRobotics: Science and Systems (RSS), 2023

    2023

  64. [64]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Siyuan Feng, Yilun Du, et al. Diffusion policy: Visuomotor policy learning via action diffusion. In Robotics: Science and Systems (RSS), 2023

    2023

  65. [65]

    PaliGemma: A versatile 3b VLM for transfer.arXiv preprint arXiv:2407.07726, 2024

    Lucas Beyer, Andreas Steiner, André Susano Pinto, et al. PaliGemma: A versatile 3b VLM for transfer.arXiv preprint arXiv:2407.07726, 2024

    Pith/arXiv arXiv 2024

  66. [66]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations (ICLR), 2019. arXiv:1711.05101

    Pith/arXiv arXiv 2019

  67. [67]

    A careful examination of large behavior models for multitask dexterous manipulation.Science Robotics, 11(113):eaea6201, 2026

    Jose Barreiros, Andrew Beaulieu, Aditya Bhat, et al. A careful examination of large behavior models for multitask dexterous manipulation.Science Robotics, 11(113):eaea6201, 2026. 33

    2026