Pith. sign in

REVIEW 3 major objections 5 minor 46 references

Fluent expert demonstrations under-supervise the critical alignment phase, and a compact dynamic feature distilled from local video recovers most of the deliberate-demonstration gain on six fine-manipulation tasks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 11:16 UTC pith:GLDXNWHN

load-bearing objection A conceptually nice paper whose central quantitative claim is not yet statistically supported: the 12.2 pp STAIR gain is within sampling noise at n=15/task, but the idea deserves a serious look. the 3 major comments →

arxiv 2606.15587 v2 pith:GLDXNWHN submitted 2026-06-14 cs.RO

Perfect Demo Makes Poor Teacher: Learning Robust Alignment from Critical Motion Segments

Mingyu Liu , Zeju Li , Jiuhe Shu , Hanqing Wang , Yuhao Chao , Hao Chen , Chunhua Shen This is my paper
classification cs.RO
keywords robot manipulationimitation learningvision-language-action modelsfine-grained manipulationdemonstration datadynamic featuresalignmentrepresentation learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that fluent expert demonstrations can be poor teachers for fine-grained manipulation: skilled operators compress the decisive alignment and recovery motions into a few frames, while the imitation loss weights every frame equally, so the policy is starved of supervision exactly where precision determines success. To address this at the representation level, the paper introduces STAIR, a compact spatio-temporal dynamic feature that compresses a short video neighborhood around each action chunk into a few tokens and uses them to condition the action expert. Trained on fluent data alone, STAIR lifts policy success from 50.0% to 62.2% across six real-world tasks, approaching the 64.4% achieved by slow deliberate demonstrations, and combining STAIR with deliberate data reaches 70.0%. The paper also shows that data-level remedies (slowing down near alignment, resampling critical segments) help mainly by expanding coverage of recovery states, not by reweighting existing frames. These results suggest that robot data should be optimized for machine learnability rather than human efficiency alone.

Core claim

The central claim is that fluent expert demonstrations fail as teachers for fine manipulation because the alignment/recovery phase occupies a small fraction of the trajectory and the uniform imitation loss allocates almost all its weight to easy transport, leaving the decisive corrections under-supervised. To address this, the paper introduces STAIR, which distills short-horizon motion from a brief video neighborhood into compact dynamic tokens and feeds them to the action expert through cross-attention, while a query module on the vision-language model learns to predict these tokens from the current frame alone. With this dynamic conditioning, a policy trained purely on fluent data recovers

What carries the argument

STAIR (Spatio-Temporal feature As an Interface for Robot learning) is a compact dynamic feature. For each training sample, a short future observation neighborhood is encoded by a frozen video encoder and compressed by a trainable 3D convolution plus attention-based projector into a few tokens; after an MLP projection, these tokens condition a flow-matching action expert via cross-attention. In a second stage, a query module on the vision-language model's hidden states is trained to predict the frozen dynamic condition from the current observation and instruction, so at deployment the video neighborhood is no longer needed and the standard single-image VLA interface is preserved. The dynamic

Load-bearing premise

The central comparison assumes that 15 evaluation trials per task can distinguish the reported success-rate differences, and that the STAIR gain comes from the dynamic feature rather than from the extra trainable query and projector modules.

What would settle it

Run the six tasks with 100–200 trials per condition and report confidence intervals, and compare STAIR against a baseline with the same trainable modules but no dynamic target; if STAIR no longer approaches the deliberate baseline or the additional modules alone match its success, the central claim is refuted.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A policy trained on fluent demonstrations can approach deliberate-demonstration performance simply by adding a motion-condensing module, so expensive slow-at-alignment data collection is not required to close most of the gap.
  • The finding that resampling raises alignment weight but not recovery-state coverage implies that future data-collection protocols should deliberately include off-axis approaches and recoveries rather than merely filming more alignment frames.
  • Since the dynamic feature is predicted from the current frame at inference, the method keeps the standard single-image vision-language-action interface and can be integrated into existing robot policies without changing the observation space.
  • The same dynamic-conditioning principle shows gains beyond the six fine-alignment tasks on a standard simulated manipulation benchmark, suggesting the motion-aware representation is broadly useful, not just for tight-tolerance tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The coverage result suggests that synthetically injecting off-axis approaches and recovery states into fluent trajectories could reproduce much of the deliberate-demonstration gain at low collection cost; this is a testable extension the paper does not run.
  • Predicting the dynamic feature deterministically at inference commits to one future; allowing multiple sampled futures could improve robustness in partially observed contact situations, a direction the paper itself flags.
  • Because the dynamic feature is visual-only, combining it with tactile or force feedback would likely further help in heavy-contact tasks; the paper names this as a limitation rather than an explored direction.
  • A simpler motion summary (e.g., optical flow or frame differences) may capture a large share of the benefit; the paper does not compare against such baselines, so the specific tokenization is not yet isolated.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper argues that fluent expert demonstrations are poor teachers for fine-grained manipulation because the decisive alignment/recovery phase occupies a small fraction of frames, and the standard per-frame imitation loss under-supervises exactly those frames. It proposes two data-level remedies—deliberate slow-at-alignment demonstrations and critical-segment resampling—and a representation-level remedy, STAIR, a compact spatio-temporal dynamic feature distilled from short video neighborhoods and used to condition a flow-matching action expert. The central empirical claim is that STAIR trained on fluent data alone recovers most of the deliberate-demonstration gain on six real-world tasks (50.0% to 62.2% overall, approaching the 64.4% of deliberate demonstrations), while remaining competitive with generalist policies on SimplerEnv. The paper includes a limitations section acknowledging the inference-time determinism of the dynamic feature, its reliance on visual dynamics, and the hyperparameter status of keyframe/window selection.

Significance. If the central empirical claim holds, the paper makes a useful contribution to imitation learning for fine manipulation: it identifies a concrete failure mode, gives a mechanistic explanation, and proposes a representation that does not change the deployment-time observation interface. Credit is due for evaluating the data-level effect on an independent base policy (pi-0.5), for comparing against SimplerEnv baselines, and for being largely self-contained: the success metric is not fitted to the method, and no external benchmark is used to define STAIR's success. The main weaknesses are statistical: the headline gains rest on 15 trials per task with no confidence intervals, repeated seeds, or significance tests, and the controlled architecture comparison is less controlled than claimed.

major comments (3)
  1. [§4.3, Tables 2 and 3] The central quantitative claim—STAIR trained on fluent data recovers most of the deliberate-demonstration gain—is not supported at conventional significance. Pooling the six tasks gives 15 trials/task, i.e., 90 trials per condition. Baseline-Fluent is 45/90 (50.0%), STAIR-Fluent 56/90 (62.2%), and Baseline-Deliberate 58/90 (64.4%). The headline difference STAIR-Fluent minus Baseline-Fluent is 11/90 = 12.2 percentage points; with a pooled standard error of roughly 7.4 pp, this is z ≈ 1.65, two-sided p ≈ 0.10. The difference between STAIR-Fluent and Baseline-Deliberate is only 2.2 pp, well below one SE. Per-task n=15 gives per-task SE ≈ 12.9 pp for a 50% success rate, so task-level gaps such as Insert Flowers 66.7% vs 80.0% reflect 2/15 trials. The 85% recovery ratio is thus compatible with a wide range of true effects, including no improvement. Please provide confidence intervals or Bayes
  2. [§4.1, §3.5] The paper states that Baseline and STAIR differ by exactly one factor—whether the conditioning tokens are supervised by the dynamic feature—but they differ in at least three ways: (i) the presence of the trainable projector P_psi and MLP adapter, (ii) the two-stage training procedure, in which Stage I conditions the action expert on the dynamic feature c_dyn and trains the projector, and (iii) the dynamic-target prediction loss L_dyn in Stage II. Holding the VLM and query module fixed does not isolate the dynamic feature. The improvement could come from the extra trainable capacity, from training-time access to a short future video clip (even though the clip is removed at inference), or from the auxiliary objective. To support the attribution, please add ablations such as: Baseline with the same extra modules but a static/random target for the query module, or STAIR without Stage I. With
  3. [§4.4, Fig. 5] The claim that deliberate demonstrations outperform resampling because of expanded recovery-state coverage, rather than because of increased alignment weight, is supported only by a qualitative scatter plot and a single tuned operating point that matches the alignment budget of resampling to deliberate (88 vs 95 frames). No quantitative coverage metric (e.g., hull area, state-space entropy) or sensitivity analysis over the resampling radius r and weight lambda is reported. This coverage claim is load-bearing for contribution 2. Please quantify coverage and vary the alignment budget to show that the remaining gap is not sensitive to the matching procedure.
minor comments (5)
  1. [Eq. (5), §3.4] In Eq. (5) the dynamic condition is written c_dyn without a time subscript, while the text defines c_dyn_t; please make the subscript consistent. Also, 'WanV AE' appears with irregular spacing in several places.
  2. [Tables 4 and 5] The SimplerEnv evaluations report 30 closed-loop rollouts per task but give no confidence intervals or seed variance. At minimum, per-task n and a variance measure should be reported, especially for the 'Ours' rows where the gains over strong baselines are often a few successes out of 30.
  3. [§3.4] N_t is described as a 'future observation neighbourhood', but the offsets Delta are 'temporal offsets around the action chunk'. Please clarify whether Delta includes past offsets and how the neighborhood is defined at trajectory boundaries.
  4. [§6] The Limitations section explicitly notes that keyframe and neighborhood selection are hyperparameters. Given that the method's gains depend on r, lambda, lambda_dyn, and Delta, a small sensitivity table would materially help the reader judge robustness.
  5. [§2] The related-work section cites the authors' own concurrent work [10, 11] without a positioning sentence. A sentence explaining the relationship of STAIR to those works would help readers.

Circularity Check

0 steps flagged

No significant circularity; the central claims are supported by controlled ablations and external benchmarks.

full rationale

The paper's derivation chain is not circular. Eq. (1) is a definitional decomposition of the uniform behavior-cloning loss into transport and alignment terms; it frames the problem but does not by itself produce the measured gains, which come from controlled empirical comparisons (Tables 1-3). STAIR's dynamic feature is not fitted to the success metric: the target c_dyn is distilled from the trajectory's own short video neighborhood (Eqs. 3-5), and the query module (Eqs. 6-8) is trained to predict that frozen representation; the claim that this helps is an empirical result against an ablation that keeps the VLM, action expert, and query module fixed. The data-level comparison in Sec. 4.4 matches alignment weight (88 vs 95 frames) precisely so that the remaining gap is attributed to coverage, not to construction. Self-citations (StaMo [31], World Guidance [27], Bridge [10]) appear in related work as positioning or contrast, not as load-bearing justifications; no uniqueness theorem or ansatz is imported from them. External baselines (pi-0.5) and SimplerEnv provide independent checkpoints. The fragility of 15 trials per task is a statistical-power/reporting concern, not a circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 1 invented entities

The paper contributes a two-stage training scheme with a frozen public video encoder (WanVAE) and standard Q-Former/flow-matching parts; no new physical entity or external free constant is introduced. The free parameters are training hyperparameters whose values are omitted, a reproducibility gap. The load-bearing assumptions are domain assumptions about the BC loss decomposition and the unmeasured sensitivity gap, plus the causal interpretation of coverage.

free parameters (4)
  • Critical-segment window radius r and upweight lambda = Not specified
    Section 3.3 defines the critical window and reweighting factor but gives no values; the paper says resampling was tuned to an alignment budget of 88 frames (Sec. 4.4), which is a fitted/chosen value.
  • Temporal offsets Delta for the observation neighborhood = Not specified
    Section 3.4 defines the dynamic clip via delta offsets; the concrete offsets and clip length are not given and are acknowledged as hyperparameters in the Limitations.
  • Keyframe moments K (gripper change, contact, insertion, placement) = Not specified
    Section 3.3 requires manual definition of task-critical moments; per-task hand choice, acknowledged as hyperparameter in Limitations.
  • lambda_dyn loss weight = Not specified
    Eq. 8 introduces lambda_dyn as a loss weight; no value reported.
axioms (5)
  • domain assumption The imitation loss weights every frame equally (uniform BC loss), yielding the decomposition in Eq. 1.
    Section 3.1 formalizes the bottleneck. Most BC losses do weight steps equally, but action-chunking flow-matching objectives (used in STAIR itself) weight every action step, so the decomposition is a reasonable approximation, not a theorem.
  • domain assumption Action sensitivity in alignment phase is much larger than in transport: sigma_al >> sigma_tr.
    Asserted in Section 3.1, plausible for contact tasks, but not measured or derived from the robot or force data.
  • domain assumption The frozen WanVAE video encoder preserves the short-horizon motion cues needed for alignment (contact, slip, correction).
    Invoked in Section 3.4 and Fig. 2. No analysis or ablation of which cues the frozen encoder preserves for these fine-manipulation tasks.
  • domain assumption The query module can predict the dynamic feature from current RGB alone at inference without losing most of the benefit.
    This is the deployment premise of STAIR (Section 3.5); the paper validates it de facto by end-task success, but does not measure prediction error of the dynamic feature, so the assumption is implicit.
  • domain assumption Coverage of recovery states, as visualized in Fig. 5, is the causal explanation for the remaining gap between deliberate and resampling.
    Section 4.4 concludes a coverage effect based on Fig. 5's scatter plots, but no intervention varies coverage while holding other factors fixed; this is an interpretive claim.
invented entities (1)
  • STAIR dynamic feature tokens (f_dyn) no independent evidence
    purpose: Compact spatio-temporal tokens bridging VLM and action expert, summarizing short-horizon motion during alignment.
    Introduced in Section 3.4. It is a learned representation internal to the policy, with no falsifiable handle outside the paper's own success rates.

pith-pipeline@v1.3.0-alltime-deepseek · 11481 in / 9253 out tokens · 78073 ms · 2026-08-02T11:16:17.029458+00:00 · methodology

0 comments
read the original abstract

Expert demonstrations are widely assumed to be the gold standard for robot imitation learning. Yet for fine-grained manipulation such as insertion, stacking, and alignment, we uncover a counterintuitive failure mode: fluent demonstrations can be poor teachers. A skilled teleoperator compresses the decisive moments of alignment and recovery into a brief temporal window, leaving the policy flooded with redundant free-space motion and starved of supervision exactly where precision determines success. We address this bottleneck at two levels. At the data level, slowing down near alignment and resampling critical segments both help, yet the gain comes mainly from broadening the coverage of recovery states the policy must learn, not from reweighting frames it already has. Such data-side fixes, however, leave the policy's per-frame view untouched: a single image still maps directly to an action, and the local motion that governs correction stays implicit. We therefore turn to the representation level and introduce STAIR (\textbf{S}patio-\textbf{T}emporal feature \textbf{A}s an \textbf{I}nterface for \textbf{R}obot learning), a compact dynamic feature that bridges the vision-language model and the action expert, distilling the short-horizon motion already recorded in each trajectory into dense, motion-aware supervision. Trained on fluent data alone, STAIR recovers most of the deliberate-demonstration gain ($50.0$ to $62.2\%$ overall, approaching the $64.4\%$ of deliberate demonstrations). These results call for a more pedagogical view of robot data, optimized for machine learnability rather than human efficiency alone.

Figures

Figures reproduced from arXiv: 2606.15587 by Chunhua Shen, Hanqing Wang, Hao Chen, Jiuhe Shu, Mingyu Liu, Yuhao Chao, Zeju Li.

Figure 1
Figure 1. Figure 1: Where policy gains actually live. On a pen-cap insertion task, a fluent expert (top) spends ≈ 90% of frames on free-space transport and compresses the decisive alignment phase into ≈ 10% of frames, yielding 35% success. A deliberate demonstration (bottom) redistributes time toward alignment (≈ 50% of frames), exposing the corrective micro-adjustments near contact and raising success to 75%. Success is deci… view at source ↗
Figure 2
Figure 2. Figure 2: Dynamic alignment feature. A short observation neighbourhood is compressed into a few compact tokens that summarize local motion. For each training sample, we associate the supervised ac￾tion chunk with a short dynamic observation clip, Nt = {ot+δ}δ∈∆, (3) where ∆ denotes a small set of temporal offsets around the action chunk. We first encode this future observation neighbourhood with a frozen WanVAE enco… view at source ↗
Figure 3
Figure 3. Figure 3: Setup. Hardware All real-world experiments are conducted on a dual-arm PiperX robot platform from AgileX Robotics, as shown in [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Six real-world manipulation tasks grouped by alignment difficulty. Data conditions and protocol. For every task we collect 100 demonstrations and train by SFT on the mixture, evaluating each policy with 15 trials per task. We compare three data conditions: Fluent (efficient expert demonstrations), Deliberate (slow-at-alignment demonstrations, matched at 100 trajectories per task), and Resample (critical-se… view at source ↗
Figure 5
Figure 5. Figure 5: Alignment-phase coverage vs. effective training weight. Each panel is one demonstra￾tion set. The scatter plots the end-effector position relative to the target (∆x, ∆z) over all alignment￾phase frames, with the success region in blue; a wider spread means broader state coverage. The inset bar chart reports the average number of frames per trajectory in the transport and alignment phases (for resampling, t… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 27 linked inside Pith

  1. [1]

    Ravichandar, A

    H. Ravichandar, A. S. Polydoros, S. Chernova, and A. Billard. Recent advances in robot learning from demonstration.Annual review of control, robotics, and autonomous systems, 3 (1):297–330, 2020

    2020

  2. [2]

    Laskey, J

    M. Laskey, J. Lee, R. Fox, A. Dragan, and K. Goldberg. Dart: Noise injection for robust imitation learning. InConference on robot learning, pages 143–156. PMLR, 2017

    2017

  3. [3]

    Akgun, M

    B. Akgun, M. Cakmak, K. Jiang, and A. L. Thomaz. Keyframe-based learning from demonstra- tion: Method and evaluation.International Journal of Social Robotics, 4(4):343–355, 2012

    2012

  4. [4]

    L. Kou, F. Ni, Y . ZHENG, J. Liu, Y . Yuan, Z. Dong, and J. HAO. KISA: A unified keyframe identifier and skill annotator for long-horizon robotics demonstrations. InForty-first Interna- tional Conference on Machine Learning, 2024. URLhttps://openreview.net/forum? id=oCI9gHocws

    2024

  5. [5]

    Hejna, S

    J. Hejna, S. Mirchandani, A. Balakrishna, A. Xie, A. Wahid, J. Tompson, P. Sanketi, D. Shah, C. Devin, and D. Sadigh. Robot data curation with mutual information estimators.arXiv preprint arXiv:2502.08623, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    Hejna, C

    J. Hejna, C. Bhateja, Y . Jiang, K. Pertsch, and D. Sadigh. Re-mix: Optimizing data mixtures for large scale imitation learning.arXiv preprint arXiv:2408.14037, 2024

    Pith/arXiv arXiv 2024

  7. [7]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al. Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024

    Pith/arXiv arXiv 2024

  8. [8]

    Zitkovich, T

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023

    2023

  9. [9]

    Y . Wang, H. Zhu, M. Liu, J. Yang, H.-S. Fang, and T. He. Vq-vla: Improving vision-language- action models via scaling vector-quantized action tokenizers. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 11089–11099, 2025

    2025

  10. [10]

    M. Liu, Z. Huang, X. Lin, M. Zhu, C. Zhao, Z. Du, Y . Wang, H. Zhu, H. Chen, and C. Shen. Bridge thinking and acting: Unleashing physical potential of vlm with generalizable action expert.arXiv preprint arXiv:2510.03896, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    Huang, M

    Z. Huang, M. Liu, X. Lin, M. Zhu, C. Zhao, Z. Du, Y . Lin, X. Li, Y . Jia, H. Zhong, et al. Notvla: Semantics-preserving robot adaptation via narrative action interfaces.arXiv preprint arXiv:2510.03895, 2025

    Pith/arXiv arXiv 2025

  12. [12]

    K. Wang, L. Lu, M. Liu, J. Jiang, Z. Li, B. Zhang, W. Zheng, X. Yu, H. Chen, and C. Shen. Odyssey: Open-world quadrupeds exploration and manipulation for long-horizon tasks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 18602–18610, 2026

    2026

  13. [13]

    Intelligence, K

    P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, et al.π 0.5: A vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025

    Pith/arXiv arXiv 2025

  14. [15]

    Bjorck, F

    J. Bjorck, F. Casta ˜neda, N. Cherniadev, X. Da, R. Ding, L. Fan, Y . Fang, D. Fox, F. Hu, S. Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025. 9

    Pith/arXiv arXiv 2025

  15. [16]

    Q. Bu, Y . Yang, J. Cai, S. Gao, G. Ren, M. Yao, P. Luo, and H. Li. Univla: Learning to act anywhere with task-centric latent actions.arXiv preprint arXiv:2505.06111, 2025

    Pith/arXiv arXiv 2025

  16. [17]

    J. Cen, C. Yu, H. Yuan, Y . Jiang, S. Huang, J. Guo, X. Li, Y . Song, H. Luo, F. Wang, et al. Worldvla: Towards autoregressive action world model.arXiv preprint arXiv:2506.21539, 2025

    Pith/arXiv arXiv 2025

  17. [18]

    Zhang, H

    W. Zhang, H. Liu, Z. Qi, Y . Wang, X. Yu, J. Zhang, R. Dong, J. He, H. Wang, Z. Zhang, et al. Dreamvla: a vision-language-action model dreamed with comprehensive world knowledge. Advances in Neural Information Processing Systems, 38:24195–24228, 2026

    2026

  18. [19]

    Liang, X

    H. Liang, X. Chen, B. Wang, M. Chen, Y . Liu, Y . Zhang, Z. Chen, T. Yang, Y . Chen, J. Pang, et al. Mm-act: Learn from multimodal parallel generation to act.arXiv preprint arXiv:2512.00975, 2025

    arXiv 2025

  19. [20]

    S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu. Rdt-1b: a diffu- sion foundation model for bimanual manipulation. InInternational Conference on Learning Representations, volume 2025, pages 29982–30009, 2025

    2025

  20. [21]

    J. Yang, Y . Shi, H. Zhu, M. Liu, K. Ma, Y . Wang, G. Wu, T. He, and L. Wang. Como: Learning continuous latent motion from internet videos for scalable robot learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 42352–42363, 2026

    2026

  21. [22]

    S. Nair, A. Rajeswaran, V . Kumar, C. Finn, and A. Gupta. R3m: A universal visual represen- tation for robot manipulation.arXiv preprint arXiv:2203.12601, 2022

    Pith/arXiv arXiv 2022

  22. [23]

    Y . J. Ma, S. Sodhani, D. Jayaraman, O. Bastani, V . Kumar, and A. Zhang. Vip: Towards universal visual reward and representation via value-implicit pre-training.arXiv preprint arXiv:2210.00030, 2022

    Pith/arXiv arXiv 2022

  23. [24]

    Caron, H

    M. Caron, H. Touvron, I. Misra, H. J ´egou, J. Mairal, P. Bojanowski, and A. Joulin. Emerging properties in self-supervised vision transformers. InProceedings of the IEEE/CVF interna- tional conference on computer vision, pages 9650–9660, 2021

    2021

  24. [25]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    Pith/arXiv arXiv 2023

  25. [26]

    Karamcheti, S

    S. Karamcheti, S. Nair, A. S. Chen, T. Kollar, C. Finn, D. Sadigh, and P. Liang. Language- driven representation learning for robotics.arXiv preprint arXiv:2302.12766, 2023

    Pith/arXiv arXiv 2023

  26. [27]

    Y . Su, S. Chen, H. Shi, M. Liu, Z. Zhang, N. Huang, W. Zhong, Z. Zhu, Y . Liu, and X. Liu. World guidance: World modeling in condition space for action generation.arXiv preprint arXiv:2602.22010, 2026

    arXiv 2026

  27. [28]

    S. Ye, J. Jang, B. Jeon, S. J. Joo, J. Yang, B. Peng, A. Mandlekar, R. Tan, Y .-W. Chao, B. Y . Lin, et al. Latent action pretraining from videos. InInternational Conference on Learning Representations, volume 2025, pages 28213–28239, 2025

    2025

  28. [29]

    Y . Chen, Y . Ge, W. Tang, Y . Li, Y . Ge, M. Ding, Y . Shan, and X. Liu. Moto: Latent motion token as the bridging language for learning robot manipulation from videos. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 19752–19763, 2025

    2025

  29. [30]

    Y . Wang, X. Li, W. Wang, J. Zhang, Y . Li, Y . Chen, X. Wang, and Z. Zhang. Unified vision- language-action model.arXiv preprint arXiv:2506.19850, 2025

    Pith/arXiv arXiv 2025

  30. [31]

    M. Liu, J. Shu, H. Chen, Z. Li, C. Zhao, J. Yang, S. Gao, H. Chen, and C. Shen. StaMo: Unsupervised learning of generalizable robot motion from compact state representation.arXiv preprint arXiv:2510.05057, 2025. 10

    Pith/arXiv arXiv 2025

  31. [32]

    O’Neill, A

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 6892–6903. IEEE, 2024

    2024

  32. [33]

    Khazatsky, K

    A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y . Chen, K. Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset.arXiv preprint arXiv:2403.12945, 2024

    Pith/arXiv arXiv 2024

  33. [34]

    Ebert, Y

    F. Ebert, Y . Yang, K. Schmeckpeper, B. Bucher, G. Georgakis, K. Daniilidis, C. Finn, and S. Levine. Bridge data: Boosting generalization of robotic skills with cross-domain datasets. arXiv preprint arXiv:2109.13396, 2021

    Pith/arXiv arXiv 2021

  34. [35]

    C. Chi, Z. Xu, C. Pan, E. Cousineau, B. Burchfiel, S. Feng, R. Tedrake, and S. Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots.arXiv preprint arXiv:2402.10329, 2024

    Pith/arXiv arXiv 2024

  35. [36]

    J. Yang, C. Finn, and D. Sadigh. Data analogies enable efficient cross-embodiment transfer. arXiv preprint arXiv:2603.06450, 2026

    arXiv 2026

  36. [37]

    Kuhar, S

    S. Kuhar, S. Cheng, S. Chopra, M. Bronars, and D. Xu. Learning to discern: Imitating hetero- geneous human demonstrations with preference and representation learning. InConference on Robot Learning, pages 1437–1449. PMLR, 2023

    2023

  37. [38]

    M. Du, S. Nair, D. Sadigh, and C. Finn. Behavior retrieval: Few-shot imitation learning by querying unlabeled datasets.arXiv preprint arXiv:2304.08742, 2023

    Pith/arXiv arXiv 2023

  38. [39]

    L.-H. Lin, Y . Cui, A. Xie, T. Hua, and D. Sadigh. Flowretrieval: Flow-guided data retrieval for few-shot imitation learning.arXiv preprint arXiv:2408.16944, 2024

    Pith/arXiv arXiv 2024

  39. [40]

    Y . Cui, D. Isele, S. Niekum, and K. Fujimura. Uncertainty-aware data aggregation for deep imitation learning. In2019 International Conference on Robotics and Automation (ICRA), pages 761–767. IEEE, 2019

    2019

  40. [41]

    Hoque, A

    R. Hoque, A. Balakrishna, E. Novoseller, A. Wilcox, D. S. Brown, and K. Goldberg. Thriftydagger: Budget-aware novelty and risk gating for interactive imitation learning.arXiv preprint arXiv:2109.08273, 2021

    Pith/arXiv arXiv 2021

  41. [42]

    X. Li, K. Hsu, J. Gu, K. Pertsch, O. Mees, H. R. Walke, C. Fu, I. Lunawat, I. Sieh, S. Kir- mani, et al. Evaluating real-world robot manipulation policies in simulation.arXiv preprint arXiv:2405.05941, 2024

    Pith/arXiv arXiv 2024

  42. [43]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    Pith/arXiv arXiv 2022

  43. [44]

    O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, et al. Octo: An open-source generalist robot policy.arXiv preprint arXiv:2405.12213, 2024

    Pith/arXiv arXiv 2024

  44. [45]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al.π0: A vision-language-action flow model for general robot control. corr, abs/2410.24164, 2024. doi: 10.48550.arXiv preprint ARXIV .2410.24164

    Pith/arXiv arXiv 2024

  45. [46]

    Pertsch, K

    K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine. Fast: Efficient action tokenization for vision-language-action models.arXiv preprint arXiv:2501.09747, 2025

    Pith/arXiv arXiv 2025

  46. [47]

    D. Wu, F. Liu, Y .-H. Hung, and Y . Duan. Spatial-mllm: Boosting mllm capabilities in visual- based spatial intelligence.arXiv preprint arXiv:2505.23747, 2025. 11

    Pith/arXiv arXiv 2025