Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

SCIZOR: A Self-Supervised Approach to Data Curation for Large-Scale Imitation Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SCIZOR, a self-supervised curation framework, filters suboptimal and redundant state-action pairs from robot demonstrations and improves imitation learning success by 15.4% on average, with no human labels.

desk verdict SCIZOR's transition-level curation idea is real and the Sirius-Fleet result is promising, but the headline average is inflated by in-sample threshold tuning; the paper deserves a serious referee once artifacts and out-of-sample thresholds are addressed. read the letter →

arxiv 2505.22626 v2 pith:ON6KY5A2 submitted 2025-05-28 cs.RO cs.AIcs.LG

Yu Zhang , Yuqi Xie , Huihan Liu , Rutav Shah , Michael Wan , Linxi Fan , Yuke Zhu This is my paper · ORCID
classification cs.ROcs.AIcs.LG
keywords imitationlearningdatacurationself-supervisedstate-actionpairfilteringtaskprogresspredictionsemanticdeduplicationvision-language-actionmodelsrobot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the quality of a robot imitation learning dataset can be curated automatically, at the level of individual state-action pairs, without any human annotation. It introduces SCIZOR, a self-supervised pipeline that removes two kinds of harmful data: suboptimal transitions, identified by a learned task-progress predictor, and redundant transitions, identified by deduplication on joint state-action features. The claim is that this fine-grained, label-free filtering consistently improves policy success, by an average of 15.4% across RoboMimic, Sirius-Fleet, and Open-X-Embodiment benchmarks, and that it beats trajectory-level and dataset-level curation baselines. If true, this means expensive manual quality labeling can be replaced by a scalable, annotation-free step when building large robot training sets.

What carries the argument

The load-bearing object is the self-supervised task-progress predictor: given the first and last frames of a two-second chunk, a frozen DINO-V2 encoder plus a transformer classification head predicts the temporal bin of the task progress $T_p$, and the suboptimality score is the gap $V = T - T_p$ between real elapsed time and predicted progress. This turns the assumption that progress grows steadily with time into a measurable per-transition score: scores are spread back over the transitions in each chunk, temporally discounted so evidence of future suboptimality penalizes earlier transitions, and mixed with the trajectory-level mean so whole bad demonstrations are caught. The second mechanism is state-action deduplication, which extends semantic deduplication to sequential decision making by concatenating a Cosmos video embedding of each chunk with its delta end-effector actions, clustering with K-means, and deleting chunks whose maximum intra-cluster cosine similarity exceeds a threshold. A single threshold pair ($\epsilon_s = 0.58$, $\epsilon_d = 0.99$) found on two simulation datasets transfers directly to the real-world Sirius-Fleet data.

What would settle it

Run SCIZOR's progress predictor on a task that requires deliberate waiting or repeated motion, such as stirring food for a set duration, and check whether those productive transitions are scored as suboptimal and deleted; if policies trained on the curated set fall below the no-curation baseline on that task, the linear-progress assumption is the cause.

Watch

Extended reading notes

Core claim

SCIZOR is the first self-supervised, transition-level data curation framework for large-scale imitation learning. It treats a demonstration as a sequence of state-action pairs and scores each pair by two complementary criteria: a progress-prediction module, trained with temporal distance classification on unlabeled video, estimates how many seconds of task progress a short sub-trajectory represents, and when predicted progress falls short of the elapsed time the transitions are flagged suboptimal and removed; a deduplication module encodes visual frames with a pretrained video encoder, concatenates the executed actions, clusters the joint features, and deletes chunks that are near-duplicates of other chunks. Across RoboMimic, Sirius-Fleet, and three Open-X-Embodiment mixtures, policies trained on the curated data improve by an average of 15.4% over policies trained on the full datasets, and the full system outperforms uniform random deletion by 16.1% on average. The method scales to the one-million-trajectory Open-X dataset and benefits both small behavior-cloning policies and a 27M-parameter Octo vision-language-action model.

Load-bearing premise

The method assumes both that task progress grows linearly with elapsed time, so that elapsed time can stand in for expected progress, and that most demonstrations are good enough to learn from, so suboptimal behavior can be detected as a deviation from the majority norm.

Editorial extensions

If this is right

  • Fine-grained curation at the transition level beats coarser filtering: SCIZOR outperforms DemInf by 19.2% on Sirius-Fleet and Re-Mix by 3.5% on average, while trajectory-level filtering stays competitive only when whole trajectories are uniformly good or bad.
  • Curation can scale to web-scale robot data: the pipeline runs on the one-million-trajectory Open-X dataset and improves the 27M-parameter Octo vision-language-action model, not just small behavior-cloning policies.
  • The two filtering signals are complementary: removing only suboptimal transitions or only duplicates each helps, but neither alone reaches the full system's performance on any of the three benchmarks.
  • Deleting the right data beats deleting the same amount of data arbitrarily: SCIZOR beats uniform random deletion by 16.1% on average, so the gains come from selection, not from reduced dataset size.
  • Training on less data can yield more robust behavior: on Sirius-Fleet, failure modes such as book-caddy collisions disappeared in policies trained on the curated dataset, and the policies recovered more quickly from errors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The progress-gap score $V = T - T_p$ is not limited to curation: the same signal could serve as a dense reward surrogate for reinforcement fine-tuning, or as an online trigger for requesting human intervention, since it requires only unlabeled video and elapsed time.
  • The linear-progress assumption limits the method to monotonic tasks; a hierarchical or multi-timescale progress model, which the paper names as future work, could plausibly extend the same pipeline to tasks with phases such as waiting or stirring.
  • Transition-level curation is orthogonal to dataset-level mixture weighting, so SCIZOR could be composed with methods like Re-Mix: first filter within each dataset, then reweight across datasets, potentially compounding the gains.
  • The joint state-action deduplication idea transfers to other domains with demonstration libraries, such as surgical robotics or autonomous driving logs, where visual similarity alone would wrongly merge distinct behaviors.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. SCIZOR proposes a self-supervised, transition-level data curation method for imitation learning. It combines two modules: a suboptimal-transition filter that trains a temporal-distance classifier to predict task progress from pairs of frames and removes chunks whose predicted progress is below the elapsed time, and a state-action deduplication module that clusters joint state-action embeddings with K-means and removes near-duplicate chunks. The method is evaluated on RoboMimic, Sirius-Fleet, and three variants of the Open X-Embodiment dataset (Magic Soup, RT-X, RT-1), using BC, BC-Transformer, and Octo policies. The authors report absolute success-rate gains of 5.4% on RoboMimic, 8.1% on OXE Magic, and 32.9% on Sirius-Fleet over training on the full datasets, an average improvement of 15.4%, and state that SCIZOR outperforms uniform deletion, DemInf, and Re-Mix in several settings.

Significance. If the reported results hold out of sample, SCIZOR would be a useful contribution: it is the first transition-level, annotation-free curation framework for large-scale imitation learning, and the idea of using temporal-distance classification as a self-supervised quality signal is simple and potentially scalable. The paper also includes reasonable ablations, comparisons against trajectory- and dataset-level baselines, qualitative analyses of the removed transitions, and a candid limitations section. However, the headline average mixes in-sample threshold-tuned benchmarks with a single held-out real-robot dataset, and the OXE results are based on only two seeds without reported variance; as a result, the strength of the central empirical claim is currently uncertain.

major comments (4)
  1. [Appendix A.1, Section 4.2, Figure 3] The unified thresholds epsilon_s=0.58 and epsilon_d=0.99 are selected by running the suboptimal-only and deduplication-only variants on exactly RoboMimic and OXE Magic with deletion ratios of 10%, 20%, and 30%, and choosing the thresholds that maximize downstream policy success on those same benchmarks. Consequently, the gains of +5.4 points on RoboMimic and +8.1 points on OXE Magic reported in Figure 3 are in-sample estimates, and they are included in the headline 15.4% average. The only fully held-out estimate is Sirius-Fleet (+32.9 points), which is a single real-robot setting with 10 trials per seed. Please provide a genuinely out-of-sample evaluation, for example by fixing thresholds on one benchmark and evaluating on untouched benchmarks, or by nested cross-validation, and report tuned and untuned gains separately.
  2. [Section 4.1, Appendix B.2, Table 9] The OXE Magic and OXE RT-X results are based on two random seeds, and the main figure and Table 9 report averaged success rates without per-seed values or confidence intervals. With two seeds and no baseline variance, the +8.1 point OXE Magic gain and the RT-X comparison against Re-Mix could be within noise. Please report per-seed results and standard errors, or increase the number of seeds for these key comparisons.
  3. [Section 4.2 (RQ2), Table 7] The comparison with the trajectory-level baseline DemInf weakens the claim that transition-level curation is generally superior. On RoboMimic Can, DemInf achieves 88.9% versus SCIZOR's 84.0%, and on Square 41.4% versus 40.8%; the text acknowledges this in RQ2 but still concludes that fine-grained curation offers advantages. The evidence supports a transition-level advantage only on datasets with complex intra-trajectory quality distributions such as Sirius-Fleet. Please temper the claim accordingly and, ideally, report a statistical comparison across all benchmarks rather than relying on a few point estimates.
  4. [Section 3.2, Section 6 (Limitation)] The suboptimality score V_{i:i+T}=T - T_p treats elapsed time as the expected task progress by construction, and the progress predictor is trained on the same demonstrations it later filters. As the Limitations section acknowledges, this fails for tasks with intentional pauses, stirring, or waiting; moreover, if slow or repetitive behavior is common in a dataset, the predictor will absorb that behavior as normal and will not flag it. This is a load-bearing assumption of the proposed detector, not merely a future extension. Please state the intended operating regime explicitly (e.g., datasets where the majority of transitions are proficient and progress is roughly monotonic) and, if possible, include a controlled experiment that varies the fraction of suboptimal demonstrations to show how the method degrades.
minor comments (5)
  1. [Introduction, Section 2] There are several typos and spacing errors, for example 'SCIZORis as elf-supervised datac uration method' in the Introduction and inconsistent spelling of the DemInf baseline ('Deminf', 'DemoInf', 'DemInf') across the text, tables, and figures.
  2. [Figure 3] The legend lists five methods ('No Deletion', 'Uniform', 'DemInf', 'Re-Mix', 'Ours'), but not every panel appears to contain all five methods; please clarify which bars correspond to which baseline on each dataset, or use separate legends per panel.
  3. [Appendix A.4, Section 3.3] The notation T is used both for the chunk duration in the deduplication module and for the elapsed time in the progress predictor, and Appendix A.4 states that all experiments use a constant 2-second interval. Please unify the notation and define T's units consistently.
  4. [Section 3.2, Equations (2)-(4)] The description of how the predicted progress bin is converted into the scalar T_p is not explicit; Appendix A.5 describes the bin edges but not the conversion used to compute V_{i:i+T}. Please state this conversion and the value of alpha used in the final score.
  5. [General] No code or trained curation masks are released, and the paper does not mention a data-release plan. For a data-curation method, releasing the curation masks or the code that produces them would substantially aid reproducibility and would also allow the out-of-sample threshold issue to be examined by other groups.

Circularity Check

1 steps flagged · score 4.0 of 10

Headline 15.4% average is partly in-sample: the unified thresholds are tuned on RoboMimic and OXE Magic, the same benchmarks that supply two of the three averaged gains.

  1. fitted input called prediction [Appendix A.1, 'Determining the Single Threshold Across Datasets'; results reported in Section 4.2 and Figure 3]
    "We conduct a hyperparameter search on two simulation datasets, RoboMimic and OXEMagic and find that ϵs=0.58 and ϵd=0.99 yield the best performance on both datasets."

    The unified thresholds are selected by maximizing downstream policy success on RoboMimic and OXE Magic, and the reported gains on those exact benchmarks (+5.4 and +8.1 points) are averaged into the headline 15.4% improvement. Thus two of the three averaged numbers are in-sample tuning results rather than out-of-sample predictions; only Sirius-Fleet is fully held out from threshold selection. The headline average therefore mixes selected and held-out estimates, so part of the evidence presented as a prediction reduces to the threshold-fitting procedure itself.

full rationale

The core derivation is not circular. The progress estimator is trained with a temporal-distance classification objective, predicting an elapsed-time bin from frame pairs, and the suboptimality score is defined as the gap between actual and predicted elapsed time (Section 3.2). This is a self-supervised anomaly signal: it does not presuppose the curation outcome, and the linear-progress assumption is explicitly stated and conceded in the Limitations section rather than being hidden or derived from the target. Training the predictor on the same dataset it filters (Appendix B.2) makes the score relative to that dataset's statistics, which is a stated assumption and a limitation, not a definitional equivalence. The deduplication module is likewise an extension of SemDeDup to joint state-action features and is not circular. Self-citations to Sirius-Fleet and Mutex are used as benchmarks, not as load-bearing uniqueness claims. The one concrete circularity-related issue is in Appendix A.1: the unified thresholds are selected using downstream success on RoboMimic and OXE Magic, and those same two benchmarks contribute +5.4 and +8.1 to the headline 15.4% average. Only Sirius-Fleet is a fully held-out estimate. This makes part of the headline evidence in-sample, supporting a score of 4: partial circularity in the headline evaluation, while the method itself retains independent, non-circular content.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central claim rests on modeling choices and assumptions that are partly acknowledged in the Limitations section: linear task progress, majority-good demonstrations, and temporal distance as a proxy for task progress. The empirical thresholds are fitted on the evaluation benchmarks, and several hyperparameters are unreported.

free parameters (7)
  • Suboptimality threshold epsilon_s = 0.58
    Tuned by sweeping deletion ratios (10, 20, 30%) on RoboMimic and OXE Magic (Appendix A.1), then applied to all datasets.
  • Deduplication threshold epsilon_d = 0.99
    Tuned jointly with epsilon_s on the same two evaluation datasets; yields very small deletion on RoboMimic (0.3%).
  • Score mixture weight alpha = 0.5
    Chosen by hand, stated as 'simply choose 0.5' in Appendix A.5, to blend transition-level and trajectory-level suboptimality scores.
  • Temporal discount factor gamma = not specified
    Defined in Section 3.2 for future suboptimality discounting; the paper does not report the value used in experiments.
  • Chunk duration T = 2 seconds
    Fixed interval for progress prediction and deduplication chunks across all datasets (Appendix A.4); a design choice.
  • Number of progress bins B and bin edges = B=5, bins [0,0.5), [0.5,1), [1,2), [2,5), [5,inf)
    Discretization for temporal-distance classification (Appendix A.5); chosen empirically.
  • K-means cluster count K = not specified
    Used for deduplication clustering; the paper does not report K or its selection procedure.
assumptions (4)
  • domain assumption Task progress is linear in time.
    Section 3.2 defines suboptimality as T - T_p where T is elapsed time. The Limitations section admits this assumption breaks for pausing or repetitive behaviors.
  • domain assumption Most demonstrations in each dataset are good quality.
    The Limitations section states that SCIZOR assumes most demonstrations within a trajectory are of good quality; otherwise the self-supervised progress predictor learns poor behavior as the norm.
  • domain assumption Predicted temporal distance between two frames measures task progress.
    Appendix A.3 trains a classifier to predict the time gap between frames, and Section 3.2 uses that prediction as progress toward task completion.
  • domain assumption Pretrained encoders DINO-V2 and Cosmos provide suitable representations for progress and similarity.
    Appendix A.2 and Section 3.3 rely on frozen pretrained models without validation that their features align with task progress.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SCIZOR: A Self-Supervised Approach to Data Curation for Large-Scale Imitation Learning." pith.science (2026). https://pith.science/paper/ON6KY5A2

@misc{pith2026250522626,
  author       = {Pith},
  title        = {Pith review of: SCIZOR: A Self-Supervised Approach to Data Curation for Large-Scale Imitation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ON6KY5A2}},
  note         = {Machine review of arXiv:2505.22626}
}
read the original abstract

Imitation learning advances robot capabilities by enabling the acquisition of diverse behaviors from human demonstrations. However, large-scale datasets used for policy training often introduce substantial variability in quality, which can negatively impact performance. As a result, automatically curating datasets by filtering low-quality samples to improve quality becomes essential. Existing robotic curation approaches rely on costly manual annotations and perform curation at a coarse granularity, such as the dataset or trajectory level, failing to account for the quality of individual state-action pairs. To address this, we introduce SCIZOR, a self-supervised data curation framework that filters out low-quality state-action pairs to improve the performance of imitation learning policies. SCIZOR targets two complementary sources of low-quality data: suboptimal data, which hinders learning with undesirable actions, and redundant data, which dilutes training with repetitive patterns. SCIZOR leverages a self-supervised task progress predictor for suboptimal data to remove samples lacking task progression, and a deduplication module operating on joint state-action representation for samples with redundant patterns. Empirically, we show that SCIZOR enables imitation learning policies to achieve higher performance with less data, yielding an average improvement of 15.4% across multiple benchmarks. More information is available at: https://ut-austin-rpl.github.io/SCIZOR/

Figures

Figures reproduced from arXiv: 2505.22626 by the authors.

Figure 1
Figure 1. SCIZOR overview. Each trajectory from the original robotic datasets is simultaneously passed through the suboptimal transitions removal module and the redundant transitions removal module. Each module removes data, resulting in a curated dataset. A policy trained on the curated dataset achieves a higher success rate. an approach that has already shown promise in fields like computer vision (CV) and natural lan￾guage… view at source ↗
Figure 2
Figure 2. SCIZOR’s architecture. We apply two curation modules: (1) Suboptimal transition removal, where we estimate chunk progress from its first and last frames and discard those below a threshold; (2) State-action deduplication, where we encode all frames, cluster their features via K-means, and remove frames whose intra-cluster cosine similarity exceeds a threshold. 3.1 Preliminaries and Formulations We formulate a robot … view at source ↗
Figure 3
Figure 3. Performance comparison across different datasets. We use the unified threshold for SCIZOR and report success rates on 4 datasets. We found that SCIZOR achieves the strongest per￾formance and outperforms the baselines. method, Deminf [6], and a dataset-level curation method, Re-Mix [10]. Deminf estimates the av￾erage contribution of a trajectory towards the mutual information between states and actions in the entire … view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Breakdown of suboptimal types classified by SCIZOR. The three dominant failure modes predicted by SCIZOR’s subop￾timal classifier are Laggy Motion, Manipula￾tion Failure, and Pause, showing SCIZOR re￾moves semantically meaningful transitions. RQ4: What types of low-qua…
Figure 5
Figure 5. Figure 5: The performance of the Suboptimal-Only method and the Deduplication-Only method [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Deletion ratio as a function of the chosen threshold for suboptimal-transition removal and [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Visualizations for suboptimal scenarios detected in Robomimic dataset [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Visualizations for suboptimal scenarios detected in Sirius-Fleet dataset [PITH_FULL_IMAGE:figures/full_fig_p021_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Auditing Instruction-Trajectory Mismatches in Multimodal Robot Demonstrations

    cs.RO 2026-08 conditional novelty 6.0 of 10

    MMPF detects and corrects instruction-trajectory mismatches in robot demonstration datasets using local neighborhood voting, global prototype similarity, and entropy-weighted multimodal fusion.

  2. SIEVE: Structure-Aware Data Selection for Imitation Learning with VLA Models

    cs.RO 2026-07 conditional novelty 6.0 of 10

    Selecting 50% of robot demonstrations by maximizing exposure to reusable primitive-transition patterns outperforms full-data training while halving training steps.

Reference graph

Works this paper leans on

54 extracted references · 16 canonical work pages · cited by 2 Pith papers

  1. [1]

    Mandlekar, D

    A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y . Zhu, and R. Mart´ın-Mart´ın. What matters in learning from offline human demonstrations for robot manipulation, 2021. URLhttps://arxiv.org/abs/2108.03298

    arXiv 2021

  2. [2]

    D. S. Brown, W. Goo, and S. Niekum. Better-than-demonstrator imitation learning via automatically-ranked demonstrations, 2019. URLhttps://arxiv.org/abs/1907.03976

    work page Pith review arXiv 2019

  3. [3]

    F. Lin, Y . Hu, P. Sheng, C. Wen, J. You, and Y . Gao. Data scaling laws in imitation learning for robotic manipulation, 2025. URLhttps://arxiv.org/abs/2410.18647

    arXiv 2025

  4. [4]

    H. Liu, S. Nasiriany, L. Zhang, Z. Bao, and Y . Zhu. Robot learning on the job: Human-in- the-loop autonomy and learning during deployment. In Robotics: Science and Systems (RSS), 2023

    2023

  5. [5]

    Hejna, C

    J. Hejna, C. Bhateja, Y . Jian, K. Pertsch, and D. Sadigh. Re-mix: Optimizing data mixtures for large scale imitation learning. arXiv preprint arXiv:2408.14037, 2024

    arXiv 2024

  6. [6]

    Hejna, S

    J. Hejna, S. Mirchandani, A. Balakrishna, A. Xie, A. Wahid, J. Tompson, P. Sanketi, D. Shah, C. Devin, and D. Sadigh. Robot data curation with mutual information estimators. 2025. URL https://arxiv.org/abs/2502.08623

    arXiv 2025

  7. [7]

    Abbas, K

    A. Abbas, K. Tirumala, D. Simig, S. Ganguli, and A. S. Morcos. Semdedup: Data-efficient learning at web-scale through semantic deduplication.arXiv preprint arXiv:2303.09540, 2023

    arXiv 2023

  8. [8]

    Albalak, Y

    A. Albalak, Y . Elazar, S. M. Xie, S. Longpre, N. Lambert, X. Wang, N. Muennighoff, B. Hou, L. Pan, H. Jeong, C. Raffel, S. Chang, T. Hashimoto, and W. Y . Wang. A survey on data selection for language models, 2024. URLhttps://arxiv.org/abs/2402.16827

    arXiv 2024

Show all 54 references
  1. [9]

    Schuhmann, R

    C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta, C. Mullis, M. Wortsman, P. Schramowski, S. Kundurthy, K. Crowson, L. Schmidt, R. Kaczmarczyk, and J. Jitsev. Laion-5b: An open large-scale dataset for training next generation image-t...

    arXiv

  2. [10]

    S. M. Xie, H. Pham, X. Dong, N. Du, H. Liu, Y . Lu, P. Liang, Q. V . Le, T. Ma, and A. W. Yu. Doremi: Optimizing data mixtures speeds up language model pretraining. ArXiv, abs/2305.10429, 2023. URLhttps://api.semanticscholar.org/CorpusID: 258741043

    2023 arXiv

  3. [11]

    Z. Wang, A. Novikov, K. Zolna, J. T. Springenberg, S. Reed, B. Shahriari, N. Siegel, J. Merel, C. Gulcehre, N. Heess, and N. de Freitas. Critic regularized regression. volume 33, pages 7768–7778, 2020

    2020

  4. [12]

    Kostrikov, A

    I. Kostrikov, A. Nair, and S. Levine. Offline reinforcement learning with implicit q-learning. In ICLR, 2021

    2021

  5. [13]

    O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, J. Luo, Y . L. Tan, L. Y . Chen, P. Sanketi, Q. Vuong, T. Xiao, D. Sadigh, C. Finn, and S. Levine. Octo: An open-source generalist robot policy, 2024. URLhttps: //arxiv.org/a...

    2024 arXiv

  6. [14]

    A. S. Chen, A. M. Lessing, Y . Liu, and C. Finn. Curating demonstrations using online experi- ence, 2025. URLhttps://arxiv.org/abs/2503.03707

    2025 arXiv

  7. [15]

    Padalkar, A

    A. Padalkar, A. Pooley, A. Jain, A. Bewley, A. Herzog, A. Irpan, A. Khazatsky, A. Rai, A. Singh, A. Brohan, et al. Open x-embodiment: Robotic learning datasets and rt-x mod- els. arXiv preprint arXiv:2310.08864, 2023. 9

    2023 arXiv

  8. [16]

    Aytar, T

    Y . Aytar, T. Pfaff, D. Budden, T. Paine, Z. Wang, and N. De Freitas. Playing hard exploration games by watching youtube. Advances in neural information processing systems, 31, 2018

    2018

  9. [17]

    Sermanet, K

    P. Sermanet, K. Xu, and S. Levine. Unsupervised perceptual rewards for imitation learning,

  10. [18]

    Zakka, A

    K. Zakka, A. Zeng, P. Florence, J. Tompson, J. Bohg, and D. Dwibedi. Xirl: Cross-embodiment inverse reinforcement learning, 2021. URLhttps://arxiv.org/abs/2106.03911

    2021 arXiv

  11. [19]

    D. A. Pomerleau. Alvinn: An autonomous land vehicle in a neural network. In NeurIPS, 1989

    1989

  12. [20]

    Zhang, Z

    T. Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel. Deep imitation learning for complex manipulation tasks from virtual reality teleoperation. In ICRA, 2018

    2018

  13. [21]

    Florence, C

    P. Florence, C. Lynch, A. Zeng, O. Ramirez, A. Wahid, L. Downs, A. Wong, J. Lee, I. Mor- datch, and J. Tompson. Implicit behavioral cloning. In CoRL, 2021

    2021

  14. [22]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, J. Ibarz, B. Ichter, A. Irpan, T. Jackson, S. Jesmonth, N. J. Joshi, R. Julian, D. Kalashnikov, Y . Kuang, I. Leal, K.-H. Lee, S. Levine, Y . Lu, U. Malla, D....

    2023 arXiv

  15. [23]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818, 2023

    2023 arXiv

  16. [24]

    M. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, Q. Vuong, T. Kollar, B. Burchfiel, R. Tedrake, D. Sadigh, S. Levine, P. Liang, and C. Finn. Openvla: An open-source vision-language-action model. arXiv preprint arX...

    2024 arXiv

  17. [25]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Haus- man, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky.π 0: A vision...

    2024 arXiv

  18. [26]

    Bjorck, F

    NVIDIA, J. Bjorck, F. Casta ˜neda, N. Cherniadev, X. Da, R. Ding, L. J. Fan, Y . Fang, D. Fox, F. Hu, S. Huang, J. Jang, Z. Jiang, J. Kautz, K. Kundalia, L. Lao, Z. Li, Z. Lin, K. Lin, G. Liu, E. Llontop, L. Magne, A. Mandlekar, A. Narayan, S. Nasiriany, S. Reed, Y . L. Tan, G...

    2025 arXiv

  19. [27]

    J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models, 2020

    2020

  20. [28]

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion, 2024

    2024

  21. [29]

    Ebert, Y

    F. Ebert, Y . Yang, K. Schmeckpeper, B. Bucher, G. Georgakis, K. Daniilidis, C. Finn, and S. Levine. Bridge data: Boosting generalization of robotic skills with cross-domain datasets, 2021

    2021

  22. [30]

    Khazatsky, K

    A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y . Chen, et al. Droid: A large-scale in-the-wild robot manipulation dataset, 2024. 10

    2024

  23. [31]

    AgiBot-World-Contributors, Q. Bu, J. Cai, L. Chen, X. Cui, Y . Ding, S. Feng, S. Gao, X. He, X. Huang, S. Jiang, Y . Jiang, C. Jing, H. Li, J. Li, C. Liu, Y . Liu, Y . Lu, J. Luo, P. Luo, Y . Mu, Y . Niu, Y . Pan, J. Pang, Y . Qiao, G. Ren, C. Ruan, J. Shan, Y . Shen, C. Shi, ...

    2025 arXiv

  24. [32]

    Mandlekar, S

    A. Mandlekar, S. Nasiriany, B. Wen, I. Akinola, Y . Narang, L. Fan, Y . Zhu, and D. Fox. Mimicgen: A data generation system for scalable robot learning using human demonstrations. In 7th Annual Conference on Robot Learning, 2023

    2023

  25. [33]

    Nasiriany, A

    S. Nasiriany, A. Maddukuri, L. Zhang, A. Parikh, A. Lo, A. Joshi, A. Mandlekar, and Y . Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots. arXiv preprint arXiv:2406.02523, 2024

    2024 arXiv

  26. [34]

    Walke, K

    H. Walke, K. Black, A. Lee, M. J. Kim, M. Du, C. Zheng, T. Zhao, P. Hansen-Estruch, Q. Vuong, A. He, V . Myers, K. Fang, C. Finn, and S. Levine. Bridgedata v2: A dataset for robot learning at scale, 2024. URLhttps://arxiv.org/abs/2308.12952

    2024 arXiv

  27. [35]

    A. . Team, J. Aldaco, T. Armstrong, R. Baruch, J. Bingham, S. Chan, K. Draper, D. Dwibedi, C. Finn, P. Florence, S. Goodrich, W. Gramlich, T. Hage, A. Herzog, J. Hoech, T. Nguyen, I. Storz, B. Tabanpour, L. Takayama, J. Tompson, A. Wahid, T. Wahrburg, S. Xu, S. Yaroshenko, K. ...

    2024

  28. [36]

    A. Iyer, Z. Peng, Y . Dai, I. Guzey, S. Haldar, S. Chintala, and L. Pinto. Open teach: A versatile teleoperation system for robotic manipulation. arXiv preprint arXiv:2403.07870, 2024

    2024 arXiv

  29. [37]

    Z. Fu, T. Z. Zhao, and C. Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation. arXiv preprint arXiv:2401.02117, 2024

    2024 arXiv

  30. [38]

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware. arXiv preprint arXiv:2304.13705, 2023

    2023 arXiv

  31. [39]

    T. Lin, Y . Zhang, Q. Li, H. Qi, B. Yi, S. Levine, and J. Malik. Learning visuotactile skills with two multifingered hands. arXiv preprint arXiv:2404.16823, 2024

    2024 arXiv

  32. [40]

    A. Fang, A. M. Jose, A. Jain, L. Schmidt, A. Toshev, and V . Shankar. Data filtering networks,

  33. [41]

    Sachdeva, B

    N. Sachdeva, B. Coleman, W.-C. Kang, J. Ni, L. Hong, E. H. Chi, J. Caverlee, J. McAuley, and D. Z. Cheng. How to train data-efficient llms, 2024. URLhttps://arxiv.org/abs/2402. 09668

    2024

  34. [42]

    X. Li, H. Zou, and P. Liu. Limr: Less is more for rl scaling, 2025. URLhttps://arxiv. org/abs/2502.11886

    2025 arXiv

  35. [43]

    Belkhale, Y

    S. Belkhale, Y . Cui, and D. Sadigh. Data quality in imitation learning, 2023. URLhttps: //arxiv.org/abs/2306.02437

    2023 arXiv

  36. [44]

    Gandhi, S

    K. Gandhi, S. Karamcheti, M. Liao, and D. Sadigh. Eliciting compatible demonstrations for multi-human imitation learning. In Conference on Robot Learning, 2022. URLhttps:// api.semanticscholar.org/CorpusID:252918784

    2022

  37. [45]

    S. Dass, A. Khaddaj, L. Engstrom, A. Madry, A. Ilyas, and R. Mart ´ın-Mart´ın. Datamil: Se- lecting data for robot imitation learning with datamodels, 2025. URLhttps://arxiv.org/ abs/2505.09603. 11

    2025 arXiv

  38. [46]

    Agarwal, A

    NVIDIA, N. Agarwal, A. Ali, M. Bala, Y . Balaji, E. Barker, T. Cai, P. Chattopadhyay, Y . Chen, Y . Cui, Y . Ding, D. Dworakowski, J. Fan, M. Fenzi, F. Ferroni, S. Fidler, D. Fox, S. Ge, Y . Ge, J. Gu, S. Gururani, E. He, J. Huang, J. Huffman, P. Jannaty, J. Jin, S. W. Kim, G....

    2025 arXiv

  39. [47]

    X. Li, K. Hsu, J. Gu, K. Pertsch, O. Mees, H. R. Walke, C. Fu, I. Lunawat, I. Sieh, S. Kir- mani, S. Levine, J. Wu, C. Finn, H. Su, Q. Vuong, and T. Xiao. Evaluating real-world robot manipulation policies in simulation. arXiv preprint arXiv:2405.05941, 2024

    2024 arXiv

  40. [48]

    H. Liu, Y . Zhang, V . Betala, E. Zhang, J. Liu, C. Ding, and Y . Zhu. Multi-task interactive robot fleet learning with visual world models, 2024. URLhttps://arxiv.org/abs/2410.22689

    2024 arXiv

  41. [49]

    Pertsch, K

    K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine. Fast: Efficient action tokenization for vision-language-action models.arXiv preprint arXiv: 2501.09747, 2025

    2025 arXiv

  42. [50]

    S. Li, Y . Gao, D. Sadigh, and S. Song. Unified video action model. arXiv preprint arXiv: 2503.00200, 2025

    2025 arXiv

  43. [51]

    R. Shah, R. Mart ´ın-Mart´ın, and Y . Zhu. Mutex: Learning unified policies from multimodal task specifications. In 7th Annual Conference on Robot Learning, 2023. URLhttps:// openreview.net/forum?id=PwqiqaaEzJ

    2023

  44. [52]

    better”, two “okay

    K. Pertsch. Rlds dataset modification, 2024. URLhttps://github.com/kpertsch/rlds_ dataset_mod. GitHub repository. 12 A Method details 46 48 50 52 54 56 58 60 62 64Success Rate (%) Policy Success Rate on Robomimic 16 18 20 22 24 26 28 30 Policy Success Rate on Open-X-Embodiment...

    2024

  45. [2017]

    URLhttps://arxiv.org/abs/1612.06699

    arXiv

  46. [2023]

    URLhttps://arxiv.org/abs/2309.17425

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.