Pith. sign in

REVIEW 2 major objections 5 minor 52 references

Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation

T0 review · 2 major / 5 minor · reviewed 2026-07-12 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Sparse retry labels turn imperfect robot demos into mistake-sensitive value functions that improve imitation learning.

desk verdict Solid practical fix for monotonic progress values on imperfect demos; real-robot gains are real, but the drop-and-rebound assumption is filtered into the labels and the policy claim lacks a preference-loss ablation. read the letter →

arxiv 2606.24633 v2 pith:5MWKWIUX submitted 2026-06-23 cs.RO

Xinyao Qin , Junjie Lu , Kaixin Wang , Chuheng Zhang , Sinjae Kang , Kimin Lee , Min Xu , Bin Liang
show 2 more authors
Jun Yang Li Zhao
This is my paper · ORCID
classification cs.RO
keywords robotlearningvaluerewardmodelingimitationsuboptimaldemonstrationsretrysupervisionweightedbehaviorcloning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Human robot demonstrations are full of mistakes and fixes—bad grasps, misalignments, retries—yet most value models treat progress as a smooth climb and smear over those local failures. This paper argues that the start of a correction is cheap, reliable supervision: around that moment, value should fall as the state degrades and rise again as recovery succeeds. ReTVL trains a value model with ordinary global progress labels plus pairwise preferences built from those sparse retry keypoints, then uses the resulting scores to reweight action chunks for behavior cloning. On four real-arm tasks, the method keeps strong global progress calibration while detecting local drops far better than progress-only baselines, and lifts average success from 41% (uniform cloning) and 63% (progress-weighted cloning) to 80%. The practical claim is that imperfect demos are not just noise: their corrective structure can teach a policy which segments to trust and which to downweight.

What carries the argument

ReTVL (ReTry-Supervised Value Learning): a value head on a vision-language backbone trained with global cross-entropy progress targets plus a soft-weighted pairwise logistic preference loss that forces pre-retry > near-retry < post-retry ordering around annotated correction starts.

What would settle it

On held-out recovery trajectories, check whether value still drops near annotated retries and rebounds after; if Pre>Retry and Drop AUC collapse to progress-baseline levels, or if reweighted cloning no longer beats progress-weighted BC on the same mixed demos, the central claim fails.

Watch

Extended reading notes

Core claim

Retry events in mixed-quality demonstrations induce a local degradation-and-recovery value shape that monotonic progress labels miss. By combining absolute progress calibration outside retry neighborhoods with soft-window pairwise preferences around sparsely annotated correction starts, ReTVL learns values that assign lower scores to harmful mistake segments and higher scores to useful recovery, improving downstream weighted behavior cloning on real robot manipulation tasks.

Load-bearing premise

Around each labeled correction start, value is assumed to form a local valley—higher before the mistake deepens, lowest near the retry, higher after recovery—so wrong or non-local correction patterns would teach the wrong ranking.

Editorial extensions

If this is right

  • Mixed-quality teleop data can be used for imitation without discarding retries, by reweighting chunks with mistake-sensitive values.
  • Sparse labels at correction starts are enough to recover local drop-and-rebound structure that pure progress regression misses.
  • Downstream weighted behavior cloning preferentially keeps recovery actions and suppresses bad-action chunks more than progress-only value models.
  • Global progress calibration and local preference losses are complementary: either alone weakens either global scale or local mistake sensitivity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same retry-keypoint idea could supervise online critics or closed-loop improvement, not only offline chunk reweighting.
  • If correction starts can be auto-detected from teleop correction interfaces or residual-policy logs, the labeling bottleneck shrinks further.
  • Tasks with exploratory or multi-attempt recovery (not clean valleys) would need a softer or multi-mode preference model than the three-window construction used here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes ReTVL, a value-learning method for mixed-quality robot demonstrations that treats sparsely annotated retry keypoints as local supervision. A VLM-based distributional value model is trained with absolute progress/success calibration outside retry neighborhoods (Eq. 2) plus soft-weighted pairwise preference losses around each keypoint (Eqs. 3–8), under a drop-and-rebound assumption that pre-retry and post-retry states should outrank near-retry states. The resulting values reweight action chunks for weighted behavior cloning (Eqs. 10–11). On four real-robot tasks, ReTVL matches or approaches progress baselines on global VOC/S-F metrics while substantially improving local mistake metrics (Table 1), and ReTVL-BC raises average success from 41% (standard BC) and 63% (RECAP-BC) to 80% over 20 trials per task (Table 2). Ablations (Table 3) and weight analyses (Fig. 4, Table 7) support the role of preference supervision and selective down-weighting of bad chunks.

Significance. If the result holds, the paper offers a practical and low-cost way to turn common human correction events into mistake-sensitive value supervision, addressing a real gap between monotonic progress models and imperfect teleop data. Strengths include real-robot evaluation on four tasks, held-out value metrics that separate global progress from local retry structure, explicit ablations of preference loss / soft window / absolute calibration, annotation-cost and consistency analysis (Appendix C), and a clear weighted-BC pipeline with independent bad-action weight diagnostics. The contribution is incremental relative to progress-value and preference-reward lines, but the retry-keypoint formulation and the demonstrated policy gains make it a useful addition for imitation from mixed-quality demos.

major comments (2)
  1. The drop-and-rebound assumption in Sec. 3.2 (pair construction via Eqs. 3–5 and L_pref in Eq. 8) is load-bearing for both local metrics and the reweighting signal, yet Appendix C.1 filters training annotations to recoverable local corrections and excludes exploratory retries, unsuccessful corrections, and non-local errors. Consequently Table 1’s Drop AUC / Pre>Retry / Post>Retry largely evaluate the same structure the labeling policy already selects for. The Limitations section acknowledges this, but the manuscript needs either (i) a stress-test on held-out trajectories that include unsuccessful or non-local corrections, or (ii) a clear quantitative statement of how often such cases appear in the raw demos and how the value model behaves on them. Without that, the claim of general mistake sensitivity is overstated relative to the filtered supervision regime.
  2. Table 2’s headline policy gains (Standard BC 41% → RECAP-BC 63% → ReTVL-BC 80%) are not causally tied to the preference objective. Table 3 shows that removing preference loss collapses local value metrics, but there is no corresponding w/o-Preference-Loss (or drop-regression) weighted-BC run under the same Eq. 10–11 pipeline. Fig. 4 and Table 7 are helpful independent weight diagnostics, yet without a policy ablation it remains unclear how much of the +17.5% over RECAP-BC is due to retry preferences versus other differences in value calibration. A single ablated BC condition on the same mixed-quality sets would make the central claim much more secure.
minor comments (5)
  1. Typo in Q1 of Sec. 4: “sublte” should be “subtle”.
  2. Table 2 text reports averages as 41.25% / 62.50% / 80.00%, while the table body shows 41 / 63 / 80; please make the rounding consistent.
  3. Sec. 3.3 and Appendix A.2 both describe chunk weighting; the main text uses a (µ−2σ)/4σ clip while A.2 also mentions a task-specific κ at the ~80th percentile. Clarify which rule is used for the reported Table 2 numbers.
  4. Figure 3 captions and the Stack Blocks example in Fig. 1 would benefit from marking the annotated retry keypoint and the pre/near/post windows on the value curves for easier visual comparison to the metrics.
  5. Related work could more explicitly position ReTVL against stage-aware reward models (e.g., SARM) and preference-based robotic reward models beyond Robometer, since the method sits at their intersection.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ReTVL is a supervised ML method whose value estimates and policy gains are not forced by construction of the loss or by self-citation.

full rationale

The paper does not present a first-principles derivation that reduces to its inputs. Absolute progress targets are the standard definition v*_t = t/T (Eq. 2), not a fit renamed as prediction. Retry-induced pairs (Eqs. 3–5, L_pref Eq. 8) encode an explicit modeling assumption (local drop-and-rebound), which the authors state openly and ablate (Table 3: w/o Preference Loss collapses local metrics while keeping global VOC high). That is an assumption, not a tautology: the same structure is not forced on held-out trajectories, and the regression-with-injected-drops alternative fails to recover Pre>Retry / Drop AUC, showing the preference objective is not equivalent to the evaluation metrics by construction. Downstream weights (Eq. 10–11) and success rates (Table 2) are empirical outcomes on separate mixed-quality data and real-robot trials; bad-action weight analysis (Fig. 4, Table 7) uses separately annotated harmful chunks. Baselines (TOPReward, Robometer, RECAP-Value) are external methods, not load-bearing self-citation uniqueness claims. Annotation filtering (Appendix C.1) and the lack of a preference-ablated BC run are validity/completeness concerns, not circular reductions of claimed results to inputs. Score 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 2 invented entities

The method rests on a domain modeling choice (local drop-and-rebound around human-defined retry starts), standard progress labeling outside those neighborhoods, and a sizable set of hand-set temporal and loss hyperparameters. No new physical entities; the invented pieces are the ReTVL objective and soft-window preference construction. Free parameters are numerous but typical for deep robot learning papers; the load-bearing non-standard axiom is the degradation-and-recovery shape used to build preference pairs.

free parameters (5)
  • Retry window sizes Δpre / Δnear / Δpost (implemented as [r−12,r−2], [r−1,r+1], [r+2,r+12] at 5 Hz)
    Hand-chosen temporal extents that define which frames enter preference pairs; change the local supervision support.
  • Soft-window temperature τw (=6.0)
    Controls exponential decay of preference pair weights with distance from the retry keypoint.
  • Preference temperature Tpref (=0.1) and loss weights λabs=1.0, λpref=3.0
    Balance sharpness of pairwise logistic loss against absolute progress CE; set by authors, not derived.
  • Task-specific chunk threshold κ (~80th percentile of positive improvements)
    Maps value improvements to BC weights; chosen per task from offline statistics.
  • Evaluation radius K=30 and drop quantile ηi (0.9)
    Define local Drop AUC / Drop Probability metrics; fixed without reported sensitivity.
assumptions (4)
  • ad hoc to paper Around a retry keypoint, value decreases toward the near-retry region and increases after correction begins (drop-and-rebound).
    Sec. 3.2 pair construction and Limitations; used to label (h+, h−) pairs without dense progress labels.
  • domain assumption Outside retry neighborhoods, absolute progress v*_t = t/T (success) or terminal 0 (failure) is a valid global calibration target.
    Sec. 3.1 Progress Supervision; standard in progress-value literature but known to be coarse.
  • domain assumption A retry keypoint can be reliably annotated as the start of corrective behavior, and such events indicate recoverable local execution error rather than irreversible failure.
    Sec. 3.1 Data Annotations and Appendix C; annotation consistency is measured but definition excludes many error types.
  • standard math Pairwise logistic preference loss and distributional CE over discretized progress bins are valid training objectives for a scalar value in [0,1].
    Bradley–Terry / preference RL and discrete value heads as in RECAP; standard tools.
invented entities (2)
  • ReTVL value objective (Labs + soft-weighted Lpref on retry windows)
    purpose: Learn mistake-sensitive values from sparse retry annotations without dense frame labels.
    Core proposed method; evidence is empirical comparison on four tasks, not independent theory.
  • Soft-window weighting w = exp(−d/τw) for preference pairs
    purpose: Downweight pairs far from the retry keypoint to tolerate imprecise window boundaries.
    Introduced in Sec. 3.2; ablation shows modest VOC benefit; no external validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation." pith.science (2026). https://pith.science/paper/5MWKWIUX

@misc{pith2026260624633,
  author       = {Pith},
  title        = {Pith review of: Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5MWKWIUX}},
  note         = {Machine review of arXiv:2606.24633}
}
read the original abstract

Human demonstrations for robot imitation learning often contain mistakes and corrective behaviors, such as imprecise grasps, object misalignment, unstable contact, and repeated attempts. While these segments are commonly treated as noisy or suboptimal data, they provide valuable evidence about when execution deviates from a desirable path and how task feasibility can be restored. However, existing reward and value models often rely on monotonic progress assumptions, which capture coarse task advancement but may overlook local execution errors and corrective behaviors in imperfect demonstrations. In this work, we propose ReTVL (ReTry-Supervised Value Learning), a framework for learning mistake-sensitive value functions from mixed-quality robot demonstrations by leveraging retry events as sparse supervision. ReTVL captures the local degradation-and-recovery structure around mistakes by combining global progress calibration with local pairwise preference learning induced by sparsely annotated retry keypoints. The learned value model is then used to reweight demonstration chunks for downstream behavior cloning, reducing the influence of harmful execution errors while preserving useful corrective behaviors. Experiments on real-robot manipulation tasks show that ReTVL produces more fine-grained value estimates than progress-based baselines and improves imitation learning from imperfect demonstrations.

Figures

Figures reproduced from arXiv: 2606.24633 by the authors.

Figure 1
Figure 1. ReTVL turns retry events into pairwise value supervision. Progress-based value models may overlook subtle execution errors and assign overly smooth increasing values. ReTVL uses retry keypoints to learn local value drops before correction and rebounds after recovery, enabling better identification of harmful and corrective trajectory segments. supervisory signals for value learning. Specifically, the temporal neighb… view at source ↗
Figure 2
Figure 2. ReTVL learns retry-sensitive value estimates from sparse retry annotations. The model takes an observation history and language instruction as input, and predicts a scalar value through a VLM backbone and discrete value head. Training combines absolute progress calibration with retry-induced preference supervision, where values drop near retry states and rebound after recovery. ri,j marks the start of the j-th corre… view at source ↗
Figure 3
Figure 3. Visualization of value evaluation. We show value predictions on three other tasks beyond stack blocks. ReTVL captures local value drops around retry keypoints and rebounds after correction more clearly than progress-based baselines. Task Standard BC RECAP -BC ReTVL -BC Pick up Spoon 60 65 85 Stack Blocks 45 80 95 Fold Towel 50 65 80 Open Drawer 10 40 60 Average 41 63 80 [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Average training weight assigned to an￾notated bad-action chunks in recovery trajectories. Lower is better. and distinguish successful executions from failures. The main advantage of ReTVL lies in local retry￾centered metrics. It achieves consistent improvements across…
Figure 5
Figure 5. Figure 5: Real-world manipulation tasks used for policy evaluation. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Representative value-curve visualizations for ablation variants on held-out trajectories. [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 25 linked inside Pith

  1. [1]

    and Ng, A

    Abbeel, P . and Ng, A. Y. Apprenticeship learning via inverse reinforcement learning. InProceedings of the twenty-first international conference on Machine learning, pp. 1, 2004

    2004

  2. [2]

    Video- language critic: Transferable reward functions for language-conditioned robotics.arXiv preprint arXiv:2405.19988, 2024

    Alakuijala, M., McLean, R., Woungang, I., Farsad, N., Kaski, S., Marttinen, P ., and Yuan, K. Video- language critic: Transferable reward functions for language-conditioned robotics.arXiv preprint arXiv:2405.19988, 2024

    arXiv 2024

  3. [3]

    Z., Sharma, C., Shi, L

    Amin, A., Aniceto, R., Balakrishna, A., Black, K., Conley, K., Connors, G., Darpinian, J., Dhabalia, K., DiCarlo, J., Driess, D., Equi, M., Esmail, A., Fang, Y., Finn, C., Glossop, C., Godden, T., Goryachev, I., Groom, L., Hancock, H., Hausman, K., Hussein, G., Ichter, B., Jakubczak, S., Jen, R., Jones, T., Katz, B., Ke, L., Kuchi, C., Lamb, M., LeBlanc, ...

    arXiv 2025

  4. [4]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    arXiv 2025

  5. [5]

    X., Tanner, J., Vuong, Q., Walling, A., Wang, H., and Zhilinsky, U

    Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch, K., Shi, L. X., Tanner, J., Vuong, Q., Walling, A., Wang, H., and Zhilinsky, U. π0: A vision-language- action flow model for general robot control, 202...

    arXiv 2026

  6. [6]

    Bradley, R. A. and Terry, M. E. Rank analysis of incomplete block designs: I. the method of paired comparisons.Biometrika, 39(3/4):324–345, 1952

    1952

  7. [7]

    G., Gopalakrishnan, K., Han, K., Hausman, K., Herzog, A., Hsu, J., Ichter, B., Irpan, A., Joshi, N., Julian, R., Kalashnikov, D., Kuang, Y., Leal, I., Lee, L., Lee, T.-W

    Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Chen, X., Choromanski, K., Ding, T., Driess, D., Dubey, A., Finn, C., Florence, P ., Fu, C., Arenas, M. G., Gopalakrishnan, K., Han, K., Hausman, K., Herzog, A., Hsu, J., Ichter, B., Irpan, A., Joshi, N., Julian, R., Kalashnikov, D., Kuang, Y., Leal, I., Lee, L., Lee, T.-W. E., Levine, S., Lu, Y., Michale...

    arXiv 2023

  8. [8]

    Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Dabis, J., Finn, C., Gopalakrishnan, K., Hausman, K., Herzog, A., Hsu, J., Ibarz, J., Ichter, B., Irpan, A., Jackson, T., Jesmonth, S., Joshi, N. J., Julian, R., Kalashnikov, D., Kuang, Y., Leal, I., Lee, K.-H., Levine, S., Lu, Y., Malla, U., Manjunath, D., Mordatch, I., Nachum, O., Parada, C., Peralta, J...

    arXiv 2023

Show all 52 references
  1. [9]

    S., Goo, W., Nagarajan, P ., and Niekum, S

    Brown, D. S., Goo, W., Nagarajan, P ., and Niekum, S. Extrapolating beyond suboptimal demonstra- tions via inverse reinforcement learning from observations. InProceedings of the 36th International Conference on Machine Learning, volume 97 ofProceedings of Machine Learning Rese...

    2019

  2. [10]

    S., Goo, W., and Niekum, S

    Brown, D. S., Goo, W., and Niekum, S. Better-than-demonstrator imitation learning via automatically-ranked demonstrations. InProceedings of the Conference on Robot Learning, volume 100 ofProceedings of Machine Learning Research, pp. 330–359. PMLR, 2020

    2020

  3. [11]

    Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems

    Bu, Q., Cai, J., Chen, L., Cui, X., Ding, Y., Feng, S., Gao, S., He, X., Hu, X., Huang, X., et al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503.06669, 2025

    2025 arXiv

  4. [12]

    in-the-wild

    Chen, A. S., Nair, S., and Finn, C. Learning generalizable robotic reward functions from “in-the-wild” human videos. InProceedings of Robotics: Science and Systems (RSS), 2021

    2021

  5. [13]

    Sarm: Stage-aware reward modeling for long horizon robot manipulation.arXiv preprint arXiv:2509.25358, 2025

    Chen, Q., Yu, J., Schwager, M., Abbeel, P ., Shentu, F., and Wu, P . Sarm: Stage-aware reward modeling for long horizon robot manipulation.arXiv preprint arXiv:2509.25358, 2025

    2025 arXiv

  6. [14]

    J., Ren, Z., Ratliff, L

    Chen, S., Harrison, C., Lee, Y.-C., Yang, A. J., Ren, Z., Ratliff, L. J., Duan, J., Fox, D., and Kr- ishna, R. Topreward: Token probabilities as hidden zero-shot rewards for robotics.arXiv preprint arXiv:2602.19313, 2026

    2026

  7. [15]

    villa-x: Enhancing latent action modeling in vision-language-action models,

    Chen, X., Wei, H., Zhang, P ., Zhang, C., Wang, K., Guo, Y., Yang, R., Wang, Y., Xiao, X., Zhao, L., Chen, J., and Bian, J. villa-x: Enhancing latent action modeling in vision-language-action models,

  8. [16]

    URLhttps://arxiv.org/abs/2507.23682

    arXiv

  9. [17]

    F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D

    Christiano, P . F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D. Deep reinforcement learning from human preferences.Advances in neural information processing systems, 30, 2017

    2017

  10. [18]

    F., Leike, J., Brown, T

    Christiano, P . F., Leike, J., Brown, T. B., Martic, M., Legg, S., and Amodei, D. Deep reinforcement learning from human preferences. InAdvances in Neural Information Processing Systems, volume 30, 2017

    2017

  11. [19]

    Guided cost learning: Deep inverse optimal control via policy optimization

    Finn, C., Levine, S., and Abbeel, P . Guided cost learning: Deep inverse optimal control via policy optimization. InInternational conference on machine learning, pp. 49–58. PMLR, 2016

    2016

  12. [20]

    Awr: Adaptive weighting regression for 3d hand pose estimation

    Huang, W., Ren, P ., Wang, J., Qi, Q., and Sun, H. Awr: Adaptive weighting regression for 3d hand pose estimation. InProceedings of the AAAI Conference on Artificial Intelligence, pp. 11061–11068, 2020

    2020

  13. [21]

    Y., Ghosh, D., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., LeBlanc, D., Levine, S., Li-Bell, A., Mothukuri, M., Nair, S., Pertsch, K., Ren, A

    Intelligence, P ., Black, K., Brown, N., Darpinian, J., Dhabalia, K., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Galliker, M. Y., Ghosh, D., Groom, L., Hausman, K., Ichter, B., Jakubczak, S., Jones, T., Ke, L., LeBlanc, D., Levine, S., Li-Bell, A., Mothukuri, M., N...

    2025 arXiv

  14. [22]

    Vima: General robot manipulation with multimodal prompts, 2023

    Jiang, Y., Gupta, A., Zhang, Z., Wang, G., Dou, Y., Chen, Y., Fei-Fei, L., Anandkumar, A., Zhu, Y., and Fan, L. Vima: General robot manipulation with multimodal prompts, 2023. URL https: //arxiv.org/abs/2210.03094

    2023 arXiv

  15. [23]

    Kelly, M., Sidrane, C., Driggs-Campbell, K., and Kochenderfer, M. J. HG-DAgger: Interactive imitation learning with human experts. InProceedings of the IEEE International Conference on Robotics and Automation, pp. 8077–8083, 2019. doi: 10.1109/ICRA.2019.8793698

    2019 doi

  16. [24]

    Demodice: Offline imitation learning with supplementary imperfect demonstrations

    Kim, G.-H., Seo, S., Lee, J., Jeon, W., Hwang, H., Yang, H., and Kim, K.-E. Demodice: Offline imitation learning with supplementary imperfect demonstrations. InInternational Conference on Learning Representations, 2022

    2022

  17. [25]

    Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakrishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., Sanketi, P ., Vuong, Q., Kollar, T., Burchfiel, B., Tedrake, R., Sadigh, D., Levine, S., Liang, P ., and Finn, C. Openvla: An open-source vision-language-action mod...

    2024 arXiv

  18. [26]

    Dart: Noise injection for robust imitation learning, 2017

    Laskey, M., Lee, J., Fox, R., Dragan, A., and Goldberg, K. Dart: Noise injection for robust imitation learning, 2017. URLhttps://arxiv.org/abs/1703.09327

    2017 arXiv

  19. [27]

    Roboreward: General- purpose vision-language reward models for robotics.arXiv preprint arXiv:2601.00675, 2026

    Lee, T., Wagenmaker, A., Pertsch, K., Liang, P ., Levine, S., and Finn, C. Roboreward: General- purpose vision-language reward models for robotics.arXiv preprint arXiv:2601.00675, 2026. 10 Retry-Supervised Value Learning for Robot Imitation

    2026

  20. [28]

    Bridgevla: Input-output alignment for efficient 3d manipulation learning with vision-language models, 2025

    Li, P ., Chen, Y., Wu, H., Ma, X., Wu, X., Huang, Y., Wang, L., Kong, T., and Tan, T. Bridgevla: Input-output alignment for efficient 3d manipulation learning with vision-language models, 2025. URLhttps://arxiv.org/abs/2506.07961

    2025

  21. [29]

    Gr-rl: Going dexterous and precise for long-horizon robotic manipulation.arXiv preprint arXiv:2512.01801, 2025

    Li, Y., Ma, X., Xu, J., Cui, Y., Cui, Z., Han, Z., Huang, L., Kong, T., Liu, Y., Niu, H., et al. Gr-rl: Going dexterous and precise for long-horizon robotic manipulation.arXiv preprint arXiv:2512.01801, 2025

    2025

  22. [30]

    S., Zettlemoyer, L., Fox, D., Xiang, Y., Li, A., Bobu, A., Gupta, A., Tu, S., Biyik, E., and Zhang, J

    Liang, A., Korkmaz, Y., Zhang, J., Hwang, M., Anwar, A., Kaushik, S., Shah, A., Huang, A. S., Zettlemoyer, L., Fox, D., Xiang, Y., Li, A., Bobu, A., Gupta, A., Tu, S., Biyik, E., and Zhang, J. Robometer: Scaling general-purpose robotic reward models via trajectory comparisons....

    2026 arXiv

  23. [31]

    Rdt-1b: a diffusion foundation model for bimanual manipulation.arXiv preprint arXiv:2410.07864, 2024

    Liu, S., Wu, L., Li, B., Tan, H., Chen, H., Wang, Z., Xu, K., Su, H., and Zhu, J. Rdt-1b: a diffusion foundation model for bimanual manipulation.arXiv preprint arXiv:2410.07864, 2024

    2024 arXiv

  24. [32]

    J., Sodhani, S., Jayaraman, D., Bastani, O., Kumar, V ., and Zhang, A

    Ma, Y. J., Sodhani, S., Jayaraman, D., Bastani, O., Kumar, V ., and Zhang, A. Vip: Towards universal visual reward and representation via value-implicit pre-training.arXiv preprint arXiv:2210.00030, 2022

    2022 arXiv

  25. [33]

    J., Liang, W., Som, V ., Kumar, V ., Zhang, A., Bastani, O., and Jayaraman, D

    Ma, Y. J., Liang, W., Som, V ., Kumar, V ., Zhang, A., Bastani, O., and Jayaraman, D. Liv: Language- image representations and rewards for robotic control. InProceedings of the 40th International Conference on Machine Learning (ICML), 2023

    2023

  26. [34]

    Arm: Advantage reward modeling for long-horizon manipulation.arXiv preprint arXiv:2604.03037, 2026

    Mao, Y., Yu, Z., Mao, W., Li, Y., Hu, Q., Lan, Z., Zhu, M., and Chen, H. Arm: Advantage reward modeling for long-horizon manipulation.arXiv preprint arXiv:2604.03037, 2026

    2026 arXiv

  27. [35]

    Awac: Accelerating online reinforcement learning with offline datasets, 2021

    Nair, A., Gupta, A., Dalal, M., and Levine, S. Awac: Accelerating online reinforcement learning with offline datasets, 2021. URLhttps://arxiv.org/abs/2006.09359

    2021 arXiv

  28. [36]

    Ng, A. Y. and Russell, S. J. Algorithms for inverse reinforcement learning. InProceedings of the Seventeenth International Conference on Machine Learning, ICML ’00, pp. 663–670, San Francisco, CA, USA, 2000. Morgan Kaufmann Publishers Inc. ISBN 1558607072

    2000

  29. [37]

    C., Shevchuk, G., and Sadigh, D

    Palan, M., Landolfi, N. C., Shevchuk, G., and Sadigh, D. Learning reward functions by integrating human demonstrations and preferences. InProceedings of Robotics: Science and Systems, 2019

    2019

  30. [38]

    B., Kumar, A., Zhang, G., and Levine, S

    Peng, X. B., Kumar, A., Zhang, G., and Levine, S. Advantage-weighted regression: Simple and scalable off-policy reinforcement learning, 2019. URLhttps://arxiv.org/abs/1910.00177

    2019 arXiv

  31. [39]

    D., Sastry, S

    Sadigh, D., Dragan, A. D., Sastry, S. S., and Seshia, S. A. Active preference-based learning of reward functions. InProceedings of Robotics: Science and Systems, 2017. doi: 10.15607/RSS.2017.XIII.053

    2017 doi

  32. [40]

    Smolvla: A vision-language-action model for affordable and efficient robotics, 2025

    Shukor, M., Aubakirova, D., Capuano, F., Kooijmans, P ., Palma, S., Zouitine, A., Aractingi, M., Pascal, C., Russi, M., Marafioti, A., Alibert, S., Cord, M., Wolf, T., and Cadene, R. Smolvla: A vision-language-action model for affordable and efficient robotics, 2025. URL https...

    2025 arXiv

  33. [41]

    Robo- dopamine: General process reward modeling for high-precision robotic manipulation.arXiv preprint arXiv:2512.23703, 2025

    Tan, H., Chen, S., Xu, Y., Wang, Z., Ji, Y., Chi, C., Lyu, Y., Zhao, Z., Chen, X., Co, P ., et al. Robo- dopamine: General process reward modeling for high-precision robotic manipulation.arXiv preprint arXiv:2512.23703, 2025

    2025

  34. [42]

    M., Ghosh, D., Walke, H., Pertsch, K., Black, K., Mees, O., Dasari, S., Hejna, J., Kreiman, T., Xu, C., Luo, J., Tan, Y

    Team, O. M., Ghosh, D., Walke, H., Pertsch, K., Black, K., Mees, O., Dasari, S., Hejna, J., Kreiman, T., Xu, C., Luo, J., Tan, Y. L., Chen, L. Y., Sanketi, P ., Vuong, Q., Xiao, T., Sadigh, D., Finn, C., and Levine, S. Octo: An open-source generalist robot policy, 2024. URL ht...

    2024 arXiv

  35. [43]

    Dexvla: Vision-language model with plug-in diffusion expert for general robot control.arXiv preprint arXiv:2502.05855, 2025

    Wen, J., Zhu, Y., Li, J., Tang, Z., Shen, C., and Feng, F. Dexvla: Vision-language model with plug-in diffusion expert for general robot control.arXiv preprint arXiv:2502.05855, 2025

    2025 arXiv

  36. [44]

    Imitation learning from imperfect demonstration, 2019

    Wu, Y.-H., Charoenphakdee, N., Bao, H., Tangkaratt, V ., and Sugiyama, M. Imitation learning from imperfect demonstration, 2019. URLhttps://arxiv.org/abs/1901.09387

    2019 arXiv

  37. [45]

    Imitation learning from imperfect demonstration

    Wu, Y.-H., Charoenphakdee, N., Bao, H., Tangkaratt, V ., and Sugiyama, M. Imitation learning from imperfect demonstration. InProceedings of the 36th International Conference on Machine Learning, volume 97 ofProceedings of Machine Learning Research, pp. 6818–6827. PMLR, 2019. 1...

    2019

  38. [46]

    Discriminator-weighted offline imitation learning from suboptimal demonstrations

    Xu, H., Zhan, X., Yin, H., and Qin, H. Discriminator-weighted offline imitation learning from suboptimal demonstrations. InProceedings of the 39th International Conference on Machine Learning, volume 162 ofProceedings of Machine Learning Research, pp. 24725–24742. PMLR, 2022

    2022

  39. [47]

    Compliant residual dagger: Improving real-world contact- rich manipulation with human corrections.Advances in Neural Information Processing Systems, 38: 139559–139581, 2026

    Xu, X., Hou, Y., Liu, Z., and Song, S. Compliant residual dagger: Improving real-world contact- rich manipulation with human corrections.Advances in Neural Information Processing Systems, 38: 139559–139581, 2026

    2026

  40. [48]

    Rise: Self-improving robot policy with compositional world model.arXiv preprint arXiv:2602.11075, 2026

    Yang, J., Lin, K., Li, J., Zhang, W., Lin, T., Wu, L., Su, Z., Zhao, H., Zhang, Y.-Q., Chen, L., et al. Rise: Self-improving robot policy with compositional world model.arXiv preprint arXiv:2602.11075, 2026

    2026 arXiv

  41. [49]

    Aloe: Action-level off-policy evaluation for vision-language-action model post-training.arXiv preprint arXiv:2602.12691, 2026

    Yang, R., Wang, H., Liu, C., Yan, X., Wang, Y., Du, X., Yue, S., Liu, Y., Zhang, C., Qi, L., et al. Aloe: Action-level off-policy evaluation for vision-language-action model post-training.arXiv preprint arXiv:2602.12691, 2026

    2026 arXiv

  42. [50]

    Confidence-aware imitation learning from demonstrations with varying optimality

    Zhang, S., Cao, Z., Sadigh, D., and Sui, Y. Confidence-aware imitation learning from demonstrations with varying optimality. InAdvances in Neural Information Processing Systems, volume 34, 2021

    2021

  43. [51]

    Vlas: Vision-language- action model with speech instructions for customized robot manipulation, 2025

    Zhao, W., Ding, P ., Zhang, M., Gong, Z., Bai, S., Zhao, H., and Wang, D. Vlas: Vision-language- action model with speech instructions for customized robot manipulation, 2025. URL https: //arxiv.org/abs/2502.13508

    2025 arXiv

  44. [52]

    D., Maas, A

    Ziebart, B. D., Maas, A. L., Bagnell, J. A., Dey, A. K., et al. Maximum entropy inverse reinforcement learning. InAaai, volume 8, pp. 1433–1438. Chicago, IL, USA, 2008. 12 Retry-Supervised Value Learning for Robot Imitation A Implementation Details A.1 Value Model Training Dat...

    2008

Pith tools

Reviewed July 12, 2026 · model on record in the stance chip above.