Pith. sign in

REVIEW 3 major objections 6 minor 50 references

ValueFormer: A Causal Transformer Value Function with Stage-Aware Labels for Semi-Autonomous Vision-Language-Action Policies

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The per-frame label, not the architecture, determines whether a learned progress value can tell a robot when it is failing.

desk verdict An honest, well-engineered paper on per-frame value labels for BC VLAs whose central label-shape claim is plausible but only tested in a legacy configuration, so the headline result is slightly ahead of the evidence. read the letter →

arxiv 2608.02958 v1 pith:KOW26RZW submitted 2026-08-03 cs.RO cs.AI

Inkyu Sa , Konstantin Stulov , Rajat Bhageria This is my paper · ORCID
classification cs.ROcs.AI
keywords valuefunctionvision-language-actionpoliciesbehaviorcloningfailuredetectionstage-awarelabelsMonteCarloreturnscausaltransformerbimanualmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the hard part of giving a behavior-cloned vision-language-action policy a sense of progress is not the model but the per-frame training label. It proposes a small causal transformer, ValueFormer, that watches the same camera views and joint state as the policy and outputs two signals per frame: a smooth Monte Carlo value for advantage estimation and a sharp binary value for mistake detection. The central claim is that a stage-aware success-then-decay label—failed episodes follow the success curve up to the failure stage and decay smoothly afterward—is the only one among five shapes that avoids leaking the final outcome into frame zero and avoids a discontinuity at the failure frame, and it also gives the best validation loss and rollout error. If correct, any behavior-cloned VLA can be monitored and post-trained with a policy-agnostic critic at low cost, without changing the policy itself.

What carries the argument

The central object is the stage-aware success-then-decay Monte Carlo label of Eq. (3), together with a two-head causal transformer trained on frozen self-supervised ViT-L/16 features (DINOv3) with six views, joint state, and time features. The label's exponential shape is load-bearing in two ways: with $\gamma=0.99$ the pre-failure prefix matches the success curve, so the network is never asked to regress identical observations onto opposite targets, and the exponential rather than linear form keeps the horizon-differenced advantage $A_t = V_{\mathrm{mc}}(t+H_A) - V_{\mathrm{mc}}(t)$ non-constant across a successful rollout, so later frames carry larger advantage. The causal mask (each frame attends only to past and present frames) makes the value valid for online prediction, and the dual head splits the smooth critic target from the sharp detection target so one forward pass serves both.

What would settle it

Run the same five-shape ablation under the deployed six-view dual-head configuration and compare validation BCE and rollout MAE; if MC-smooth no longer wins on both, or if a re-annotation of the 88 failures by independent judges shows stage disagreement above 10%, the central label-shape claim is not supported.

Watch

Extended reading notes

Core claim

The discovery is that the per-frame label shape dominates the error: across five fail-episode label shapes under one fixed architecture, the validation loss spread is roughly 4x, while no architectural change moved results comparably. ValueFormer's stage-aware label sets a failed episode's pre-failure frames to the same rising exponential used for successes, $v^{\mathrm{succ}}_k = \gamma^{N-1-k}$, and after the annotated failure stage decays with the same discount, $v^{\mathrm{fail}}_k = v^{\mathrm{succ}}_{k_{\mathrm{fail}}} \cdot \gamma^{k-k_{\mathrm{fail}}}$. This is claimed to be the only shape that simultaneously has no outcome-dependent leakage at $t=0$, no hard cliff at the failure frame, and a smooth post-failure tail that preserves partial credit for completed stages. In addition, a second per-frame binary head supervised by mistake intervals $(t_{\mathrm{start}}, t_{\mathrm{end}})$ rather than a single failure time lets transient mistakes the policy recovers from also contribute training signal. On a real-robot sandwich-assembly task, the model reaches validation MSE about $3\times10^{-4}$ and reproduces four rollout signatures, and critic-derived per-frame training weights raise task completion from 70% to 85%, with the paper noting that at $n=20$ this gain is within noise.

Load-bearing premise

The stage-aware labels rest on hand-annotated failure stages and mistake intervals for 88 failure rollouts and 70 recovered-mistake successes, with no reported inter-annotator agreement; if those annotations are systematically off, the label shape and every downstream result inherit the error.

Editorial extensions

If this is right

  • Any behavior-cloned VLA in the same policy family can be given a per-frame progress value and mistake alert without retraining the policy, since ValueFormer is policy-agnostic and consumes the same observation stream.
  • The stage-aware label recipe should transfer to other multi-stage tasks with ordered stages and sparse success/failure outcomes, because the label shape, not the encoder, carries the result.
  • Human-in-the-loop intervention flags provide mistake intervals at fleet scale, so the detection head can be improved without manual video annotation.
  • Critic-derived per-frame weights can be baked into VLA post-training to remove specific failure modes such as repeat-picks, even when overall completion gains are within noise.
  • The batched bf16 encoder path cuts the live serving cost by 3–5x, making it feasible to run the critic at 2 Hz on the same GPU as the policy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same 'leakage at t=0' failure mode likely afflicts any dense reward or process-reward label whose early-frame target is set by the final outcome; testing for it should be standard when designing per-frame supervision in other sequential decision domains.
  • The label-shape ordering was established under a legacy four-view single-head configuration, and the paper explicitly leaves the full five-shape ablation under the deployed six-view dual-head setup to future work, so whether the ordering survives that configuration is an open question rather than a demonstrated result.
  • A foundation-model labeler could plausibly pre-populate mistake intervals with a human confirming them, which would combine the paper's interval supervision with automated labeling and remove the manual-annotation scaling bottleneck.
  • Because the advantage signal $A_t$ is non-constant only near the terminal, the same critic could serve as a trajectory-ranking filter for offline post-training data, not just a per-frame loss weight.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper tackles the absence of a per-frame progress signal in behavior-cloned VLA policies, proposing ValueFormer, a compact causal transformer over frozen DINOv3 features with two output heads: a smooth Monte Carlo value V_mc for advantage estimation and a sharp binary head V_bin for online mistake detection. The two labeling contributions are a stage-aware success-then-decay return for failed episodes (Eq. 3) and segment-based mistake-interval supervision (Eq. 4). The central claim, developed in Sections V-C and VI-D, is that the per-frame label shape, not the architecture, is the binding constraint on learning a useful progress value; it is supported by a five-scheme ablation (MC-smooth, outcome-scaled, cliff, alpha-linear, late-diverge) run under the legacy four-view single-head recipe. Deployment results include intervention-derived labels that roughly double held-out detection average precision (Section VI-E), an on-robot A/B in which two critic-derived training weights raise completion from 70% to 85% though explicitly within noise at n=20 (Section VI-G), and a batched bf16 encoder path that cuts per-tick serving cost 3-5x (Section VI-H). The paper is unusually transparent, disclosing the n=20 power limitation, the circularity of the HIL ground truth, and the failure of its own vf-awr prediction.

Significance. If the central claim holds, the paper offers a transferable design pattern for value learning in behavior-cloned systems: dense continuous supervision is only useful when its shape encodes when a failure began and how much partial credit remains, and the dual-head pattern (smooth critic plus sharp detector sharing one backbone) cleanly resolves two conflicting target requirements in one forward pass. The controlled five-scheme ablation is a genuine isolation of the label effect, and the engineering content is strong, with exact hyperparameters, parameter counts, measured bf16 numerics, and an instructive latency analysis. The reporting is exemplary in places: a wrong prediction of the authors' own (vf-awr) is presented as such rather than revised, consistent with the paper's explicit statistical hedging. The significance is bounded by the single-task evidence base, the small evaluation sizes (n=20 A/B, 21 held-out rollouts without intervals), and the fact that the label-shape ordering is not yet demonstrated in the deployed six-view dual-head configuration.

major comments (3)
  1. [V-C and VI-D] The paper's central claim that the per-frame label, rather than the architecture, is the hard part is established only under the legacy four-view, 32-frame, single-head recipe: Section V-C asserts that the label-shape ordering is 'independent of the encoder width or window length' without reporting an experiment, and explicitly defers the five-scheme ablation under the dual-head recipe to future work. The deployed system (Table I; Sections VI-E through VI-H) uses six views, a 16-frame window, per-frame V_bin supervision, and intervention-augmented training, so the evidence for the central claim does not currently cover the configuration in which the headline results are produced, and every downstream use of ValueFormer inherits this gap. Please re-run the five-scheme ablation under the deployed configuration, or provide a targeted experiment varying window length and view count under the MC-smooth label that demonstrates the ordering of Section VI-D is stable.
  2. [Tables II and III; Sections VI-A and VI-D] There is a numerical tension between Table II and Table III that bears directly on the 'label, not architecture' claim. Under the legacy configuration, MC-smooth gives mean-V-fail = 0.390 and Delta V = 0.143 on the held-out 21-rollout set, while the main configuration reports mean-V-fail = 0.000 and Delta V = 0.537 on the validation split; the mean-V-success values nearly coincide (0.533 vs 0.537), so the discrepancy is unlikely to be explained by the different evaluation sets. If the deployed architecture with the same Eq. (3) label really drives failure-frame predictions to zero, the architectural and data changes alone move Delta V by nearly a factor of four, the same magnitude as the four-fold validation-loss spread attributed to label shape in Section VI-D, which contradicts the Section VII.1 assertion that no architectural change produced a comparable effect; it also sits uneasily with the positive pre-failure labels of Eq. (3) and the reported validation MSE of 3e-4. Please state exactly how mean-V-fail is computed in each table and reconcile the two numbers.
  3. [VI-E and VIII(b)] The detection evaluation in Section VI-E is partly circular: the manual+intervention critic is trained on a shifted version of the same per-frame intervention flag against which it is evaluated on the held-out 24 episodes, so the reported average-precision gain (0.38 to 0.82) and the 35% to 95% detected-before-takeover improvement partly measure agreement with the label source rather than with an independent notion of failure. The paper's caveat in Section VI-E is accurate, but Section VIII(b) restates the result as 'roughly double held-out detection quality' without it, and the contribution claims inherit that framing. The lead-time column is the one label-independent quantity and should carry the claim; an evaluation against hand-labeled mistake onsets on the held-out episodes, which the paper itself names as the next measurement, is needed before the detection claim is taken at face value.
minor comments (6)
  1. [VI-D, Table III] The rollout-level metrics in Table III rest on 12 success and 9 failure rollouts with no confidence intervals or per-rollout values; please add error bars or a scatter so the reader can assess whether MC-smooth's roughly 2x MAE advantage is robust at this sample size.
  2. [V-D, Table III] Because each scheme is selected on validation MSE and then evaluated in Table III on BCE against its own target, part of MC-smooth's BCE advantage may reflect that a smooth target is intrinsically easier to fit than a step or scaled target; please discuss or control for this by reporting BCE on a common reference target.
  3. [IV-B] No inter-annotator agreement or verification of the keyword matcher's stage mapping is reported for the manual annotations that drive Eqs. (3) and (4); a small double-annotation reliability check would materially strengthen confidence in the label route that the whole paper rests on.
  4. [VI-G] The on-robot A/B was run one to two weeks after the control with uncontrolled ingredient staging across sessions, and the completion gain is already within noise (Fisher p=0.45); the paired re-evaluation with shared initial conditions and a McNemar test that the authors propose should be reported before the 85% figure is presented as a result rather than a trend.
  5. [VII.1] The claim that no architectural tuning produced a change comparable to the label effect is not supported by any reported architecture-ablation table; please cite the specific experiments or soften the sentence to match the presented evidence.
  6. [IV-D, Figure 7] Figure 7 and its caption describe the single-head recipe while the surrounding text describes the deployed dual-head configuration; please update the figure or the caption so that the configuration used for the reported results is the one illustrated.

Circularity Check

2 steps flagged · score 6.0 of 10

Partial circularity: the label-shape ablation is scored on each scheme's own target, and the HIL detector is trained and evaluated on the same intervention flag; both reduce partly by construction, though the paper discloses the second issue.

  1. self definitional [Section VI-D, Table III (labeling-scheme ablation)]
    "Validation BCE loss is on the scheme’s own target; all other columns are computed on the same 12-success / 9-failure rollout set. ... When evaluated as a regression problem, MC-smooth wins by roughly a factor of three on validation BCE loss and a factor of two on rollout MAE."

    The quantitative support for the central claim ('the label, not the architecture') is that MC-smooth has the lowest validation BCE/MAE. But the validation loss is explicitly computed on each scheme's own target, i.e., the very label shape being ablated; there is no common reference target. A model's BCE on its own labels measures how well it fits that label, not how good the label is for the downstream value task. The reported ~4x spread and 'factor of three' win are therefore partly determined by the target definition itself, making the headline ablation evidence self-referential rather than an independent comparison across label shapes.

  2. fitted input called prediction [Section VI-E, Table IV (intervention-derived segments)]
    "the held-out ground truth is itself intervention-derived (with the λ=1.5s reaction-lag correction), so it measures agreement with where the operator intervened rather than an independent notion of failure; the lead-time column, measured against the raw onset, is the exception."

    The manual+intervention critic is trained on Eq. (4) targets built from the HIL intervention flag, and its held-out detection metrics are computed against ground truth derived from that same flag (with a lag correction). The large gains in average precision (0.38→0.82) and detected-before-takeover rate (35%→95%) therefore partly measure the model's agreement with the label source it was fitted to, not an independent failure signal. The paper explicitly identifies the lead-time column as the only label-independent measurement, which confirms that the core detection claim reduces to same-source supervision for the other metrics.

full rationale

The paper is unusually transparent: it discloses that the HIL held-out ground truth is intervention-derived and that the completion gain is within noise at n=20, and it publicly records that its vf-awr prediction was wrong. Those disclosures prevent an 8-10 score. However, two load-bearing quantitative claims have a circular component. First, the label-shape ablation that motivates the central 'label, not architecture' thesis reports validation BCE on each scheme's own target, so the claimed advantage of MC-smooth is partly a comparison of each model against its own definition rather than against a shared measure of label quality. Second, the intervention-supervision detection result is trained and evaluated against the same operator intervention flag, so the AP/AUROC improvement largely reflects label-source agreement; the paper's caveat explicitly limits the independent part to the lead-time column. The theoretical arguments about t=0 leakage and discontinuity are independent of these metrics and give the label design genuine content, so the circularity is partial rather than total. Overall score 6: one or more reported predictions/results reduce in part by construction, while the central label design still retains independent, non-circular content.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claims rest on a small number of hand-set parameters, notably the discount gamma and the advantage lookahead, and on domain assumptions about task structure and annotation quality. No new physical or model entities are invented beyond label curves and architecture components, which are not separate entities.

free parameters (6)
  • gamma = 0.99
    Horizon discount used in the success and failure label formulas (Eq. 2 and 3); ablated values 0.95, 0.99, 0.999 in Figure 3.
  • H_A (advantage lookahead) = 50 frames
    Lookahead horizon in A_t = V_mc(t+H_A) - V_mc(t) for vf-awr; chosen without reported sensitivity analysis.
  • beta_awr = 0.3
    Temperature in advantage weight clip(e^{A_t/beta_awr}); no sweep reported.
  • lambda = 1.5 s
    Lead shift applied to intervention flags when constructing binary labels; fixed assumption about human reaction time.
  • amb = 5.0 s
    Ambiguous run-up window excluded from binary loss for intervention-derived segments.
  • intervention_share_cap = 0.2
    Cap on the fraction of intervention-labeled samples in training, chosen to avoid displacing manual annotations.
assumptions (4)
  • domain assumption The sandwich task decomposes into six ordered stages with known order and stage boundaries.
    Stage-aware labels in Section IV-A rely on s_fail and N_s=6; no stage detector or learned segmentation is used.
  • domain assumption Manual annotations of failure stage, failure time, and mistake intervals are ground truth.
    Section IV-B describes a single annotator and a keyword matcher, but provides no inter-annotator agreement or validation.
  • domain assumption Frozen DINOv3 features are sufficient visual representations for value prediction.
    The visual frontend in Section IV-D uses a frozen DINOv3 ViT-L/16 and the paper does not fine-tune or compare representation families.
  • domain assumption The behavior-cloning policy remains fixed while the value model is trained and used.
    The claimed policy-agnostic property in Section II relies on the policy being fixed and the critic running externally.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ValueFormer: A Causal Transformer Value Function with Stage-Aware Labels for Semi-Autonomous Vision-Language-Action Policies." pith.science (2026). https://pith.science/paper/KOW26RZW

@misc{pith2026260802958,
  author       = {Pith},
  title        = {Pith review of: ValueFormer: A Causal Transformer Value Function with Stage-Aware Labels for Semi-Autonomous Vision-Language-Action Policies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KOW26RZW}},
  note         = {Machine review of arXiv:2608.02958}
}
read the original abstract

Vision-Language-Action (VLA) policies trained by behavior cloning fail silently: from the action stream alone, a collapsing rollout looks much like one making clean progress, because imitation supplies no notion of progress. Reinforcement learning would supply one, but it is impractical here, where real-robot experience is costly and deformable food resists simulation. The cheap alternative, a terminal success / failure bit, is learnable in principle yet far too sparse to say when a rollout went wrong. We argue that the per-frame label, not the architecture, is the hard part: to be useful it must be dense, continuous, and correctly shaped. We present ValueFormer, a compact policy-agnostic causal transformer over a frozen DINOv3 backbone that emits two per-frame signals in one forward pass: a smooth Monte Carlo value, V_mc, for advantage estimation and a sharp binary value for online mistake detection, targets that pull in opposite directions by design. Failed episodes are labeled with a stage-aware, success-then-decay return that preserves the success curve before the failure stage, and detection is supervised from mistake intervals rather than a single failure time, so mistakes the policy recovers from also carry signal. On a real-robot bimanual sandwich-assembly task 1,427 episodes), a critic-derived per-frame training weight lifts task completion from 70% to 85% (within noise at n=20), and a batched bf16 encoder cuts the live serving cost 3~5 times so the critic runs at 2 Hz alongside the policy on a single GPU.

Figures

Figures reproduced from arXiv: 2608.02958 by the authors.

Figure 1
Figure 1. ValueFormer running live alongside a VLA policy [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. ValueFormer system overview covering dataset construction, training, and real-time inference. Details in Sections IV-A– [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Monte Carlo return labels. (Left) effect of the discount [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (12 more)
Figure 5
Figure 5. Figure 5: The four alternative fail-episode label shapes (A: [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The six views that make up one ValueFormer input frame. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: ValueFormer data flow for one prediction. The six views of Figure 6 pass through the shared frozen DINOv3 ViT-L/16 [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Why the transformer body is causal. Attention for a [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Dataset composition: expert successes, rollout suc [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Four canonical rollout types on held-out episodes: [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Per-episode predicted V (s) (blue) vs. MC label (green dashed) curves on held-out episodes, with the clean-success reference for the same episode length (gray dotted) shown for the two failures. Green title: success; red title: failure. cheese is placed. On a stuck-sc…
Figure 12
Figure 12. Figure 12: Per-episode predicted V (blue) versus the scheme’s own MC label (green dashed) on six held-out rollouts (columns) under the five label shapes (rows: MC-smooth, A: outcome-scaled, B: cliff, C-linear, C-late). MC-smooth tracks the success target tightly on the success c…
Figure 14
Figure 14. Figure 14: Every manual+intervention configuration (48 runs, blue) beats every manual-only one (6 runs, gray) on held-out average precision, with a clean gap. The effect is robust to label shape, intervention share, context length, and detection￾head weight [PITH_FULL_IMAGE:fig…
Figure 13
Figure 13. Figure 13: Threshold-free detection on the 24 held-out HIL episodes. The manual+intervention critic (blue) dominates the manual-only critic (gray) across the whole precision–recall (left) and ROC (right) range: AP 0.38 → 0.82, AUROC 0.69 → 0.94. operator reacts goes from 35% to …
Figure 16
Figure 16. Figure 16: How the critic’s outputs become per-frame training weights, on one training episode; shaded spans are human takeovers. [PITH_FULL_IMAGE:figures/full_fig_p017_16.png]
Figure 17
Figure 17. Figure 17: The critic-derived weights against the flag-only IWR control. (Left) per-subtask mean score, pooled over both [PITH_FULL_IMAGE:figures/full_fig_p018_17.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 26 canonical work pages

  1. [1]

    π 0: A vision-language-action flow model for general robot control,

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, L. X. Shi, J. Tanner, Q. Vuong, A. Walling, H. Wang, and U. Zhilinsky, “π 0: A vision-language-action flow model for general robot control,”arXiv preprint arXiv:2410...

    arXiv

  2. [2]

    π 0.5: A vision- language-action model with open-world generalization,

    K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, M. Y . Galliker, D. Ghosh, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, D. LeBlanc, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, A. Z. Ren, L. X. Shi, L. Smith, J. T. Springenberg, K. Stachowicz, J. Tanner, Q. Vuong, H. Walke, A. Walling, H. Wan...

    arXiv 2025

  3. [3]

    π ∗ 0.6: A VLA that learns from experience,

    A. Amin, R. Aniceto, A. Balakrishna, K. Black, K. Conley, G. Connors, J. Darpinian, K. Dhabalia, J. DiCarlo, D. Driess, M. Equi, Y . Fang, C. Finn, C. Glossop, T. Godden, I. Goryachev, L. Groom, H. Hancock, K. Hausman, G. Hussein, B. Ichter, S. Jakubczak, R. Jen, T. Jones, B. Katz, L. Ke, C. Kuchi, M. Lamb, D. LeBlanc, Y . Lu, V . Mano, M. Mothukuri, K. P...

    arXiv 2025

  4. [4]

    OpenVLA: An open- source vision-language-action model,

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketiet al., “OpenVLA: An open- source vision-language-action model,”arXiv preprint arXiv:2406.09246, 2024

    arXiv 2024

  5. [5]

    RDT-1B: A diffusion foundation model for bimanual manipulation,

    S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu, “RDT-1B: A diffusion foundation model for bimanual manipulation,”arXiv preprint arXiv:2410.07864, 2024

    arXiv 2024

  6. [6]

    GigaBrain-0.5M ∗: A VLA that learns from world model-based reinforcement learning,

    GigaAI, “GigaBrain-0.5M ∗: A VLA that learns from world model-based reinforcement learning,”arXiv preprint arXiv:2602.12099, 2026

    arXiv 2026

  7. [7]

    LeRobot: State-of-the-art machine learning for real-world robotics in PyTorch,

    R. Cadene, S. Alibert, A. Soare, Q. Gallouedec, A. Zouitine, and T. Wolf, “LeRobot: State-of-the-art machine learning for real-world robotics in PyTorch,” https://github.com/huggingface/lerobot, 2024

    work page 2024

  8. [8]

    Sim ´eoni, H

    O. Sim ´eoni, H. V . V o, M. Seitzeret al., “DINOv3,”arXiv preprint arXiv:2508.10104, 2025

    arXiv 2025

Show all 50 references
  1. [9]

    RT-1: Robotics transformer for real-world control at scale,

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsuet al., “RT-1: Robotics transformer for real-world control at scale,”arXiv preprint arXiv:2212.06817, 2022

    2022 arXiv

  2. [10]

    RT-2: Vision-language- action models transfer web knowledge to robotic control,

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, X. Chen, K. Choroman- ski, T. Ding, D. Driess, A. Dubey, C. Finnet al., “RT-2: Vision-language- action models transfer web knowledge to robotic control,”arXiv preprint arXiv:2307.15818, 2023

    2023 arXiv

  3. [11]

    OpenPI: An open-source implementation of theπ 0 vision-language-action model,

    Physical Intelligence, “OpenPI: An open-source implementation of theπ 0 vision-language-action model,” https://github.com/ Physical-Intelligence/openpi, 2024

    2024

  4. [12]

    Flow matching for generative modeling,

    Y . Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow matching for generative modeling,” inInternational Conference on Learning Representations (ICLR), 2023

    2023

  5. [13]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, S. Feng, Y . Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” in Robotics: Science and Systems (RSS), 2023

    2023

  6. [14]

    Learning fine-grained bimanual manipulation with low-cost hardware,

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” inRobotics: Science and Systems (RSS), 2023

    2023

  7. [15]

    Xiaomi-Robotics-0: An open-sourced vision- language-action model with real-time execution,

    Xiaomi Robotics, “Xiaomi-Robotics-0: An open-sourced vision- language-action model with real-time execution,”arXiv preprint arXiv:2602.12684, 2026

    2026

  8. [16]

    Human-level control through deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, 2015

    2015

  9. [17]

    Continuous control with deep reinforcement learning,

    T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y . Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,”arXiv preprint arXiv:1509.02971, 2015

    2015 arXiv

  10. [18]

    Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,” inInternational Conference on Machine Learning (ICML), 2018

    2018

  11. [19]

    Q-Transformer: Scalable offline reinforcement learning via autoregressive Q-Functions,

    Y . Chebotar, Q. Vuong, A. Irpan, K. Hausman, F. Xia, Y . Lu, A. Kumar, T. Yu, A. Herzog, K. Pertschet al., “Q-Transformer: Scalable offline reinforcement learning via autoregressive Q-Functions,” inConference on Robot Learning (CoRL), 2023

    2023

  12. [20]

    Offline reinforcement learning with implicit Q-Learning,

    I. Kostrikov, A. Nair, and S. Levine, “Offline reinforcement learning with implicit Q-Learning,” inInternational Conference on Learning Representations (ICLR), 2022

    2022

  13. [21]

    Conservative Q-Learning for offline reinforcement learning,

    A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative Q-Learning for offline reinforcement learning,” inAdvances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  14. [22]

    VIP: Towards universal visual reward and representation via value-implicit pre-training,

    Y . J. Ma, S. Sodhani, D. Jayaraman, O. Bastani, V . Kumar, and A. Zhang, “VIP: Towards universal visual reward and representation via value-implicit pre-training,” inInternational Conference on Learning Representations (ICLR), 2023

    2023

  15. [23]

    LIV: Language-image representations and rewards for robotic control,

    Y . J. Ma, V . Kumar, A. Zhang, O. Bastani, and D. Jayaraman, “LIV: Language-image representations and rewards for robotic control,” in International Conference on Machine Learning (ICML), 2023

    2023

  16. [24]

    Contrastive learning as goal-conditioned reinforcement learning,

    B. Eysenbach, T. Zhang, S. Levine, and R. Salakhutdinov, “Contrastive learning as goal-conditioned reinforcement learning,” inAdvances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  17. [25]

    GoFAR: Offline goal- conditioned reinforcement learning via state-occupancy matching,

    Y . J. Ma, J. Yan, D. Jayaraman, and O. Bastani, “GoFAR: Offline goal- conditioned reinforcement learning via state-occupancy matching,” in Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  18. [26]

    Diffusion policy policy optimization,

    A. Z. Ren, J. Lidard, L. L. Ankile, A. Simeonov, P. Agrawal, A. Majum- dar, B. Burchfiel, H. Dai, and M. Simchowitz, “Diffusion policy policy optimization,”arXiv preprint arXiv:2409.00588, 2024

    2024 arXiv

  19. [27]

    VLAC: A vision-language-action-critic model for robotic real-world reinforcement learning,

    S. Zhaiet al., “VLAC: A vision-language-action-critic model for robotic real-world reinforcement learning,”arXiv preprint arXiv:2509.15937, 2025

    2025

  20. [28]

    SAFE: Multitask failure detection for vision-language- action models,

    Q. Guet al., “SAFE: Multitask failure detection for vision-language- action models,” inAdvances in Neural Information Processing Systems (NeurIPS), 2025, arXiv:2506.09937

    2025

  21. [29]

    AHA: A vision-language model for detecting and reason- ing over failures in robotic manipulation,

    J. Duanet al., “AHA: A vision-language model for detecting and reason- ing over failures in robotic manipulation,” inInternational Conference on Learning Representations (ICLR), 2025, arXiv:2410.00371

    2025 arXiv

  22. [30]

    I-FailSense: General robotic failure detection with vision-language models,

    I-FailSense Authors, “I-FailSense: General robotic failure detection with vision-language models,”arXiv preprint arXiv:2509.16072, 2025

    2025

  23. [31]

    Score the steps, not just the goal: VLM-based subgoal evaluation for long-horizon manipulation,

    StepEval Authors, “Score the steps, not just the goal: VLM-based subgoal evaluation for long-horizon manipulation,”arXiv preprint arXiv:2509.19524, 2025

    2025

  24. [32]

    RoboCLIP: One demonstration is enough to learn robot policies,

    S. A. Sontakke, J. Zhang, S. M. R. Arnold, K. Pertsch, E. Biyik, D. Sadigh, C. Finn, and L. Itti, “RoboCLIP: One demonstration is enough to learn robot policies,”arXiv preprint arXiv:2310.07899, 2023

    2023 arXiv

  25. [33]

    Eureka: Human-level reward design via coding large language models,

    Y . J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Jayaraman, Y . Zhu, L. Fan, and A. Anandkumar, “Eureka: Human-level reward design via coding large language models,” inInternational Conference on Learning Representations (ICLR), 2024

    2024

  26. [34]

    JHU-ISI gesture and skill assessment working set (JIGSAWS): A surgical activity dataset for human motion modeling,

    Y . Gao, S. S. Vedula, C. E. Reiley, N. Ahmidi, B. Varadarajan, H. C. Lin, L. Tao, L. Zappella, B. B ´ejar, D. D. Yuh, C. C. G. Chen, R. Vidal, S. Khudanpur, and G. D. Hager, “JHU-ISI gesture and skill assessment working set (JIGSAWS): A surgical activity dataset for human mot...

    2014

  27. [35]

    EndoNet: A deep architecture for recognition tasks on laparoscopic videos,

    A. P. Twinanda, S. Shehata, D. Mutter, J. Marescaux, M. de Mathelin, and N. Padoy, “EndoNet: A deep architecture for recognition tasks on laparoscopic videos,”IEEE Transactions on Medical Imaging, vol. 36, no. 1, pp. 86–97, 2017

    2017

  28. [36]

    MS-TCN: Multi-stage temporal convolutional network for action segmentation,

    Y . A. Farha and J. Gall, “MS-TCN: Multi-stage temporal convolutional network for action segmentation,” inCVPR, 2019

    2019

  29. [37]

    ASFormer: Transformer for action segmentation,

    F. Yi, H. Wen, and T. Jiang, “ASFormer: Transformer for action segmentation,” inBMVC, 2021

    2021

  30. [38]

    The THUMOS challenge on action recognition for videos “in the wild

    H. Idrees, A. R. Zamir, Y .-G. Jiang, A. Gorban, I. Laptev, R. Sukthankar, and M. Shah, “The THUMOS challenge on action recognition for videos “in the wild”,”Computer Vision and Image Understanding, vol. 155, pp. 1–23, 2017

    2017

  31. [39]

    ActivityNet: A large-scale video benchmark for human activity under- standing,

    F. Caba Heilbron, V . Escorcia, B. Ghanem, and J. Carlos Niebles, “ActivityNet: A large-scale video benchmark for human activity under- standing,” inCVPR, 2015

    2015

  32. [40]

    Scaling egocentric vision: The EPIC-KITCHENS dataset,

    D. Damen, H. Doughty, G. M. Farinella, S. Fidler, A. Furnari, E. Kaza- kos, D. Moltisanti, J. Munro, T. Perrett, W. Price, and M. Wray, “Scaling egocentric vision: The EPIC-KITCHENS dataset,” inECCV, 2018

    2018

  33. [41]

    ActionFormer: Localizing moments of actions with transformers,

    C.-L. Zhang, J. Wu, and Y . Li, “ActionFormer: Localizing moments of actions with transformers,” inECCV, 2022

    2022

  34. [42]

    Let’s verify step by step,

    H. Lightman, V . Kosaraju, Y . Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe, “Let’s verify step by step,” inICLR, 2024

    2024

  35. [43]

    R3M: A universal visual representation for robot manipulation,

    S. Nair, A. Rajeswaran, V . Kumar, C. Finn, and A. Gupta, “R3M: A universal visual representation for robot manipulation,” inCoRL, 2022

    2022

  36. [44]

    Value prediction network,

    J. Oh, S. Singh, and H. Lee, “Value prediction network,” inNeurIPS, 2017. 22

    2017

  37. [45]

    Hi Robot: Open-ended instruction following with hierarchical vision-language-action models,

    L. X. Shi, B. Ichter, M. Equi, L. Ke, K. Pertsch, Q. Vuong, J. Tanner, A. Walling, H. Wang, N. Fusai, A. Li-Bell, D. Driess, L. Groom, S. Levine, and C. Finn, “Hi Robot: Open-ended instruction following with hierarchical vision-language-action models,” inProceedings of the 42n...

    2025

  38. [46]

    Do as i can, not as i say: Grounding language in robotic affordances,

    M. Ahn, A. Brohan, N. Brown, Y . Chebotar, O. Cortes, B. David, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzoget al., “Do as i can, not as i say: Grounding language in robotic affordances,”arXiv preprint arXiv:2204.01691, 2022

    2022 arXiv

  39. [47]

    Inner monologue: Embod- ied reasoning through planning with language models,

    W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y . Chebotaret al., “Inner monologue: Embod- ied reasoning through planning with language models,” inConference on Robot Learning (CoRL), 2022

    2022

  40. [48]

    Interactive language: Talking to robots in real time,

    C. Lynch, A. Wahid, J. Tompson, T. Ding, J. Betker, R. Baruch, T. Armstrong, and P. Florence, “Interactive language: Talking to robots in real time,” inIEEE Robotics and Automation Letters, 2023

    2023

  41. [49]

    What matters in language- conditioned robotic imitation learning over unstructured data,

    O. Mees, L. Hermann, and W. Burgard, “What matters in language- conditioned robotic imitation learning over unstructured data,”IEEE Robotics and Automation Letters, 2022

    2022

  42. [50]

    HG-DAgger: Interactive imitation learning with human experts,

    M. Kelly, C. Sidrane, K. Driggs-Campbell, and M. J. Kochenderfer, “HG-DAgger: Interactive imitation learning with human experts,” in IEEE International Conference on Robotics and Automation (ICRA), 2019

    2019

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.