{"id":"d872c38e-cc1d-457d-aa17-5c79a6e2b641","arxiv_id":"2608.02958","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"ValueFormer trains a causal transformer on stage-aware success-then-decay Monte Carlo labels to give robot policies a per-frame progress value and binary mistake alert in one forward pass.","lead":"ValueFormer is a compact causal transformer that reads a frozen visual backbone and outputs both a smooth progress value and a sharp failure probability for robot manipulation. The paper argues that the per-frame label, not the architecture, decides whether such a progress signal works.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central label-shape claim is tested only in a legacy four-view single-head configuration; whether it transfers to the deployed six-view dual-head system is untested, so the main evidence for 'label, not architecture' may not apply.","rationale":"The reader's conditional verdict is appropriate, and I agree with the identification of the transfer assumption. I focus on it rather than annotation noise because Section VI-D is the explicit evidence for the strongest claim, and the paper itself flags the configuration gap in Sections V-C and VIII as future work. The paper deserves credit for honest reporting: it gives parameter counts, runtime, label definitions, and explicitly admits statistical limits and the legacy-configuration limitation. A positive result on the proposed check would substantially de-risk the central claim; a negative result would mean the paper's quantitative support applies only to a configuration it does not deploy. The annotation-quality concern is real (no inter-annotator agreement, keyword-mapped stages) but secondary: the label-shape ablation holds annotation fixed, so the relative ordering of label shapes is less sensitive to annotation noise than the configuration transfer. Therefore the verdict remains conditional pending the dual-head ablation; no change from the reader's verdict is needed.","tokens_in":28097,"tokens_out":6709,"duration_ms":67669,"concrete_test":"Re-run the Section V label-shape comparison on the same 1,427-episode corpus under the exact deployed configuration of Table I (six views, 16-frame window, dual heads with per-frame V_bin, beta=1, same balanced sampler and 100-epoch budget), training one run per label shape (MC-smooth, A, B, C-linear, C-late). Report, in addition to each scheme's own-target validation BCE and MAE, label-independent held-out metrics: mean predicted V on success versus failure rollouts, and V_bin detection AUROC and lead on the 24-episode HIL set. If MC-smooth is not the best on the relevant metrics, or if another shape matches it within error while also giving a larger success/failure separation, then the transfer assumption fails and the central claim must be restricted to the legacy configuration.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim points to Section VI-D as the quantitative demonstration that the label, not the architecture, is the hard part. That demonstration is deliberately run under the 'legacy four-view single-head recipe' (Section V-B/C): four views, 32-frame causal window, no per-frame V_bin head, and an episode-level auxiliary loss. The headline system in Table I and the deployment results in Sections VI-E through VI-H use six views, a 16-frame window, a per-frame binary head, and intervention-augmented training. Section V-C asserts that the label-shape ordering is independent of encoder width and window length, but reports no experiment testing that assertion, and explicitly leaves the five-scheme ablation under the dual-head recipe to future work. If the ordering does not transfer—for example, if V_bin supervision changes what the shared backbone learns, or the shorter window changes the relative value of smooth decay versus a cliff—then the evidence for 'label, not architecture' does not cover the configuration in which the paper's headline claims are made. Every downstream use of ValueFormer inherits this gap, so it is the load-bearing assumption in the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper tackles the absence of a per-frame progress signal in behavior-cloned VLA policies, proposing ValueFormer, a compact causal transformer over frozen DINOv3 features with two output heads: a smooth Monte Carlo value V_mc for advantage estimation and a sharp binary head V_bin for online mistake detection. The two labeling contributions are a stage-aware success-then-decay return for failed episodes (Eq. 3) and segment-based mistake-interval supervision (Eq. 4). The central claim, developed in Sections V-C and VI-D, is that the per-frame label shape, not the architecture, is the binding constraint on learning a useful progress value; it is supported by a five-scheme ablation (MC-smooth, outcome-scaled, cliff, alpha-linear, late-diverge) run under the legacy four-view single-head recipe. Deployment results include intervention-derived labels that roughly double held-out detection average precision (Section VI-E), an on-robot A/B in which two critic-derived training weights raise completion from 70% to 85% though explicitly within noise at n=20 (Section VI-G), and a batched bf16 encoder path that cuts per-tick serving cost 3-5x (Section VI-H). The paper is unusually transparent, disclosing the n=20 power limitation, the circularity of the HIL ground truth, and the failure of its own vf-awr prediction.","tokens_in":28370,"tokens_out":17926,"duration_ms":152785,"significance":"If the central claim holds, the paper offers a transferable design pattern for value learning in behavior-cloned systems: dense continuous supervision is only useful when its shape encodes when a failure began and how much partial credit remains, and the dual-head pattern (smooth critic plus sharp detector sharing one backbone) cleanly resolves two conflicting target requirements in one forward pass. The controlled five-scheme ablation is a genuine isolation of the label effect, and the engineering content is strong, with exact hyperparameters, parameter counts, measured bf16 numerics, and an instructive latency analysis. The reporting is exemplary in places: a wrong prediction of the authors' own (vf-awr) is presented as such rather than revised, consistent with the paper's explicit statistical hedging. The significance is bounded by the single-task evidence base, the small evaluation sizes (n=20 A/B, 21 held-out rollouts without intervals), and the fact that the label-shape ordering is not yet demonstrated in the deployed six-view dual-head configuration.","major_comments":[{"comment":"The paper's central claim that the per-frame label, rather than the architecture, is the hard part is established only under the legacy four-view, 32-frame, single-head recipe: Section V-C asserts that the label-shape ordering is 'independent of the encoder width or window length' without reporting an experiment, and explicitly defers the five-scheme ablation under the dual-head recipe to future work. The deployed system (Table I; Sections VI-E through VI-H) uses six views, a 16-frame window, per-frame V_bin supervision, and intervention-augmented training, so the evidence for the central claim does not currently cover the configuration in which the headline results are produced, and every downstream use of ValueFormer inherits this gap. Please re-run the five-scheme ablation under the deployed configuration, or provide a targeted experiment varying window length and view count under the MC-smooth label that demonstrates the ordering of Section VI-D is stable.","section":"V-C and VI-D"},{"comment":"There is a numerical tension between Table II and Table III that bears directly on the 'label, not architecture' claim. Under the legacy configuration, MC-smooth gives mean-V-fail = 0.390 and Delta V = 0.143 on the held-out 21-rollout set, while the main configuration reports mean-V-fail = 0.000 and Delta V = 0.537 on the validation split; the mean-V-success values nearly coincide (0.533 vs 0.537), so the discrepancy is unlikely to be explained by the different evaluation sets. If the deployed architecture with the same Eq. (3) label really drives failure-frame predictions to zero, the architectural and data changes alone move Delta V by nearly a factor of four, the same magnitude as the four-fold validation-loss spread attributed to label shape in Section VI-D, which contradicts the Section VII.1 assertion that no architectural change produced a comparable effect; it also sits uneasily with the positive pre-failure labels of Eq. (3) and the reported validation MSE of 3e-4. Please state exactly how mean-V-fail is computed in each table and reconcile the two numbers.","section":"Tables II and III; Sections VI-A and VI-D"},{"comment":"The detection evaluation in Section VI-E is partly circular: the manual+intervention critic is trained on a shifted version of the same per-frame intervention flag against which it is evaluated on the held-out 24 episodes, so the reported average-precision gain (0.38 to 0.82) and the 35% to 95% detected-before-takeover improvement partly measure agreement with the label source rather than with an independent notion of failure. The paper's caveat in Section VI-E is accurate, but Section VIII(b) restates the result as 'roughly double held-out detection quality' without it, and the contribution claims inherit that framing. The lead-time column is the one label-independent quantity and should carry the claim; an evaluation against hand-labeled mistake onsets on the held-out episodes, which the paper itself names as the next measurement, is needed before the detection claim is taken at face value.","section":"VI-E and VIII(b)"}],"minor_comments":[{"comment":"The rollout-level metrics in Table III rest on 12 success and 9 failure rollouts with no confidence intervals or per-rollout values; please add error bars or a scatter so the reader can assess whether MC-smooth's roughly 2x MAE advantage is robust at this sample size.","section":"VI-D, Table III"},{"comment":"Because each scheme is selected on validation MSE and then evaluated in Table III on BCE against its own target, part of MC-smooth's BCE advantage may reflect that a smooth target is intrinsically easier to fit than a step or scaled target; please discuss or control for this by reporting BCE on a common reference target.","section":"V-D, Table III"},{"comment":"No inter-annotator agreement or verification of the keyword matcher's stage mapping is reported for the manual annotations that drive Eqs. (3) and (4); a small double-annotation reliability check would materially strengthen confidence in the label route that the whole paper rests on.","section":"IV-B"},{"comment":"The on-robot A/B was run one to two weeks after the control with uncontrolled ingredient staging across sessions, and the completion gain is already within noise (Fisher p=0.45); the paired re-evaluation with shared initial conditions and a McNemar test that the authors propose should be reported before the 85% figure is presented as a result rather than a trend.","section":"VI-G"},{"comment":"The claim that no architectural tuning produced a change comparable to the label effect is not supported by any reported architecture-ablation table; please cite the specific experiments or soften the sentence to match the presented evidence.","section":"VII.1"},{"comment":"Figure 7 and its caption describe the single-head recipe while the surrounding text describes the deployed dual-head configuration; please update the figure or the caption so that the configuration used for the reported results is the one illustrated.","section":"IV-D, Figure 7"}],"recommendation":"major_revision","confidential_remarks":"The paper's self-reporting is a genuine strength and should be protected during revision: the vf-awr prediction failure, the n=20 hedging, and the HIL circularity caveat are all stated plainly. My main concern is that the three load-bearing gaps are exactly the items the paper labels as future work, so I would make the dual-head label-shape ablation, the reconciliation of Tables II and III, and an independent detection evaluation conditions of acceptance rather than suggestions. The serving-cost analysis (Section VI-H) and the label-design argument are solid and should survive revision largely unchanged; a paired A/B with shared initial conditions would also convert the deployment section from suggestive to credible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is an honest, unusually self-aware engineering paper about building a per-frame progress signal for behavior-cloned VLA policies. Second, its central claim—label shape matters more than architecture—is plausible and partly demonstrated, but the quantitative evidence for it was collected in a legacy configuration that is not the deployed system, so the claim is not yet as established as the paper's framing suggests.\n\nWhat is actually new: the stage-aware success-then-decay MC label (Eq. 3) preserves the success curve before the failure stage and decays smoothly after; the dual-head design emits a smooth V_mc and a sharp interval-supervised V_bin from one forward pass; and intervention flag segments are used as cheap weak labels for detection. The paper also ships a measured serving-cost optimization (batched bf16 encoder, 3–5×) and an on-robot A/B of critic-derived training weights. The related-work coverage is careful, and the writing is refreshingly candid: the authors record that vf-awr, their least-likely variant, matched vf-mask, and they openly state that the completion gain is within noise. That kind of reporting earns trust.\n\nSoft spots, in proportion. The label-shape ablation in Section VI-D runs under the legacy four-view single-head recipe with a 32-frame window and no per-frame V_bin, on 21 held-out rollouts without confidence intervals. The deployed system uses six views, a 16-frame window, and the dual-head setup. Section V-C asserts the ordering should transfer but leaves the five-scheme ablation in the deployed configuration to future work. That is a real gap because V_bin supervision and the shorter window change what the shared backbone learns; the main evidence for 'label, not architecture' therefore does not cover the configuration in which the paper's headline claims are made.\n\nSecond, the on-robot A/B is underpowered: 20 sandwiches per variant, 70 to 85 percent completion, Fisher p=0.45. The authors say this themselves, and the repeat-pick elimination and agreement of two different critic weights are suggestive, but the policy-improvement result is not established. Third, the HIL detection numbers are partly circular: the model is trained and evaluated against the same intervention flag, so AP/AUROC mostly measure agreement with where the operator intervened rather than an independent failure ground truth. The lead-time column is label-independent, and the authors disclose the circularity, but the headline 0.38 to 0.82 AP should be read with that caveat. Fourth, the manual annotation has no inter-annotator agreement, and no code or data is released, which limits independent checks.\n\nWho this is for: anyone building value critics, failure detectors, or semi-autonomous gating for VLA policies in real deployment. It deserves a serious referee. I would send it to review and ask for the label-shape ablation in the deployed dual-head config, confidence intervals or a paired re-evaluation, and code/data release; none of these are deal-breakers, but they would make the central claim match the evidence.","headline":"An honest, well-engineered paper on per-frame value labels for BC VLAs whose central label-shape claim is plausible but only tested in a legacy configuration, so the headline result is slightly ahead of the evidence.","tokens_in":28865,"tokens_out":3537,"would_cite":true,"duration_ms":30575,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The per-frame label, not the architecture, determines whether a learned progress value can tell a robot when it is failing.","keywords":["value function","vision-language-action policies","behavior cloning","failure detection","stage-aware labels","Monte Carlo returns","causal transformer","bimanual manipulation"],"falsifier":"Run the same five-shape ablation under the deployed six-view dual-head configuration and compare validation BCE and rollout MAE; if MC-smooth no longer wins on both, or if a re-annotation of the 88 failures by independent judges shows stage disagreement above 10%, the central label-shape claim is not supported.","tokens_in":27939,"feed_emoji":"🤖","tokens_out":7827,"duration_ms":66047,"temperature":0.7,"pith_summary":"This paper tries to establish that the hard part of giving a behavior-cloned vision-language-action policy a sense of progress is not the model but the per-frame training label. It proposes a small causal transformer, ValueFormer, that watches the same camera views and joint state as the policy and outputs two signals per frame: a smooth Monte Carlo value for advantage estimation and a sharp binary value for mistake detection. The central claim is that a stage-aware success-then-decay label—failed episodes follow the success curve up to the failure stage and decay smoothly afterward—is the only one among five shapes that avoids leaking the final outcome into frame zero and avoids a discontinuity at the failure frame, and it also gives the best validation loss and rollout error. If correct, any behavior-cloned VLA can be monitored and post-trained with a policy-agnostic critic at low cost, without changing the policy itself.","feed_headline":"A stage-aware label gives robot policies a per-frame failure alert","feed_subtitle":"With stage-aware labels, a small critic runs at 2 Hz and flags mistakes before they cascade.","key_machinery":"The central object is the stage-aware success-then-decay Monte Carlo label of Eq. (3), together with a two-head causal transformer trained on frozen self-supervised ViT-L/16 features (DINOv3) with six views, joint state, and time features. The label's exponential shape is load-bearing in two ways: with $\\gamma=0.99$ the pre-failure prefix matches the success curve, so the network is never asked to regress identical observations onto opposite targets, and the exponential rather than linear form keeps the horizon-differenced advantage $A_t = V_{\\mathrm{mc}}(t+H_A) - V_{\\mathrm{mc}}(t)$ non-constant across a successful rollout, so later frames carry larger advantage. The causal mask (each frame attends only to past and present frames) makes the value valid for online prediction, and the dual head splits the smooth critic target from the sharp detection target so one forward pass serves both.","core_discovery":"The discovery is that the per-frame label shape dominates the error: across five fail-episode label shapes under one fixed architecture, the validation loss spread is roughly 4x, while no architectural change moved results comparably. ValueFormer's stage-aware label sets a failed episode's pre-failure frames to the same rising exponential used for successes, $v^{\\mathrm{succ}}_k = \\gamma^{N-1-k}$, and after the annotated failure stage decays with the same discount, $v^{\\mathrm{fail}}_k = v^{\\mathrm{succ}}_{k_{\\mathrm{fail}}} \\cdot \\gamma^{k-k_{\\mathrm{fail}}}$. This is claimed to be the only shape that simultaneously has no outcome-dependent leakage at $t=0$, no hard cliff at the failure frame, and a smooth post-failure tail that preserves partial credit for completed stages. In addition, a second per-frame binary head supervised by mistake intervals $(t_{\\mathrm{start}}, t_{\\mathrm{end}})$ rather than a single failure time lets transient mistakes the policy recovers from also contribute training signal. On a real-robot sandwich-assembly task, the model reaches validation MSE about $3\\times10^{-4}$ and reproduces four rollout signatures, and critic-derived per-frame training weights raise task completion from 70% to 85%, with the paper noting that at $n=20$ this gain is within noise.","pith_inferences":["The same 'leakage at t=0' failure mode likely afflicts any dense reward or process-reward label whose early-frame target is set by the final outcome; testing for it should be standard when designing per-frame supervision in other sequential decision domains.","The label-shape ordering was established under a legacy four-view single-head configuration, and the paper explicitly leaves the full five-shape ablation under the deployed six-view dual-head setup to future work, so whether the ordering survives that configuration is an open question rather than a demonstrated result.","A foundation-model labeler could plausibly pre-populate mistake intervals with a human confirming them, which would combine the paper's interval supervision with automated labeling and remove the manual-annotation scaling bottleneck.","Because the advantage signal $A_t$ is non-constant only near the terminal, the same critic could serve as a trajectory-ranking filter for offline post-training data, not just a per-frame loss weight."],"forward_implications":["Any behavior-cloned VLA in the same policy family can be given a per-frame progress value and mistake alert without retraining the policy, since ValueFormer is policy-agnostic and consumes the same observation stream.","The stage-aware label recipe should transfer to other multi-stage tasks with ordered stages and sparse success/failure outcomes, because the label shape, not the encoder, carries the result.","Human-in-the-loop intervention flags provide mistake intervals at fleet scale, so the detection head can be improved without manual video annotation.","Critic-derived per-frame weights can be baked into VLA post-training to remove specific failure modes such as repeat-picks, even when overall completion gains are within noise.","The batched bf16 encoder path cuts the live serving cost by 3–5x, making it feasible to run the critic at 2 Hz on the same GPU as the policy."],"supporting_citations":[{"why":"Defines the flow-matching VLA family whose rollouts ValueFormer supervises, supplying the policy context.","marker":"[1]"},{"why":"Closest prior value-function work predicting steps-to-success and conditioning the policy on binarized advantage; the contrast for policy-agnostic, stage-indexed labels.","marker":"[3]"},{"why":"Provides the episode format that carries rollouts, feature caches, and the annotation tables.","marker":"[7]"},{"why":"Provides the frozen self-supervised ViT-L/16 encoder whose CLS tokens are ValueFormer's visual input.","marker":"[8]"},{"why":"Supplies the flow-matching action head of the supervised VLA, fixing the policy-side context.","marker":"[12]"},{"why":"Pre-trains per-frame progress values from video alone, the value-from-video family this paper extends with explicit failure timing.","marker":"[22]"},{"why":"Same family: language-image representations as rewards, one of the baselines for per-frame progress signals.","marker":"[23]"},{"why":"Supplies step-level binary process-reward supervision precedent that justifies interval-based mistake labels.","marker":"[42]"},{"why":"Provides the Monte Carlo value-prediction formulation that Eq. (3) makes stage-aware and success-then-decay.","marker":"[44]"}],"fun_headline_variants":["Label shape beats architecture: 4x error spread","Per-frame failure alerts from stage-aware labels","2 Hz critic flags mistakes before they cascade","Sparse success, dense signals: a better robot critic"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The stage-aware labels rest on hand-annotated failure stages and mistake intervals for 88 failure rollouts and 70 recovered-mistake successes, with no reported inter-annotator agreement; if those annotations are systematically off, the label shape and every downstream result inherit the error.","fun_headline_variants_meta":{"raw":{"variants":["Label shape beats architecture: 4x error spread","Per-frame failure alerts from stage-aware labels","2 Hz critic flags mistakes before they cascade","Sparse success, dense signals: a better robot critic"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000897,"raw_usage":{"total_tokens":3958,"prompt_tokens":1131,"completion_tokens":2827,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":747,"completion_tokens_details":{"reasoning_tokens":2776}},"tokens_in":747,"tokens_out":2827,"duration_ms":20052,"temperature":1.0,"reasoning_tokens":2776,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:55:26.165976+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same five-shape ablation under the deployed six-view dual-head configuration and compare validation BCE and rollout MAE; if MC-smooth no longer wins on both, or if a re-annotation of the 88 failures by independent judges shows stage disagreement above 10%, the central label-shape claim is not supported.","supporting_citations":[{"cited_title":"LeRobot: State-of-the-art machine learning for real-world robotics in PyTorch,","cited_arxiv_id":null,"evidence_quote":"Provides the episode format that carries rollouts, feature caches, and the annotation tables."},{"cited_title":"VIP: Towards universal visual reward and representation via value-implicit pre-training,","cited_arxiv_id":null,"evidence_quote":"Pre-trains per-frame progress values from video alone, the value-from-video family this paper extends with explicit failure timing."},{"cited_title":"LIV: Language-image representations and rewards for robotic control,","cited_arxiv_id":null,"evidence_quote":"Same family: language-image representations as rewards, one of the baselines for per-frame progress signals."},{"cited_title":"Value prediction network,","cited_arxiv_id":null,"evidence_quote":"Provides the Monte Carlo value-prediction formulation that Eq. (3) makes stage-aware and success-then-decay."}],"review_version":2}