{"id":"334f0eda-a10c-4a1a-9320-66da8b67f527","arxiv_id":"2601.02295","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"CycleVLA adds progress-triggered VLM failure checks, subtask backtracking, and MBR consensus decoding to VLAs, raising LIBERO average success from 89.3% to 95.3% and claiming 91% real-robot success.","lead":"CycleVLA wraps robot control models in a check-and-retry loop: a vision-language model decides whether to rewind to an earlier subtask when a task is at risk, then the robot retries using a consensus-selected action. It reports large success-rate gains on LIBERO and claims 91% success on a real robot, but the real-robot claim is contradicted by the paper's own conclusion.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Abstract's real-robot 91%, ~80% perturbation-correction, π0.5, and LIBERO-Plus claims are contradicted by the paper's own conclusion ('Experiments on real robot hardware to be added') and are absent from Section V.","rationale":"The reader's central verdict (REJECT) is supported by their rationale, which highlights the contradiction between the abstract and the Conclusion. However, the reader's weakest_assumption field focuses on reverse-execution reversibility. I regard the unsupported real-robot and π0.5/LIBERO-Plus claims as the single most load-bearing problem because they are part of the abstract's central claim and are contradicted by the manuscript's own limitation statement. The reverse-execution assumption is a genuine and acknowledged limitation, but it only affects the generalizability and robustness of the simulation results; the abstract's headline claims are unsupported and internally inconsistent. Hence my agreement is partial: same verdict, different primary concern. If the paper is revised to remove or substantiate the real-robot and baseline claims, the reversibility issue (Algorithm 1 line 17) and missing error bars in Tables I-VI would become the key scientific checks.","tokens_in":20700,"tokens_out":5389,"duration_ms":55898,"concrete_test":"Audit the manuscript for three things: (1) any section after Section V that reports real-robot rollouts with episode counts and task descriptions; (2) any table or quantitative statement reporting a π0.5 baseline on LIBERO; (3) any table or subsection reporting LIBERO-Plus results. If none exists and the Conclusion still says real-robot experiments are future work, the abstract's 91%, ~80%, π0.5, and LIBERO-Plus claims are unsupported by this version. If the authors claim the results exist in an external artifact, obtain that artifact and verify the reported numbers against the exact evaluation protocol described in Section V.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's advertised central claim includes results that the manuscript does not contain. The abstract states 'On a real robot, CycleVLA reaches a 91% average success rate' and reports stress tests correcting '~80% of injected failures', and it says experiments on 'LIBERO and LIBERO-Plus simulation benchmarks' show that CycleVLA 'surpasses the state-of-the-art VLA π0.5'. Section V, however, reports only LIBERO: Table I has no π0.5 row, no LIBERO-Plus results appear anywhere, and there is no real-robot experiment. The Conclusion explicitly says 'Experiments on real robot hardware to be added in the near future.' This is an internal inconsistency, not just a missing optional baseline. Since the abstract's headline numbers (91%, ~80%) and SOTA comparison are the strongest advertised results, the central claim as stated is unverifiable from the submitted text. A revision that removes these claims or supplies the corresponding sections and baselines could make the simulation contribution evaluable on its own merits.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CycleVLA, a system that wraps a pretrained Vision-Language-Action model (VLA) with three components: (a) finetuning that augments the VLA action space with stop and progress signals learned from LLM-decomposed demonstration subtasks; (b) a VLM-based failure predictor/planner that is triggered when predicted progress is high and may command backtracking to an earlier subtask; and (c) Minimum Bayes Risk (MBR) decoding to select among sampled action chunks after backtracking. Experiments are reported on LIBERO simulation, with average success improving from 89.3% to 95.3% for the 500K checkpoint and consistent gains for earlier checkpoints, together with ablations and a runtime analysis. The abstract additionally claims real-robot 91% success, ~80% recovery from injected perturbations, LIBERO-Plus results, and superiority over the π0.5 VLA; these claims do not appear in Section V and are directly contradicted by the Conclusion's statement that real-robot experiments are to be added in the near future.","tokens_in":20971,"tokens_out":4885,"duration_ms":52945,"significance":"If the simulation results are taken at face value, the core idea is practically valuable and timely: a generic VLA can be augmented with progress-triggered VLM failure prediction, subtask backtracking, and MBR consensus selection to improve long-horizon success without training a replacement policy. The paper is also commendably detailed in its appendices, providing exact prompts, hyperparameters, and a human evaluation of the subtask-decomposition pipeline. However, the central empirical contribution as advertised—real-robot 91% success, ~80% stress-test recovery, LIBERO-Plus evaluation, and superiority over π0.5—is not verifiable from the submitted text. The verifiable contribution is a LIBERO-only simulation study, and even that rests on an unexamined reversibility assumption for the backtracking mechanism.","major_comments":[{"comment":"The abstract advertises (i) a 91% average success rate on a real robot, (ii) correction of ~80% of injected failures in stress tests, (iii) evaluation on LIBERO and LIBERO-Plus simulation benchmarks, and (iv) superiority over the state-of-the-art VLA π0.5. None of these are present in Section V: Tables I–VI report only LIBERO; Table I contains no π0.5 row; no stress-test protocol or perturbation results appear anywhere; and the Conclusion explicitly states 'Experiments on real robot hardware to be added in the near future.' The headline empirical claims are therefore unsupported and internally inconsistent with the manuscript's own text, and the central claim as stated cannot be evaluated.","section":"Abstract vs. Section V and Conclusion"},{"comment":"The backtracking mechanism restores the robot to the start of subtask g_j by 'reverse-executing recorded delta actions.' This assumes reversible state transitions. In contact-rich manipulation, objects may be displaced, grasps may drop objects, or the robot state may drift during the failed attempt, so the retry can begin from a state different from the recorded start. The Conclusion acknowledges this limitation ('assumes reversible state transitions, which may not hold in highly dynamic or irreversible environments'). Because every reported recovery gain depends on this assumption, the paper needs evidence of state-restoration fidelity (e.g., measured state deviation after reverse execution, or success rates conditioned on the actual post-backtracking state) before the gains can be attributed to the proposed correction mechanism.","section":"Section IV-B, Algorithm 1 line 17, and Conclusion"},{"comment":"The MBR evaluation metric P_succ estimates success probability by averaging per-chunk success labels z_e,t 'inherited from the recorded rollout.' But if MBR selects a different hypothesis at an early decision step, the subsequent observations, action outcomes, and success labels would not be those recorded in that rollout. Thus Eq. (5) is not a valid closed-loop success estimate; it is an open-loop selection metric that can be optimistic. The end-to-end gains in Table II combine MBR with backtracking, so the isolated MBR contribution in Table III needs a closed-loop evaluation (execute the selected chunk and continue) or an explicit label as an oracle/upper-bound.","section":"Section V-C, Eq. (5)"}],"minor_comments":[{"comment":"Typo: 'computationn' should be 'computation.'","section":"Section V-E"},{"comment":"The abbreviation LAO is used in the table caption but not spelled out in the main text; please define at first use.","section":"Table VI caption"},{"comment":"The term 'proactive self-correction' is cited to a reinforcement-learning intervention paper; the term is not standard in that reference. Consider citing a more directly relevant work or defining the term operationally.","section":"Introduction, reference [1]"}],"recommendation":"reject","confidential_remarks":"The simulation study may be salvageable, but the abstract's real-robot 91%, ~80% stress-test recovery, LIBERO-Plus, and π0.5 superiority claims are absent from the experiments and contradicted by the Conclusion. This is not a matter of polishing: the advertised central claim is unverifiable from the submitted manuscript. I would be willing to reconsider a resubmission in which the abstract is strictly aligned with the experiments actually reported, the reversibility assumption is tested or explicitly scoped, and the MBR evaluation is closed-loop or clearly labeled as an oracle."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the simulation core of CycleVLA is coherent and the within-paper comparisons consistently support the mechanism — progress-triggered VLM checks, subtask backtracking, and MBR consensus decoding for VLA retries. The gains on LIBERO are meaningful, especially for under-trained checkpoints, and the MBR scaling analysis with N and distance-metric sweeps is a nice piece of work. But the abstract overstates the evidence in a way that cannot be ignored: the 91% real-robot success and ~80% perturbation-correction rates appear nowhere in Section V, and the Conclusion explicitly says real-robot experiments are 'to be added in the near future.' The same goes for the claim of surpassing π0.5 — no π0.5 baseline appears in Table I — and for 'LIBERO-Plus,' which is mentioned in the abstract but never defined or reported. These are headline claims, not optional extras, so the paper as submitted is not acceptable.\n\nOn the positive side, the idea of making the VLA predict stop and progress signals as extra action dimensions is a clean adaptation, and the VLM prompt design is detailed. The ablation in Table VI (removing MBR, swapping the VLM, dropping the stop signal) gives a fair sense of each component's contribution. The runtime analysis is honest about the ~30% overhead.\n\nSoft spots beyond the abstract: the 'progress' signal p_t is just a normalized timestep within the subtask, not a perceptual estimate of completion, which is a weaker interpretation of 'progress-aware' than the paper sometimes implies. The reverse-execution assumption for backtracking is acknowledged in the limitations, but it is load-bearing — every reported gain depends on the world being reversible enough to replay retries from the same state.\n\nBottom line: the system is worth a careful look after a revision. Remove or substantiate the real-robot and LIBERO-Plus claims, add the π0.5 baseline, report error bars, and clarify the time-based nature of the progress signal. I'd send it to peer review with that expectation, but I wouldn't cite the abstract's headline numbers in my own writing until they exist.","headline":"Solid simulation wrapper with a genuine MBR-for-VLA idea, but the abstract claims real-robot results the paper itself says are not yet done.","tokens_in":21532,"tokens_out":3483,"would_cite":false,"duration_ms":35257,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CycleVLA claims that wrapping a vision-language-action model with progress-aware failure prediction, subtask backtracking, and consensus retry lifts success rates on long-horizon manipulation tasks.","keywords":["vision-language-action models","proactive self-correction","failure prediction","subtask backtracking","minimum Bayes risk decoding","test-time scaling","robot manipulation","long-horizon tasks"],"falsifier":"Run CycleVLA on a task where an object is knocked over or a grasped item is dropped and displaced during the failed attempt, and record whether success after backtracking drops to the no-correction baseline; if it does, reverse execution is not actually restoring preconditions.","tokens_in":20537,"feed_emoji":"🤖","tokens_out":4267,"duration_ms":42761,"temperature":0.7,"pith_summary":"The paper argues that robot failures can be caught before they fully manifest by teaching a vision-language-action model to estimate its own subtask progress, having a vision-language model judge whether the current subtask will fail, and if so rewinding to an earlier subtask and retrying with a consensus-selected action. It claims this wrapping improves success rates for both fully trained and under-trained policies, most strongly on long-horizon tasks, and that Minimum Bayes Risk decoding is an effective zero-shot test-time scaling strategy. A reader should care because it offers a way to make generalist robot policies more reliable without retraining the underlying policy.","feed_headline":"Proactive self-correction lifts robot policy success to 95%","feed_subtitle":"CycleVLA predicts failures at subtask transitions, rewinds, and retries via consensus decoding.","key_machinery":"The central mechanism is a progress-aware VLA: the action space is extended from 7 to 9 dimensions, adding a per-step stop signal and a discretized progress value, trained on a subtask-decomposed dataset. At inference, when predicted progress reaches a threshold (τ_p=0.9), an off-the-shelf VLM is queried to decide whether to transit or backtrack, with backtracking executed by reverse-executing recorded delta actions. After backtracking, MBR decoding samples N=8 stochastic action chunks and selects the medoid of the densest cluster under L2 trajectory distance, a training-free consensus selection that favors high-density regions of the policy output space.","core_discovery":"The central claim is that equipping a VLA with explicit subtask progress and stop signals, then querying a VLM at ~90% progress to predict failure and, if predicted, backtracking to the earliest subtask that restores missing preconditions and retrying with MBR-decoded consensus actions, yields large success-rate improvements. The paper shows average success on the LIBERO simulation benchmark rises from 89.3% without correction to 95.3% with correction, with gains across under-trained checkpoints, and reports a 91% average success rate on a real robot on precise and long-horizon manipulation tasks while correcting about 80% of injected failures.","pith_inferences":["The reversibility assumption means the approach is limited to near-static scenes; in contact-rich or irreversible settings, reverse-executing recorded actions will not restore preconditions, so the reported gains should shrink or vanish.","The paper's ablation shows a ~10% drop when terminating on predicted failure, consistent with VLM sycophancy; a calibrated failure predictor would likely recover some of that gap.","The density-based MBR selection suggests consensus in action space correlates with success; extending this to visual state consensus or trajectory-level features could improve robustness.","One could test whether MBR's benefit transfers to other action representations (e.g., waypoint deltas) and whether using an oracle success label confirms that consensus corresponds to ground-truth success."],"forward_implications":["If correct, any stochastic VLA can be wrapped with progress prediction, VLM-gated backtracking, and MBR retry to gain several points of success without retraining the policy.","Test-time scaling via MBR becomes a zero-shot alternative to learned verifiers or reward models for VLA action selection.","Under-trained policies benefit more, suggesting that inference-time consensus can partially compensate for training deficiency.","Long-horizon tasks, where errors accumulate across subtasks, stand to gain the most from this self-correction loop."],"fun_headline_variants":["Predict, backtrack, retry: CycleVLA lifts robot success to 95%","CycleVLA: proactive robot self-correction boosts success to 95%","Anticipating failures: CycleVLA rewinds and retries for 95% success","Subtask backtracking and consensus decoding push robot success to 95%","Robot learns to foresee and fix errors: 95% success via CycleVLA"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that backtracking can restore the robot to the exact start of a subtask by reversing previously executed actions; if the world does not reverse cleanly (contacts, friction, object shifts), the retried state differs from the recorded one and the VLM's plan may no longer apply.","fun_headline_variants_meta":{"raw":{"variants":["Predict, backtrack, retry: CycleVLA lifts robot success to 95%","CycleVLA: proactive robot self-correction boosts success to 95%","Anticipating failures: CycleVLA rewinds and retries for 95% success","Subtask backtracking and consensus decoding push robot success to 95%","Robot learns to foresee and fix errors: 95% success via CycleVLA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000872,"raw_usage":{"total_tokens":3649,"prompt_tokens":822,"completion_tokens":2827,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":2721}},"tokens_in":566,"tokens_out":2827,"duration_ms":19852,"temperature":1.0,"reasoning_tokens":2721,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T12:33:12.467823+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CycleVLA on a task where an object is knocked over or a grasped item is dropped and displaced during the failed attempt, and record whether success after backtracking drops to the no-correction baseline; if it does, reverse execution is not actually restoring preconditions.","supporting_citations":[],"review_version":1}