REVIEW 3 major objections 6 minor 35 references
Continue or Replan? Bernoulli-Continuation Policy Learning for Adaptive Horizon Execution
T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read The paper claims that replanning-timing misalignment, not action quality alone, causes many chunk-based VLA failures, and a lightweight continuation policy trained by reinforcement learning can place replanning boundaries before critical ma
desk verdict Fresh adaptive-horizon idea with a load-bearing reporting discrepancy between the main text and its own appendix. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Bernoulli-Continuation Head. Instead of classifying among candidate horizons with a softmax, BCP emits continuation probabilities p_i; horizon e_k is selected only if the policy continues through k−1 decisions and then stops, so P(E=e_k) = (∏_{i<k} p_i)(1−p_k), with the longest horizon requiring all continues. This factorization gives candidate horizons an ordinal, prefix-sharing structure: a 40-step execution and a 50-step execution share the same first 40 actions, and adjacent horizons get similar probability. The other mechanism is the Replanning-Efficiency Reward, which makes success dominant, rewards fewer VLA calls, and only mildly penalizes extra replann
What would settle it
Force the continuation head to make its horizon choice from a randomly shuffled version of the VLA's internal features, or from the timestep index alone, and retrain on the same reward. If success remains close to BCP's reported levels, then the specific feature-based placement of replanning boundaries is not what drives the gains. Conversely, if a fixed-horizon schedule using the best oracle phase already matches BCP's success, the misalignment mechanism would be falsified.
Extended reading notes
Core claim
The central claim is that for chunk-based VLAs, success depends on whether a replanning boundary lands before each precision-critical manipulation stage, not on the horizon length alone, and that this boundary placement can be learned. The paper supports this with a phase-shift experiment in which all 50 RoboTwin 2.0 tasks are run under a fixed 50-step horizon from every initial phase; the best phase is spread uniformly across possibilities and the gap between best and worst phase averages 11.30 percentage points. BCP turns horizon selection into a chain of Bernoulli continuation decisions over candidate horizons, reusing the VLA's own visual-language and action-velocity features so no extra
Load-bearing premise
The continuation head must be able to tell, from the information the frozen model already produces at prediction time, when the current action plan is about to go stale; if those clues carry no task-progress signal, the learned stopping decisions cannot place replanning boundaries before critical stages.
Editorial extensions
If this is right
- A frozen VLA can be made more reliable by a small learned head, so adaptive execution does not require retraining or fine-tuning the expensive action model.
- Replanning before critical stages raises success more on tasks where the base policy is weakest, suggesting timing misalignment is a controllable failure mode rather than an unavoidable action-generation error.
- The learned policy transfers across visual randomization and across different base policies, indicating it captures task-stage structure rather than memorizing training conditions.
- Because BCP executes fewer control steps on average while issuing slightly more VLA calls, its end-to-end runtime stays at or below the fixed-horizon baseline.
- The same continue-or-replan design is directly applicable to other chunk-generating models, including the world action models named as future work in the paper.
Reading between the lines
- Editorial: BCP's gains likely come from learning a task-stage classifier inside the VLA's representation space; probing continue probabilities against hand-labeled stages such as approach, grasp, and place would test this directly.
- Editorial: Combining BCP with uncertainty-based or entropy-based action-quality signals could give further gains when the VLA's own features are less informative than a separate confidence estimator.
- Editorial: The same prefix-sharing factorization could be applied to variable chunk lengths or to hierarchical task plans, where the continue decision is about whether the current macro-action remains valid.
- Editorial: A useful stress test is to measure BCP against an oracle that replans exactly at hand-labeled critical stages; the gap would quantify how much headroom remains in learned horizon selection.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Bernoulli-Continuation Policy (BCP), a lightweight plug-and-play module that adaptively selects how many actions from a frozen VLA's predicted chunk should be executed before replanning. BCP decomposes horizon selection into a sequence of continue-or-replan Bernoulli decisions, preserving the ordinal, prefix-sharing structure of candidate horizons, and is trained with GRPO using a Replanning-Efficiency Reward that balances task success against VLA query cost. Experiments on RoboTwin 2.0 with LingBot-VLA, ABot-M0, and ACT, on LIBERO/LIBERO-PRO with π0.5, and on a real AGIBOT G1 robot report success-rate improvements over fixed-horizon baselines with negligible runtime overhead. The central technical idea is clear and the evaluation is broad, but several reporting inconsistencies and missing statistical details affect the strength of the empirical claims.
Significance. If the reported gains hold, BCP is a valuable contribution: it addresses a real failure mode of chunk-based VLAs, keeps the base policy frozen, imposes a principled inductive bias over horizons, and requires only a small trainable head. The phase-shift experiment provides a concrete falsifiable demonstration that replanning timing matters, and the inclusion of multiple base policies, two simulation benchmarks, ablations, and real-robot results is commendable. The main weakness is not the method's design but the reliability of the headline numbers: the stated inclusion criterion for the '13 low-success tasks' is contradicted by the paper's own appendix table, and no uncertainty quantification is provided for any of the central success-rate comparisons. These issues are correctable within the manuscript's scope.
major comments (3)
- [§4.1 / Table 2 / Appendix Table A.5] The paper states that it focuses on '13 low-success tasks where the original LingBot-VLA achieves a success rate below 90% under the Clean setting.' However, Appendix Table A.5 lists 16 tasks with Clean success below 90%: the three omitted rows are Place Bread Basket (89→98), Stamp Seal (89→96), and Stack Bowls Three (89→87). Recomputing the low-success average over all 16 tasks gives a baseline of 77.56% and a BCP success of 87.63%, i.e., a +10.06 pp gain, not the advertised +11.08 pp. Moreover, one of the omitted tasks (Stack Bowls Three) actually regresses. The headline claim in the abstract and §4.1 therefore overstates the gain and does not match the stated selection rule. Please correct the subset, justify the selection criterion, or report the full 16-task average alongside the 13-task subset.
- [§4 (Tables 2, 3, 5; real-robot results)] All success-rate comparisons are reported as single point estimates without confidence intervals, per-seed variation, or justification of the number of evaluation episodes beyond the environment count (e.g., 100 eval environments for RoboTwin 2.0). For 100 episodes, a 4.06 percentage-point gain is within a few binomial standard errors of zero, so the central claim needs uncertainty quantification. The identical +4.06% gain under both Clean (89.88→93.94) and Randomized (88.78→92.84) settings is particularly fragile without variance estimates or per-task results: it could arise from rounding, but the reader cannot assess it. Please report CIs or seed-level replicates, and explain the exact match in the rounded gains.
- [§4.1 / Table 2] The selection of low-success tasks is additionally entangled with an inconsistency between Table 2 and Table A.5: for Click Alarm Clock, Table A.5 reports LingBot-VLA Clean success of 80%, while Table 2 reports 83%. Since these tables are both described as success rates on RoboTwin 2.0 tasks under the Clean setting, the discrepancy must be reconciled, especially because the 13-task average in Table 2 (75.15%) appears to use the 83% value.
minor comments (6)
- [§3.3 / §4.1] The method section names the reward 'Replanning-Efficiency Reward', but the experiments section refers to 'Reference-Relative Reward' when setting δ+ = 0.7 and δ− = 0.3. Please use consistent terminology.
- [Table A.4] The text states that 'performance is insensitive to moderate variations in [δ+ and δ−]' but no supporting ablation is shown. Either provide the data or soften the claim.
- [§4.3 / Table 4] The runtime comparison is useful, but the reported VLA inference time (938.10 ms vs. 940.13 ms) and executed steps are averaged over tasks without error bars. A brief note on variance across tasks would strengthen the efficiency claim.
- [§4.5 / Real-robot experiments] The description 'tested on the same set of 50 seeds for each task with three trials' is ambiguous. Please clarify whether the 50 seeds are distinct initial configurations or random seeds, and report the total number of evaluation episodes and per-seed success variation.
- [Table A.2] In the simulation settings, the candidate horizons for RoboTwin are listed as {15, 20, 25, ..., 50}, while the real-robot setting uses {30, 35, 40, 45, 50}. The main text says 'the candidate execution horizons are set to (15, 20, 25,..., 50)' without mentioning the real-robot difference; please state this explicitly.
- [§4.2 / Table 3] The text says BCP improves the average LIBERO-PRO success rate 'from 30.9% to 37.7%, corresponding to a 2.9% gain over AAC.' The 2.9% is relative to AAC (37.7 − 34.8); please make this comparison explicit to avoid confusion.
Circularity Check
No significant circularity: BCP gains are empirical RL outcomes; the 13-task subset mismatch is a reporting issue, not a circular step.
full rationale
BCP's central derivation is a learned policy: the continuation head (Eq. 4) is a reparameterization of discrete horizon choice, and the GRPO objective (Eq. 5) maximizes a trajectory-level reward (Eq. 8) defined over task success and relative VLA-call efficiency. Success on RoboTwin 2.0, LIBERO, and the real robot is an external outcome measured on evaluation episodes; no equation in the paper reduces the reported improvement to an identity or to the reward's own definition. The Bernoulli factorization, while motivated as an ordinal inductive bias, is a general representation of any categorical distribution over ordered horizons, so it does not smuggle in the result. The only self-citations are to the frozen base VLA (LingBot-VLA, ABot-M0) and to RLinf; these are used as tools/baselines and are not invoked as a uniqueness theorem or as proof of the method's effectiveness. The Appendix limitation correctly states BCP cannot fix fundamentally wrong action chunks, which is a scope condition, not circularity. Separately, the 13-task 'low-success' subset in Table 2 conflicts with Table A.5, which lists 16 tasks below 90%; this affects the transparency of the headline +11.08% figure, but it is a reporting inconsistency, not a circular derivation. Accordingly, no circular step is identified.
Assumptions & free parameters
free parameters (2)
- Replanning-Efficiency Reward coefficients δ+ and δ- =
δ+ = 0.7, δ- = 0.3
- Candidate execution horizon set =
{15,20,25,...,50} on RoboTwin; {30,35,40,45,50} on real robot; {2,4,5,6,8,10} on LIBERO
assumptions (3)
- domain assumption The base VLA's predicted action chunks contain enough useful actions that replanning timing, rather than action quality, is the dominant failure cause.
- domain assumption The frozen VLA's internal visual-language tokens and per-step action/velocity features carry sufficient information to predict when a chunk will become stale before the next observation.
- standard math GRPO with a trajectory-level reward broadcast to all horizon decisions is a valid optimization for this credit-assignment problem.
Cite this review
Pith. "Pith review of Continue or Replan? Bernoulli-Continuation Policy Learning for Adaptive Horizon Execution." pith.science (2026). https://pith.science/paper/F4AIX7C2
@misc{pith2026260803483,
author = {Pith},
title = {Pith review of: Continue or Replan? Bernoulli-Continuation Policy Learning for Adaptive Horizon Execution},
year = {2026},
howpublished = {\url{https://pith.science/paper/F4AIX7C2}},
note = {Machine review of arXiv:2608.03483}
}
abstract
Existing chunk-based Vision-Language-Action (VLA) models execute a fixed number of actions (i.e., execution horizon) before replanning, turning replanning into a task-agnostic periodic schedule that is independent of task progress. As a result, when no replanning boundary falls before a critical manipulation stage, it is executed from a stale chunk rather than a freshly replanned one. To address this limitation, we propose Bernoulli-Continuation Policy (BCP), a lightweight, plug-and-play framework for adaptive horizon execution that keeps the base VLA frozen. Given a fixed-length action chunk, its continuation head decomposes execution-horizon selection into a sequence of continue-or-replan decisions, which imposes an ordinal, prefix-sharing inductive bias over candidate horizons rather than treating them as independent classes. Since the optimal horizon for each chunk is not observable, we train this head with reinforcement learning from trajectory-level outcomes and introduce a Replanning-Efficiency Reward that jointly rewards task success and efficient VLA usage, discouraging the policy from collapsing to unnecessarily short horizons. On RoboTwin 2.0 with LingBot-VLA as the base policy, BCP improves the average success rate by +11.08% on 13 low-success tasks and from 89.88% to 93.94% (+4.06%) across all 50 tasks. Although trained only under the Clean setting, BCP generalizes to the Randomized setting, raising the average success rate by +4.06%. It also transfers to a different base policy $\pi_{0.5}$, achieving a better result on LIBERO (+1.7%) and, notably, on the harder LIBERO-PRO (+6.8%). On a real robot, BCP lifts success from 74% to 92% and from 44% to 84% on two manipulation tasks. Meanwhile, its negligible overhead, combined with higher success, makes BCP's overall runtime even lower than the fixed-horizon baselines.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
arXiv preprint arXiv:2509.09674 , year=
Simplevla-rl: Scaling vla training via reinforcement learning , author=. arXiv preprint arXiv:2509.09674 , year=
-
[2]
arXiv preprint arXiv:2510.25889 , year=
RL: Online rl fine-tuning for flow-based vision-language-action models , author=. arXiv preprint arXiv:2510.25889 , year=
-
[3]
Intelligence, Physical and Amin, Ali and Aniceto, Raichelle and Balakrishna, Ashwin and Black, Kevin and Conley, Ken and Connors, Grace and Darpinian, James and Dhabalia, Karan and DiCarlo, Jared and others , journal=. ^*_
-
[4]
Advances in Neural Information Processing Systems , volume=
What can rl bring to vla generalization? an empirical study , author=. Advances in Neural Information Processing Systems , volume=
-
[5]
Advances in Neural Information Processing Systems , volume=
Reinforcement learning with action chunking , author=. Advances in Neural Information Processing Systems , volume=
-
[6]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
Actor-critic for continuous action chunks: A reinforcement learning framework for long-horizon robotic manipulation with sparse reward , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
-
[7]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Adaptive action chunking at inference-time for vision-language-action models , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[8]
arXiv preprint arXiv:2602.21445 , year=
Vla knows its limits , author=. arXiv preprint arXiv:2602.21445 , year=
Show all 35 references
-
[9]
Zhu, Weiye and Zhang, Zekai and Wang, Xiangchen and Pan, Hewei and Wang, Teng and Geng, Tiantian and Xu, Rongtao and Zheng, Feng , journal=
-
[10]
arXiv preprint arXiv:2511.19433 , year=
Mixture of Horizons in Action Chunking , author=. arXiv preprint arXiv:2511.19433 , year=
-
[11]
arXiv preprint arXiv:2604.02965 , year=
Open-Loop Planning, Closed-Loop Verification: Speculative Verification for VLA , author=. arXiv preprint arXiv:2604.02965 , year=
-
[12]
arXiv preprint arXiv:2605.06222 , year=
When to Trust Imagination: Adaptive Action Execution for World Action Models , author=. arXiv preprint arXiv:2605.06222 , year=
-
[13]
arXiv preprint arXiv:2304.13705 , year=
Learning fine-grained bimanual manipulation with low-cost hardware , author=. arXiv preprint arXiv:2304.13705 , year=
-
[14]
The International Journal of Robotics Research , volume=
Diffusion policy: Visuomotor policy learning via action diffusion , author=. The International Journal of Robotics Research , volume=. 2025 , publisher=
2025
-
[15]
arXiv preprint arXiv:2406.09246 , year=
Openvla: An open-source vision-language-action model , author=. arXiv preprint arXiv:2406.09246 , year=
-
[16]
arXiv preprint arXiv:2410.24164 , year=
_0 : A Vision-Language-Action Flow Model for General Robot Control , author=. arXiv preprint arXiv:2410.24164 , year=
-
[17]
arXiv preprint arXiv:2411.19650 , year=
Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation , author=. arXiv preprint arXiv:2411.19650 , year=
-
[18]
arXiv preprint arXiv:2502.19645 , year=
Fine-tuning vision-language-action models: Optimizing speed and success , author=. arXiv preprint arXiv:2502.19645 , year=
-
[19]
arXiv preprint arXiv:2605.30280 , year=
Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments , author=. arXiv preprint arXiv:2605.30280 , year=
-
[20]
arXiv preprint arXiv:2601.18692 , year=
A Pragmatic VLA Foundation Model , author=. arXiv preprint arXiv:2601.18692 , year=
-
[21]
arXiv preprint arXiv:2510.10274 , year=
X-vla: Soft-prompted transformer as scalable cross-embodiment vision-language-action model , author=. arXiv preprint arXiv:2510.10274 , year=
-
[22]
arXiv preprint arXiv:2503.14734 , year=
Gr00t n1: An open foundation model for generalist humanoid robots , author=. arXiv preprint arXiv:2503.14734 , year=
-
[23]
arXiv preprint arXiv:2506.18088 , year=
Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation , author=. arXiv preprint arXiv:2506.18088 , year=
-
[24]
Advances in Neural Information Processing Systems , volume=
Dapo: An open-source llm reinforcement learning system at scale , author=. Advances in Neural Information Processing Systems , volume=
-
[25]
arXiv preprint arXiv:2402.03300 , year=
Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=
-
[26]
Advances in Neural Information Processing Systems , volume=
Libero: Benchmarking knowledge transfer for lifelong robot learning , author=. Advances in Neural Information Processing Systems , volume=
-
[27]
arXiv preprint arXiv:2510.03827 , year=
LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization , author=. arXiv preprint arXiv:2510.03827 , year=
-
[28]
arXiv preprint arXiv:2602.11236 , year=
Abot-m0: Vla foundation model for robotic manipulation with action manifold learning , author=. arXiv preprint arXiv:2602.11236 , year=
-
[29]
arXiv preprint arXiv:2607.01804 , year=
VLA-Corrector: Lightweight Detect-and-Correct Inference for Adaptive Action Horizon , author=. arXiv preprint arXiv:2607.01804 , year=
-
[30]
20th USENIX Symposium on Operating Systems Design and Implementation (OSDI 26) , pages=
RLinf: Flexible and Efficient Large-Scale Reinforcement Learning via Macro-to-Micro Flow Transformation , author=. 20th USENIX Symposium on Operating Systems Design and Implementation (OSDI 26) , pages=
-
[31]
Intelligence, Physical and Black, Kevin and Brown, Noah and Darpinian, James and Dhabalia, Karan and Driess, Danny and Esmail, Adnan and Equi, Michael and Finn, Chelsea and Fusai, Niccolo and others , journal=. _
-
[32]
arXiv preprint arXiv:2601.21998 , year=
Causal World Modeling for Robot Control , author=. arXiv preprint arXiv:2601.21998 , year=
-
[33]
arXiv preprint arXiv:2512.13030 , year=
Motus: A unified latent action world model , author=. arXiv preprint arXiv:2512.13030 , year=
-
[34]
arXiv preprint arXiv:2601.16163 , year=
Cosmos policy: Fine-tuning video models for visuomotor control and planning , author=. arXiv preprint arXiv:2601.16163 , year=
-
[35]
arXiv preprint arXiv:2602.15922 , year=
World action models are zero-shot policies , author=. arXiv preprint arXiv:2602.15922 , year=
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.