Pith. sign in

REVIEW 4 major objections 55 references

Frozen robot VLAs already contain good actions; a distilled search evaluator can pick them out without fine-tuning the backbone.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 00:11 UTC pith:5OJXHCH5

load-bearing objection Clean diagnostic + practical frozen-VLA reranker; the 9B-vs-27B cost claim is real within suite, but "unseen tasks" is episode hold-out, not cross-task evaluator transfer. the 4 major comments →

arxiv 2607.03751 v1 pith:5OJXHCH5 submitted 2026-07-04 cs.RO

Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models

Xinyi Xie , Zican Hu , Zhanyu Liu , Yicheng Dong , Wenhao Wu , Zhenhong Sun , Haoran Li , Chunlin Chen
show 2 more authors
Zhi Wang Pichao Wang
This is my paper
classification cs.RO
keywords Vision-Language-Action modelstest-time scalingMonte-Carlo tree searchaction evaluationQ-value modelfrozen policiesembodied agents
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Vision-language-action models often fail on modestly new tasks not because they cannot generate competent moves, but because they cannot tell a good move from a bad one before acting. A pass@k diagnostic shows success climbing from 33 percent with one try to 92 percent with 32 independent tries, so high-quality behaviors already sit inside the frozen policy’s output distribution. The paper’s answer is SVA: run Monte-Carlo tree search in simulation to mine long-horizon returns, distill those returns into a small Q-value model, then at deployment let the frozen VLA propose several candidates and let the evaluator choose the one with the best uncertainty-regularized score—no simulator needed at runtime. Because the multi-billion-parameter backbone stays untouched, its generalist knowledge is preserved while task success rises. On embodied reasoning and manipulation benchmarks the same recipe lifts multiple backbones, and a 9B model with SVA beats a 27B model by seven points at lower latency, arguing that test-time evaluation can be cheaper than scaling model size.

Core claim

VLA failures are an evaluation bottleneck as much as a generation bottleneck: competent actions already exist in the frozen policy’s distribution, and a lightweight Q-model distilled from Monte-Carlo tree search can rank those actions by long-term consequence well enough to raise success rates without any backbone update.

What carries the argument

SVA (Search, Value, and Act): MCTS explores the frozen VLA’s action distribution in simulation and labels edges with empirical returns; those labels train a small ensemble Q-model; at test time the frozen VLA proposes N candidates and the Q-model selects the highest uncertainty-regularized score.

Load-bearing premise

That success labels gathered by tree search only on the training split of a resettable simulator, once distilled into a small Q-model, remain accurate enough to rank actions on held-out episodes and tasks without further online search or real-robot recalibration.

What would settle it

Train the Q-model on the paper’s MCTS procedure for a fixed VLA and benchmark, then measure whether best-of-N selection with that Q-model still beats the same frozen VLA’s single-shot success rate (and the 27B single-shot baseline) on a held-out physical-robot counterpart of the same tasks; collapse of the ranking advantage would falsify the transfer claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Frozen multi-billion-parameter VLAs can be improved at deployment by adding only a small external evaluator, avoiding costly and generality-narrowing backbone fine-tuning.
  • Test-time compute can be spent on more candidate proposals and Q-scoring rather than larger models; the paper’s 9B+SVA result already beats 27B single-shot at lower latency.
  • Pass@k diagnostics become a practical probe: if success rises sharply with k, an evaluation-centric fix is more promising than rewriting the generator.
  • The same Search–Value–Act recipe applies across discrete high-level skills and continuous action chunks once actions are tokenized for the Q-model.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the evaluation bottleneck is as general as claimed, other imitation-trained robot policies (not only VLAs) may also gain more from a distilled long-horizon verifier than from further supervised fine-tuning.
  • Closing the online search–learning loop the authors leave for future work could turn SVA into a self-improving cycle that still never updates the generalist backbone.
  • Real-robot deployment may need only light residual calibration of the Q-heads, because the proposal distribution already comes from real-data-pretrained VLAs and the evaluator never relies on privileged simulator features at inference.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 0 minor

Summary. The paper argues that frozen VLA failures are largely an action-evaluation bottleneck rather than pure generation failure, supported by a pass@k diagnostic (overall success rising from ~33% at pass@1 to ~92% at pass@32). It proposes SVA: MCTS in simulation mines return-annotated trajectories from the frozen policy’s distribution; those returns are distilled into a lightweight ensemble Q-model (small VLM + LoRA + MLP heads); at deployment the frozen policy proposes N candidates and the Q-model selects via uncertainty-regularized scores, without simulator access. Experiments on EmbodiedBench (multiple LM/VLM backbones), SimplerEnv (π0), and RoboTwin 2.0 (π0.5) report consistent gains over base policies and over a RoboMonkey-style preference reranker, plus a test-time scaling plot in which Qwen3.5-9B+SVA (best-of-8) exceeds single-shot Qwen3.5-27B by ~7 points at lower latency. Ablations remove MCTS, the Q-model, or multi-candidate selection.

Significance. If the results hold under tighter evaluation, the work is a useful contribution to test-time scaling for embodied policies: it cleanly separates proposal from consequence evaluation, keeps multi-billion-parameter backbones frozen, and shows that a cheap verifier can convert latent pass@k competence into higher single-trajectory success. Strengths include a coherent diagnostic, a simple Search–Value–Act recipe, multi-backbone and multi-benchmark evidence, direct comparison to RoboMonkey under matched candidate budgets, ablations of each stage, and an explicit latency-vs-N analysis. The framing that scaling evaluation can be more cost-effective than scaling model size is timely for robotics, where fine-tuning large VLAs is expensive and generality is fragile. The limitations appendix is appropriately candid about staged search/value learning, simulator dependence, and missing real-robot validation.

major comments (4)
  1. Abstract and §5.1 claim that SVA “consistently improves generalization on unseen tasks,” but Appendix D.2–D.4 show only episode/seed hold-outs within the same task suites (EB episodes 1–30/1–36 train vs 31–50/37–60 eval; SimplerEnv 15 train seeds / 9 held-out; RoboTwin 30/20 seeds), with separate Q-models trained per benchmark (D.3). That is within-suite reranking, not transfer to a new task distribution or cross-benchmark transfer of the evaluator. The headline “preserve generality + cheaper than scaling model size” depends on the Q-model learning transferable long-horizon consequences rather than suite-specific return structure. Either add a cross-task/cross-benchmark Q-transfer experiment (train Q on one suite, evaluate on another with the same frozen proposer) or substantially revise the abstract/§5.1 language to match the actual split protocol.
  2. Tables 1, 7, 8 and Fig. 4 report point success rates without error bars, confidence intervals, or multi-seed standard errors, despite stochastic sampling of candidates and environment stochasticity. The load-bearing 9B+SVA vs 27B comparison in §5.3/Fig. 6 (53.6% vs 46.7%, “7 points … 27% lower latency”) and the RoboMonkey margins (+5.2 on RoboTwin average) cannot be assessed for reliability without uncertainty. Please report mean±std (or bootstrap CIs) over seeds/episodes for main tables and the scaling curve, and state the number of evaluation episodes per cell.
  3. The abstract’s “9B VLA … outperform a 27B VLA” claim is drawn from EmbodiedBench with Qwen3.5 backbones acting via high-level discrete skills (§5, Table 1, Fig. 6), not from continuous-control OpenVLA/π0/π0.5 policies. Calling these agents “VLAs” and equating the result with VLA model-size scaling blurs two regimes. Clarify in abstract and §5.3 that the cost-effectiveness result is on EmbodiedBench language-conditioned agents, and either replicate a size-scaling comparison for a true continuous VLA family or qualify the claim accordingly.
  4. §4.2–4.3 and Eq. (5) present the Q-ensemble as a general consequence evaluator, but D.3 trains a separate Q-model for each of EB-Habitat, EB-Navigation, SimplerEnv, and RoboTwin. Combined with suite-specific MCTS labels, this weakens the claim that evaluation is a portable, backbone-preserving capability rather than per-benchmark supervised reranking. State this limitation prominently near the main results, and discuss how much of the gain would remain if a single Q-model were shared across suites or if only offline policy rollouts (no MCTS) were available outside the training suite.

Circularity Check

0 steps flagged

No significant circularity: SVA is an empirical search-distill-rerank pipeline scored by external simulator success, not a derivation that reduces to its own inputs.

full rationale

The paper’s load-bearing chain is methodological, not definitional. Pass@k (Sec. 3) is a diagnostic of the frozen policy’s sampling distribution under independent environment rollouts; it does not define or fit the later success metric. Search (Sec. 4.1) collects MCTS-backed returns G(s_L) from a resettable simulator under the frozen π_θ; Value (Sec. 4.2) regresses a separate Q-model to those external targets via Smooth-L1 (Eq. 3); Act (Sec. 4.3) selects among N candidates by uncertainty-regularized Q (Eq. 5). Reported gains (Tables 1, 7–8; Fig. 6) are environment success rates on held-out episodes/seeds, not Q-values or fitted parameters renamed as predictions. The 9B+SVA vs 27B latency/success comparison is a measured compute–accuracy tradeoff under the same external scoring, not forced by construction. There is no self-definitional loop, no uniqueness theorem imported from the authors, no ansatz smuggled via self-citation, and no renaming of a known closed-form result. Mild suite-internal train/eval splitting and simulator dependence are transfer/assumption risks, not circularity of the derivation chain. Score 0 is therefore appropriate.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 1 invented entities

The central empirical claim rests on standard MDP formalism, the assumption that simulator success returns are valid long-horizon labels for a deployable Q-model, and several hand-chosen search and reranking hyperparameters. No new physical entities are postulated; the Q-model and SVA stages are engineered constructs with independent empirical handles (held-out success rates).

free parameters (3)
  • λ1 (uncertainty penalty), λ2 (log-prior bonus) = 0.1 / 0.1 (EmbodiedBench); 0.1 / 0 (SimplerEnv); 0 / 0 (RoboTwin)
    Hand-set regularization coefficients in the Act-stage selection rule (Eq. 5); values 0.1/0.1 or 0/0 depending on benchmark (Table 5).
  • c_puct, N candidates, search depth/sims, γ = e.g. c_puct 2.0–4.5; N=16–32; γ=0.98–0.99
    MCTS and Act budgets chosen per benchmark (Tables 4–5); control exploration and test-time compute; not derived from first principles.
  • Q-ensemble size and LoRA rank = 5 heads; LoRA r=16
    Architecture choices (5 heads, rank 16, α=32) that affect uncertainty estimates and capacity; selected by authors.
axioms (4)
  • domain assumption Embodied tasks are language-conditioned MDPs with a simulator-provided success/return signal usable as Q-targets.
    Stated in Sec. 4; underpins both MCTS backup and supervised Q regression.
  • domain assumption Frozen VLA sampling distributions place non-trivial mass on successful trajectories (pass@k premise).
    Diagnostic Sec. 3; if false, best-of-N evaluation cannot recover success.
  • ad hoc to paper A lightweight VLM with LoRA and MLP heads can regress MCTS returns well enough to rank candidates on held-out states/tasks.
    Core Value-stage modeling choice (Sec. 4.2); validated only empirically in this work.
  • standard math Standard PUCT MCTS and Smooth-L1 regression are valid search/learning procedures for this setting.
    Sec. 4.1–4.2; conventional algorithms applied without new theory.
invented entities (1)
  • SVA (Search–Value–Act) pipeline / search-distilled Q-evaluator for frozen VLAs independent evidence
    purpose: Amortize MCTS into a deployable action scorer that preserves a frozen backbone.
    Named framework of the paper; independent evidence is the reported held-out success and latency results, not an external physical prediction.

pith-pipeline@v1.1.0-grok45 · 24986 in / 3344 out tokens · 28180 ms · 2026-07-12T00:11:51.044089+00:00 · methodology

0 comments
read the original abstract

Vision-Language-Action (VLA) models acquire broad embodied capabilities through large-scale pretraining, yet their generalization remains far more fragile than that of LLMs and VLMs. The prevailing remedy, post-training via supervised fine-tuning or reinforcement learning, improves task-specific performance but narrows the generalist capability that makes pretraining valuable. We identify a key bottleneck: VLA failures stem not only from action generation but also from action evaluation. A diagnostic pass@k study confirms that frozen VLAs already contain competent behaviors in their output distribution, with overall success rates rising from 33% at pass@1 to 92% at pass@32. Inspired by this, we propose SVA (Search, Value, and Act), a simple framework that equips frozen VLA policies with long-term consequence awareness. SVA first uses Monte-Carlo tree search in simulation to fully explore the VLA's output distribution and collect diverse trajectories annotated with empirical returns; this knowledge is then distilled into a lightweight Q-value model that predicts the expected consequence of candidate actions; at deployment, the frozen VLA proposes multiple candidates and the evaluator selects the one with the highest uncertainty-regularized Q-value, requiring no simulator access. By decoupling action proposal from consequence evaluation, SVA preserves the generalization capacity of the VLA backbone while substantially improving task success rates. Experiments across embodied benchmarks show that SVA consistently improves generalization on unseen tasks and exhibits strong test-time scaling behavior. Strikingly, SVA enables a 9B VLA to outperform a 27B VLA by 7 points at 27% lower inference latency, suggesting that scaling test-time evaluation is more cost-effective than scaling model size.

Figures

Figures reproduced from arXiv: 2607.03751 by Chunlin Chen, Haoran Li, Pichao Wang, Wenhao Wu, Xinyi Xie, Yicheng Dong, Zhanyu Liu, Zhenhong Sun, Zhi Wang, Zican Hu.

Figure 1
Figure 1. Figure 1: Post-training vs. test-time scaling. We argue that the root cause of VLA deployment failures lies not only in action generation but also in the absence of action evaluation, i.e., the inabil￾ity to anticipate the consequences of a proposed action before execution. VLAs are trained to im￾itate, not to evaluate: they produce locally plau￾sible actions given the current observation [15], but receive no signal… view at source ↗
Figure 2
Figure 2. Figure 2: Pass@k results across embodied benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of SVA. (a) Search: MCTS explores the frozen VLA’s policy distribution in simulation, collecting trajectories with empirical returns. (b) Value: A lightweight Q-model is distilled from the searched data to predict action consequences. (c) Act: The frozen VLA proposes N candidates and the Q-model selects the best one without simulator access. compounding execution errors, assign non-trivial probabi… view at source ↗
Figure 4
Figure 4. Figure 4: Success rates on SimplerEnv and RoboTwin. Full results in Tables [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Ablation on EB-Navigation (Qwen3.5-9B). Full results in [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Scaling candidates N improves suc￾cess rate while Q-model scoring adds negligible overhead versus proposal latency (Qwen3.5-9B). SVA exhibits strong test-time scaling. We ana￾lyze SVA’s scaling behavior by sweeping the num￾ber of candidate actions N at deployment and mea￾suring both success rate and inference latency. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Case 1 – distractor-aware instruction following. The base policy follows the salient but irrelevant apple-on-sofa clause and fails after repeated invalid recovery actions. SVA selects a plate-centric plan and completes the task in 4 steps. CASE 2 Find a wrench and move it to the right of the sink. Base Policy ✗ Failure Step 1: wrong search Step 2: invalid pickup Step 10: repeated failure Step 20: failure S… view at source ↗
Figure 8
Figure 8. Figure 8: Case 2 – spatial-relation grounding. For the instruction Find a wrench and move it to the right of the sink, the base policy searches an incorrect location and issues repeated invalid pickup actions, while SVA grounds the spatial relation and completes the rearrangement. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

55 extracted references · 18 linked inside Pith

  1. [1]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. RT-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    Pith/arXiv arXiv 2022

  2. [2]

    Zitkovich, T

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. InConference on Robot Learning, pages 2165–2183, 2023

    2023

  3. [3]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. P. Foster, P. R. Sanketi, Q. Vuong, et al. OpenVLA: An open-source vision-language-action model. In Annual Conference on Robot Learning, 2024

    2024

  4. [4]

    Intelligence, K

    P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, et al. π0.5: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Bjorck, F

    J. Bjorck, F. Castañeda, N. Cherniadev, X. Da, R. Ding, L. Fan, Y . Fang, D. Fox, F. Hu, S. Huang, et al. GR00T N1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. GPT-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  7. [7]

    X. Li, K. Hsu, J. Gu, O. Mees, K. Pertsch, H. R. Walke, C. Fu, I. Lunawat, I. Sieh, S. Kirmani, S. Levine, J. Wu, C. Finn, H. Su, Q. Vuong, and T. Xiao. Evaluating real-world robot manip- ulation policies in simulation. InAnnual Conference on Robot Learning, volume 270, pages 3705–3728, 2024

    2024

  8. [8]

    O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, et al. Octo: An open-source generalist robot policy.arXiv preprint arXiv:2405.12213, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Y . Chen, S. Tian, S. Liu, Y . Zhou, H. Li, and D. Zhao. ConRFT: A reinforced fine-tuning method for VLA models via consistency policy.arXiv preprint arXiv:2502.05450, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    H. Zang, M. Wei, S. Xu, Y . Wu, Z. Guo, Y . Wang, H. Lin, L. Shi, Y . Xie, Z. Xu, et al. RLinf- VLA: A unified and efficient framework for reinforcement learning of vision-language-action models.arXiv preprint arXiv:2510.06710, 2025

    arXiv 2025

  11. [11]

    J. Wen, Y . Zhu, J. Li, M. Zhu, Z. Tang, K. Wu, Z. Xu, N. Liu, R. Cheng, C. Shen, et al. TinyVLA: Towards fast, data-efficient vision-language-action models for robotic manipulation. IEEE Robotics and Automation Letters, 2025

    2025

  12. [12]

    Pertsch, K

    K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine. FAST: Efficient action tokenization for vision-language-action models.arXiv preprint arXiv:2501.09747, 2025

    Pith/arXiv arXiv 2025

  13. [13]

    M. J. Kim, C. Finn, and P. Liang. Fine-tuning vision-language-action models: Optimizing speed and success.arXiv preprint arXiv:2502.19645, 2025. 10

    Pith/arXiv arXiv 2025

  14. [14]

    B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone. LIBERO: Benchmarking knowledge transfer for lifelong robot learning. InAdvances in Neural Information Processing Systems, volume 36, pages 44776–44791, 2023

    2023

  15. [15]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. π0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024

    Pith/arXiv arXiv 2024

  16. [16]

    De Haan, D

    P. De Haan, D. Jayaraman, and S. Levine. Causal confusion in imitation learning. InAdvances in Neural Information Processing Systems, volume 32, 2019

    2019

  17. [17]

    Codevilla, E

    F. Codevilla, E. Santana, A. M. López, and A. Gaidon. Exploring the limitations of behavior cloning for autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9329–9338, 2019

    2019

  18. [18]

    Nakamoto, O

    M. Nakamoto, O. Mees, A. Kumar, and S. Levine. Steering your generalists: Improving robotic foundation models via value guidance. InAnnual Conference on Robot Learning, pages 4996–5013, 2025

    2025

  19. [19]

    R. S. Sutton, A. G. Barto, et al.Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998

    1998

  20. [20]

    R. Sutton. The bitter lesson, 2019. URL http://www.incompleteideas.net/IncIdeas/B itterLesson.html

    2019

  21. [21]

    H. Li, Y . Zuo, J. Yu, Y . Zhang, Z. Yang, K. Zhang, X. Zhu, Y . Zhang, T. Chen, G. Cui, et al. SimpleVLA-RL: Scaling VLA training via reinforcement learning. InInternational Conference on Learning Representations, 2026

    2026

  22. [22]

    Intelligence, A

    P. Intelligence, A. Amin, R. Aniceto, A. Balakrishna, K. Black, K. Conley, G. Connors, J. Darpinian, K. Dhabalia, J. DiCarlo, et al. π0.6: a VLA that learns from experience.arXiv preprint arXiv:2511.14759, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    W. Xiao, H. Lin, A. Peng, H. Xue, T. He, Z. Luo, Y . Xie, F. Hu, L. Fan, G. Shi, and Y . Zhu. Self- improving vision-language-action models with data generation via residual RL. InInternational Conference on Learning Representations, 2026

    2026

  24. [24]

    J. Yang, M. S. Mark, B. Vu, A. Sharma, J. Bohg, and C. Finn. Robot fine-tuning made easy: Pre-training rewards and policies for autonomous real-world reinforcement learning. InIEEE International Conference on Robotics and Automation, pages 4804–4811, 2024

    2024

  25. [25]

    R. Zhao, S. Xu, R. Jin, Y . Deng, Y . Tai, K. Jia, and G. Liu. Sim2real VLA: Zero-shot generalization of synthesized skills to realistic manipulation. InInternational Conference on Learning Representations, 2026

    2026

  26. [26]

    A. J. Hancock, X. Wu, L. Zha, O. Russakovsky, and A. Majumdar. Actions as language: Fine-tuning VLMs into VLAs without catastrophic forgetting. InInternational Conference on Learning Representations, 2026

    2026

  27. [27]

    Lightman, V

    H. Lightman, V . Kosaraju, Y . Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe. Let’s verify step by step. InInternational Conference on Learning Representations, volume 2024, pages 39578–39601, 2024

    2024

  28. [28]

    Uesato, N

    J. Uesato, N. Kushman, R. Kumar, F. Song, N. Siegel, L. Wang, A. Creswell, G. Irving, and I. Higgins. Solving math word problems with process-and outcome-based feedback.arXiv preprint arXiv:2211.14275, 2022

    Pith/arXiv arXiv 2022

  29. [29]

    S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y . Cao, and K. Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. InAdvances in Neural Information Processing Systems, volume 36, pages 11809–11822, 2023. 11

    2023

  30. [30]

    Z. Wan, X. Feng, M. Wen, S. M. Mcaleer, Y . Wen, W. Zhang, and J. Wang. AlphaZero-like tree-search can guide large language model decoding and training. InInternational Conference on Machine Learning, volume 235, pages 49890–49920, 2024

    2024

  31. [31]

    Zhang, S

    D. Zhang, S. Zhoubian, Z. Hu, Y . Yue, Y . Dong, and J. Tang. ReST-MCTS∗: LLM self-training via process reward guided tree search. InAdvances in Neural Information Processing Systems, volume 37, pages 64735–64772, 2024

    2024

  32. [32]

    Huang, F

    W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y . Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. InConference on Robot Learning, pages 1769–1782, 2023

    2023

  33. [33]

    Liang, X

    Y . Liang, X. Wang, K. Wang, S. Wang, X. Peng, H. Chen, D. K. H. Chua, and P. Vadakkepat. Adaptive action chunking at inference-time for vision-language-action models.arXiv preprint arXiv:2604.04161, 2026

    Pith/arXiv arXiv 2026

  34. [34]

    Ayanthi, D

    A. Ayanthi, D. Priyasad, T. Fernando, S. Sridharan, C. Fookes, and P. Moghadam. ReVer: Reasoning-guided verification for embodied agents. InICRA 2026 Workshop: From Data to Decisions: VLA Pipelines for Real Robots, 2026

    2026

  35. [35]

    Ichter, A

    B. Ichter, A. Brohan, Y . Chebotar, C. Finn, K. Hausman, et al. Do as I can, not as I say: Grounding language in robotic affordances. InAnnual Conference on Robot Learning, 2022

    2022

  36. [36]

    J. Kwok, C. Agia, R. Sinha, M. Foutter, S. Li, I. Stoica, A. Mirhoseini, and M. Pavone. RoboMonkey: Scaling test-time sampling and verification for vision-language-action models. InAnnual Conference on Robot Learning, 2025

    2025

  37. [37]

    K. Ren, A. Salamatian, K. Pattison, and C. Neary. V-VLAPS: Value-guided planning for vision-language-action models.arXiv preprint arXiv:2601.00969, 2026

    Pith/arXiv arXiv 2026

  38. [38]

    Singhi, C

    N. Singhi, C. Bialas, S. Jauhri, V . Prasad, G. Chalvatzaki, M. Rohrbach, and A. Rohrbach. Think twice, act once: Verifier-guided action selection for embodied agents.arXiv preprint arXiv:2605.12620, 2026

    Pith/arXiv arXiv 2026

  39. [39]

    H. Song, D. Qu, Y . Yao, Q. Chen, Q. Lv, Y . Tang, M. Shi, G. Ren, M. Yao, B. Zhao, et al. Hume: Introducing system-2 thinking in visual-language-action model.arXiv preprint arXiv:2505.21432, 2025

    Pith/arXiv arXiv 2025

  40. [40]

    J. Kwok, X. Zhang, M. Xu, Y . Liu, A. Mirhoseini, C. Finn, and M. Pavone. Scaling verification can be more effective than scaling policy learning for vision-language-action alignment.arXiv preprint arXiv:2602.12281, 2026

    arXiv 2026

  41. [41]

    S. Yang, Y . Zhang, H. He, L. Pan, X. Li, C. Bai, and X. Li. Steering vision-language-action models as anti-exploration: A test-time scaling approach.arXiv preprint arXiv:2512.02834, 2025

    arXiv 2025

  42. [42]

    Z. Li, J. Liu, Z. Dong, T. Teng, Q. Rouxel, D. Caldwell, and F. Chen. Towards deploying VLA without fine-tuning: Plug-and-play inference-time VLA policy steering via embodied evolutionary diffusion.IEEE Robotics and Automation Letters, 11(5):6234–6241, 2026

    2026

  43. [43]

    M. S. Mark, T. Gao, G. G. Sampaio, M. K. Srirama, A. Sharma, C. Finn, and A. Kumar. Policy agnostic RL: Offline RL and online RL fine-tuning of any class and backbone.arXiv preprint arXiv:2412.06685, 2024

    Pith/arXiv arXiv 2024

  44. [44]

    Y . Wu, A. Li, T. Hermans, F. Ramos, A. Bajcsy, and C. PÊrez-D’Arpino. Do what you say: Steering vision-language-action models via runtime reasoning-action alignment verification. arXiv preprint arXiv:2510.16281, 2025. 12

    arXiv 2025

  45. [45]

    S. Jang, D. Kim, C. Kim, Y . Kim, and J. Shin. Verifier-free test-time sampling for vision- language-action models. InThe Fourteenth International Conference on Learning Representa- tions, 2026

    2026

  46. [46]

    S. Yang, Y . Du, S. Ghasemipour, J. Tompson, L. Kaelbling, D. Schuurmans, and P. Abbeel. Learning interactive real-world simulators. InInternational Conference on Learning Represen- tations, volume 2024, pages 45210–45234, 2024

    2024

  47. [47]

    S. Zhou, Y . Du, J. Chen, Y . Li, D.-Y . Yeung, and C. Gan. RoboDreamer: Learning compositional world models for robot imagination. InInternational Conference on Machine Learning, volume 235, pages 61885–61896, 2024

    2024

  48. [48]

    G. Zhou, H. Pan, Y . Lecun, and L. Pinto. DINO-WM: World models on pre-trained visual features enable zero-shot planning. InInternational Conference on Machine Learning, volume 267, pages 79115–79135, 2025

    2025

  49. [49]

    Neary, O

    C. Neary, O. G. Younis, A. Kuramshin, O. Aslan, and G. Berseth. Improving pre-trained vision-language-action policies with model-based search.arXiv preprint arXiv:2508.12211, 2025

    arXiv 2025

  50. [50]

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brockman, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    Pith/arXiv arXiv 2021

  51. [51]

    Lu and T

    K. Lu and T. M. Lab. On-policy distillation.Thinking Machines Lab: Connectionism, 2025. https://thinkingmachines.ai/blog/on-policy-distillation

    2025

  52. [52]

    Silver, A

    D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V . Panneershelvam, M. Lanctot, et al. Mastering the game of Go with deep neural networks and tree search.Nature, 529(7587):484–489, 2016

    2016

  53. [53]

    R. Yang, H. Chen, J. Zhang, M. Zhao, C. Qian, K. Wang, Q. Wang, T. V . Koripella, M. Movahedi, M. Li, H. Ji, H. Zhang, and T. Zhang. EmbodiedBench: Comprehensive benchmarking multi- modal large language models for vision-driven embodied agents. InInternational Conference on Machine Learning, volume 267, pages 70576–70631, 2025

    2025

  54. [54]

    T. Chen, Z. Chen, B. Chen, Z. Cai, Y . Liu, Z. Li, Q. Liang, X. Lin, Y . Ge, Z. Gu, et al. RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation.arXiv preprint arXiv:2506.18088, 2025

    Pith/arXiv arXiv 2025

  55. [55]

    Snell, J

    C. Snell, J. Lee, K. Xu, and A. Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. InInternational Conference on Learning Representations, volume 2025, pages 10131–10165, 2025. 13 Appendix A Limitations and Future Work 15 B Benchmarks 15 C On the Real-Robot Relevance of Our Simulation Study 16 D Exp...

    arXiv 2025