{"id":"eec1e2d8-809e-4c40-a03a-d38105b5ecf1","arxiv_id":"2607.03751","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Distilling MCTS returns into a lightweight Q-model lets frozen VLAs select higher-return actions at test time, lifting success rates and letting a 9B model beat a 27B baseline at lower latency.","lead":"Frozen robot VLAs already contain good actions in their sampling distribution; the bottleneck is picking them. SVA distills Monte-Carlo tree search into a small Q-evaluator so a frozen policy can choose better actions at test time without fine-tuning.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The 9B-vs-27B cost-effectiveness claim rests on Q-models trained with same-benchmark MCTS labels, so held-out gains may not demonstrate true unseen-task transfer of the evaluator.","rationale":"The pass@k diagnostic (Sec. 3, Fig. 2) and the within-suite ablations (Fig. 5, Table 9) are solid: competent actions exist in the proposal distribution, and multi-candidate + Q-scoring are necessary for the reported lifts. The reader’s weakest assumption (sim-trained Q transfer without real-robot recalibration) is real and already acknowledged in Appendix A, but it is slightly broader than the single most load-bearing soft spot for the *central* claim. That claim is not merely “works in sim”; it is that decoupling proposal from evaluation preserves generality and that scaling evaluation is more cost-effective than scaling model size (abstract; Sec. 5.3; 9B+SVA vs 27B). Those statements require the Q-model to be a transferable consequence evaluator, not a same-suite return memorizer. The paper never reports a Q-model trained on one benchmark/suite and applied zero-shot to another, nor a true out-of-family task split. Without that, the multi-backbone gains and the latency comparison remain conditional on in-distribution episode hold-outs. I therefore keep CONDITIONAL (same as the reader) rather than REJECT: the method is still a clear, useful test-time recipe with honest limitations, but the generality/cost-effectiveness framing needs the cross-suite check above. Agreement is partial because the reader correctly located transfer risk; I narrow it from “sim vs real / train-split mismatch” to “in-suite vs cross-task transfer of the distilled evaluator,” which more directly undercuts the strongest claim as written.","tokens_in":20881,"tokens_out":863,"duration_ms":7078,"concrete_test":"Train the Q-model only on EB-Habitat MCTS data (or only on SimplerEnv), freeze it, and evaluate best-of-N selection on EB-Navigation (or RoboTwin) with the same frozen proposers and N as in Tables 1/7–8. If average Δ over base falls below ~half the in-suite Δ (or becomes negative), the evaluator is suite-specific and the 9B-vs-27B cost-effectiveness claim does not support general test-time evaluation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim is that frozen-VLA generality is preserved while success rises, and that 9B+SVA (best-of-8, 53.6%) beats single-shot 27B (46.7%) at lower latency. That comparison and the multi-benchmark gains all use a Q-model whose targets come from MCTS on the training split of the same benchmark (Appendix D.2: EB episodes 1–30/1–36; SimplerEnv 15 seeds; RoboTwin 30 seeds), then evaluated on held-out episodes of those same task families (D.4). Sec. 5.1 and the abstract phrase this as “generalization on unseen tasks,” but the split is episode/seed hold-out within a fixed task suite, not a new task distribution or a Q-model trained on one suite and tested on another. If the evaluator mainly memorizes suite-specific return structure rather than learning transferable long-horizon consequences, the “preserve generality + cheaper than scaling model size” headline is overstated: gains would be suite-specific reranking, not a general evaluation capability that travels with the frozen backbone. The reader correctly flags simulator/train-split transfer; the sharper load-bearing point is that even within sim, the paper never isolates cross-task or cross-benchmark transfer of the distilled Q-model.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper argues that frozen VLA failures are largely an action-evaluation bottleneck rather than pure generation failure, supported by a pass@k diagnostic (overall success rising from ~33% at pass@1 to ~92% at pass@32). It proposes SVA: MCTS in simulation mines return-annotated trajectories from the frozen policy’s distribution; those returns are distilled into a lightweight ensemble Q-model (small VLM + LoRA + MLP heads); at deployment the frozen policy proposes N candidates and the Q-model selects via uncertainty-regularized scores, without simulator access. Experiments on EmbodiedBench (multiple LM/VLM backbones), SimplerEnv (π0), and RoboTwin 2.0 (π0.5) report consistent gains over base policies and over a RoboMonkey-style preference reranker, plus a test-time scaling plot in which Qwen3.5-9B+SVA (best-of-8) exceeds single-shot Qwen3.5-27B by ~7 points at lower latency. Ablations remove MCTS, the Q-model, or multi-candidate selection.","tokens_in":21309,"tokens_out":1103,"duration_ms":16389,"significance":"If the results hold under tighter evaluation, the work is a useful contribution to test-time scaling for embodied policies: it cleanly separates proposal from consequence evaluation, keeps multi-billion-parameter backbones frozen, and shows that a cheap verifier can convert latent pass@k competence into higher single-trajectory success. Strengths include a coherent diagnostic, a simple Search–Value–Act recipe, multi-backbone and multi-benchmark evidence, direct comparison to RoboMonkey under matched candidate budgets, ablations of each stage, and an explicit latency-vs-N analysis. The framing that scaling evaluation can be more cost-effective than scaling model size is timely for robotics, where fine-tuning large VLAs is expensive and generality is fragile. The limitations appendix is appropriately candid about staged search/value learning, simulator dependence, and missing real-robot validation.","major_comments":[{"comment":"Abstract and §5.1 claim that SVA “consistently improves generalization on unseen tasks,” but Appendix D.2–D.4 show only episode/seed hold-outs within the same task suites (EB episodes 1–30/1–36 train vs 31–50/37–60 eval; SimplerEnv 15 train seeds / 9 held-out; RoboTwin 30/20 seeds), with separate Q-models trained per benchmark (D.3). That is within-suite reranking, not transfer to a new task distribution or cross-benchmark transfer of the evaluator. The headline “preserve generality + cheaper than scaling model size” depends on the Q-model learning transferable long-horizon consequences rather than suite-specific return structure. Either add a cross-task/cross-benchmark Q-transfer experiment (train Q on one suite, evaluate on another with the same frozen proposer) or substantially revise the abstract/§5.1 language to match the actual split protocol.","section":null},{"comment":"Tables 1, 7, 8 and Fig. 4 report point success rates without error bars, confidence intervals, or multi-seed standard errors, despite stochastic sampling of candidates and environment stochasticity. The load-bearing 9B+SVA vs 27B comparison in §5.3/Fig. 6 (53.6% vs 46.7%, “7 points … 27% lower latency”) and the RoboMonkey margins (+5.2 on RoboTwin average) cannot be assessed for reliability without uncertainty. Please report mean±std (or bootstrap CIs) over seeds/episodes for main tables and the scaling curve, and state the number of evaluation episodes per cell.","section":null},{"comment":"The abstract’s “9B VLA … outperform a 27B VLA” claim is drawn from EmbodiedBench with Qwen3.5 backbones acting via high-level discrete skills (§5, Table 1, Fig. 6), not from continuous-control OpenVLA/π0/π0.5 policies. Calling these agents “VLAs” and equating the result with VLA model-size scaling blurs two regimes. Clarify in abstract and §5.3 that the cost-effectiveness result is on EmbodiedBench language-conditioned agents, and either replicate a size-scaling comparison for a true continuous VLA family or qualify the claim accordingly.","section":null},{"comment":"§4.2–4.3 and Eq. (5) present the Q-ensemble as a general consequence evaluator, but D.3 trains a separate Q-model for each of EB-Habitat, EB-Navigation, SimplerEnv, and RoboTwin. Combined with suite-specific MCTS labels, this weakens the claim that evaluation is a portable, backbone-preserving capability rather than per-benchmark supervised reranking. State this limitation prominently near the main results, and discuss how much of the gain would remain if a single Q-model were shared across suites or if only offline policy rollouts (no MCTS) were available outside the training suite.","section":null}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful core is the pass@k diagnosis plus a simple Search–Value–Act recipe: MCTS over a frozen VLA’s proposals in sim, distill returns into a small ensemble Q-model, then uncertainty-regularized best-of-N at deploy with no simulator. That framing is clearer than most VLA post-training papers, and the multi-backbone EmbodiedBench table plus the RoboMonkey-matched comparison on SimplerEnv/RoboTwin are enough to take seriously.\n\nWhat is actually new is not MCTS or verifiers—those are known—but the packaging for frozen VLAs: keep the multi-billion backbone untouched, amortize search into a 0.8B Q-head with LoRA, and show sub-linear latency growth so best-of-N is practical. The 9B+SVA vs single-shot 27B numbers (roughly +7 points at lower wall-clock) are the sharpest result; ablations (drop MCTS, drop Q, drop multi-cand) line up with the story. Related work is fair; they cite SayCan, V-GPS, RoboMonkey, world-model rollouts and position against them without hand-waving.\n\nSoft spots, in proportion. The stress-test is right on the language: “generalization on unseen tasks” is held-out episodes/seeds within the same suites (Appendix D), not a Q-model trained on one family and tested on another. So the cost-effectiveness claim is suite-local reranking, not proven transferable evaluation that rides free with any frozen backbone. Simulator-only, staged (not online) search–learning, free knobs (λ, c_puct, N, depth), no error bars, no code—standard for this venue, but they limit how hard you should lean on the headline. Appendix A is honest about that.\n\nWho it is for: people shipping or evaluating VLAs who care about test-time compute vs fine-tuning cost. Not a theory paper. Math is standard PUCT + Smooth-L1; data pattern is coherent. I would send it to referees; they will push on cross-task transfer and real robots, which is the right conversation. Worth reading and citing if you work on VLA inference or verifiers.","headline":"Clean diagnostic + practical frozen-VLA reranker; the 9B-vs-27B cost claim is real within suite, but \"unseen tasks\" is episode hold-out, not cross-task evaluator transfer.","tokens_in":21956,"tokens_out":570,"would_cite":true,"duration_ms":5442,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Frozen robot VLAs already contain good actions; a distilled search evaluator can pick them out without fine-tuning the backbone.","keywords":["Vision-Language-Action models","test-time scaling","Monte-Carlo tree search","action evaluation","Q-value model","frozen policies","embodied agents"],"falsifier":"Train the Q-model on the paper’s MCTS procedure for a fixed VLA and benchmark, then measure whether best-of-N selection with that Q-model still beats the same frozen VLA’s single-shot success rate (and the 27B single-shot baseline) on a held-out physical-robot counterpart of the same tasks; collapse of the ranking advantage would falsify the transfer claim.","tokens_in":21788,"feed_emoji":"🤖","tokens_out":974,"duration_ms":7439,"temperature":0.7,"pith_summary":"Vision-language-action models often fail on modestly new tasks not because they cannot generate competent moves, but because they cannot tell a good move from a bad one before acting. A pass@k diagnostic shows success climbing from 33 percent with one try to 92 percent with 32 independent tries, so high-quality behaviors already sit inside the frozen policy’s output distribution. The paper’s answer is SVA: run Monte-Carlo tree search in simulation to mine long-horizon returns, distill those returns into a small Q-value model, then at deployment let the frozen VLA propose several candidates and let the evaluator choose the one with the best uncertainty-regularized score—no simulator needed at runtime. Because the multi-billion-parameter backbone stays untouched, its generalist knowledge is preserved while task success rises. On embodied reasoning and manipulation benchmarks the same recipe lifts multiple backbones, and a 9B model with SVA beats a 27B model by seven points at lower latency, arguing that test-time evaluation can be cheaper than scaling model size.","feed_headline":"9B robot model with search evaluator beats 27B at lower latency","feed_subtitle":"Frozen VLAs already hold good actions; a small distilled Q-model picks them without fine-tuning.","key_machinery":"SVA (Search, Value, and Act): MCTS explores the frozen VLA’s action distribution in simulation and labels edges with empirical returns; those labels train a small ensemble Q-model; at test time the frozen VLA proposes N candidates and the Q-model selects the highest uncertainty-regularized score.","core_discovery":"VLA failures are an evaluation bottleneck as much as a generation bottleneck: competent actions already exist in the frozen policy’s distribution, and a lightweight Q-model distilled from Monte-Carlo tree search can rank those actions by long-term consequence well enough to raise success rates without any backbone update.","pith_inferences":["If the evaluation bottleneck is as general as claimed, other imitation-trained robot policies (not only VLAs) may also gain more from a distilled long-horizon verifier than from further supervised fine-tuning.","Closing the online search–learning loop the authors leave for future work could turn SVA into a self-improving cycle that still never updates the generalist backbone.","Real-robot deployment may need only light residual calibration of the Q-heads, because the proposal distribution already comes from real-data-pretrained VLAs and the evaluator never relies on privileged simulator features at inference."],"forward_implications":["Frozen multi-billion-parameter VLAs can be improved at deployment by adding only a small external evaluator, avoiding costly and generality-narrowing backbone fine-tuning.","Test-time compute can be spent on more candidate proposals and Q-scoring rather than larger models; the paper’s 9B+SVA result already beats 27B single-shot at lower latency.","Pass@k diagnostics become a practical probe: if success rises sharply with k, an evaluation-centric fix is more promising than rewriting the generator.","The same Search–Value–Act recipe applies across discrete high-level skills and continuous action chunks once actions are tokenized for the Q-model."],"fun_headline_variants":["Frozen 9B VLA plus search-distilled Q beats 27B at lower latency","Distilled MCTS Q ranks frozen VLA actions by long-term consequence","Competent actions already in frozen VLAs; lightweight Q selects them","SVA: tree-search knowledge distilled so frozen VLAs evaluate before acting","No backbone update: search evaluator raises VLA success and scales at test"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That success labels gathered by tree search only on the training split of a resettable simulator, once distilled into a small Q-model, remain accurate enough to rank actions on held-out episodes and tasks without further online search or real-robot recalibration.","fun_headline_variants_meta":{"raw":{"variants":["Frozen 9B VLA plus search-distilled Q beats 27B at lower latency","Distilled MCTS Q ranks frozen VLA actions by long-term consequence","Competent actions already in frozen VLAs; lightweight Q selects them","SVA: tree-search knowledge distilled so frozen VLAs evaluate before acting","No backbone update: search evaluator raises VLA success and scales at test"]},"model":"grok-4.5","effort":"low","cost_usd":0.005416,"raw_usage":{"total_tokens":1540,"prompt_tokens":860,"num_sources_used":0,"completion_tokens":84,"cost_in_usd_ticks":54160000,"prompt_tokens_details":{"text_tokens":860,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":596,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":860,"tokens_out":84,"duration_ms":5146,"temperature":1.0,"reasoning_tokens":596,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-12T00:11:51.044089+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train the Q-model on the paper’s MCTS procedure for a fixed VLA and benchmark, then measure whether best-of-N selection with that Q-model still beats the same frozen VLA’s single-shot success rate (and the 27B single-shot baseline) on a held-out physical-robot counterpart of the same tasks; collapse of the ranking advantage would falsify the transfer claim.","supporting_citations":[],"review_version":1}