{"id":"54733652-7aee-4095-b5fd-07ebf3594203","arxiv_id":"2608.09591","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A vision-language driving planner that adapts its chain-of-thought to the scene and uses tree-search-guided reinforcement learning reports best average trajectory error and planning score on nuScenes and NAVSIM.","lead":"FactorDrive is a driving model that plans trajectories by first writing a short reasoning chain that adapts to the scene, then improving that reasoning with a search-guided reinforcement step. On two standard driving benchmarks it reports the best open-loop and closed-loop-oriented planning scores to date.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"nuScenes navigation command is built from the GT future trajectory (Supp. A.2.1) and is fed to the planner at test time; because the claimed SOTA margin over Drive-R1 is only 0.01 m average L2, this future-derived route cue may be the load-bearing difference.","rationale":"The reader's weakest assumption identified the nuScenes navigation-command construction as a possible leak of future maneuver information. My independent reading of the supplementary material confirms that this is the most load-bearing concern: the command is computed from the expert future trajectory, it is a direct inference-time input, and the claimed nuScenes SOTA margin over the previous best method is only 0.01 m average L2. If the command were removed or neutralized and the metric did not degrade, the concern would be refuted; if it does degrade, the open-loop SOTA claim is an artifact of the evaluation protocol rather than of the proposed method's reasoning or search components. I considered other potential issues, including the absence of variance reporting, the lack of released code, and the off-policy reuse of cached MCTS responses in QS-GRPO. These are real limitations, but they are secondary: the single-run, no-code issues affect reproducibility, while the off-policy reuse concern affects the mechanism's theoretical framing more than the benchmark numbers. None is as directly tied to the central SOTA assertion as the future-derived command. The NAVSIM result is less affected because official route instructions exist and are normalized by the benchmark, so the central claim on NAVSIM can stand independently. The conditional verdict remains appropriate: the method is well documented and the internal ablations are directionally consistent, but the nuScenes open-loop SOTA claim should not be accepted until the command-leak concern is settled with the concrete test above. No verdict change is needed because the reader already assigned CONDITIONAL; the same concern, made more precise, does not push the assessment to REJECT, since a corrected protocol or a robustness check could restore the claim.","tokens_in":33356,"tokens_out":6352,"duration_ms":63775,"concrete_test":"Hold the released FactorDrive nuScenes checkpoint and decoding settings fixed, and re-run the full 4,669-frame validation split with c_nav set to 'Go Straight' for every sample, leaving all other inputs unchanged. If the average L2 error rises above 0.31 m, or if FactorDrive no longer beats Drive-R1 on average L2, then the open-loop SOTA claim depends on the future-derived navigation command and the protocol must be corrected (e.g., by using only map/current-pose-derived commands or by re-computing all baselines with the same command input) before the claim can be accepted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim on nuScenes rests on a comparison where FactorDrive receives a navigation command that is itself a deterministic function of the expert future trajectory. Section 3.1 defines the model input as z = (q, V, e, c_nav), where c_nav is Go Straight, Turn Left, or Turn Right. Supplementary A.2.1 states that, since nuScenes lacks route labels, the authors 'construct a navigation target point by accumulating approximately 20 m along the expert future trajectory polyline' and then label the command by the lateral displacement of that point relative to the trajectory start. This command is part of the inference-time input (Figure 4 prompt) and is computed from the ground-truth future trajectory at evaluation time. A coarse 'Turn Left' versus 'Turn Right' versus 'Go Straight' label constrains the most important lateral decision in the trajectory and can therefore reduce L2 error, especially at the 3 s horizon, even if the model has no other information about the future. The paper's own ablation study (Table 4) does not vary the navigation command, so there is no evidence isolating its contribution. The claimed improvement over the previous best nuScenes result, Drive-R1, is 0.30 m versus 0.31 m average L2, a one-centimeter margin that is exactly the kind of difference a future-derived route cue could produce. This is a protocol-level concern, not an accusation of fabrication: the comparison is against published baselines whose inputs are not controlled for this command, so the open-loop SOTA claim is not yet established under a fair evaluation protocol.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"FactorDrive is a VLM-based end-to-end driving planner whose main contributions are (i) PCF-CoT, a chain-of-thought dataset in which reasoning steps are organized around planning-critical factors and grounded in spatial-physical evidence; and (ii) QS-GRPO, a post-training method that uses Monte Carlo Tree Search with trajectory-level planning rewards to find high-quality reasoning paths and then optimizes the policy with GRPO. The model is trained with two-stage SFT on a driving-domain instruction corpus and PCF-CoT, then post-trained on hard samples. The paper reports state-of-the-art numbers on nuScenes (0.30 m average L2, 0.08% average collision rate) and NAVSIM (91.0 PDMS), with ablations showing that QS-GRPO contributes a 2.6 PDMS gain over the SFT baseline on NAVSIM and that PCF-CoT reduces collision rate on nuScenes.","tokens_in":33695,"tokens_out":5004,"duration_ms":49687,"significance":"If the reported numbers hold under a fair protocol, the paper is a useful engineering contribution to VLM-based driving planning. Its strengths are that the full QS-GRPO algorithm is specified precisely (Algorithm 1, supplementary C), the training configurations are detailed, and the ablations are internally consistent. The NAVSIM result in particular is supported by a clean training-stage ablation (Table 3), and the qualitative before/after QS-GRPO example in Supplementary E.4 shows a concrete mechanism by which trajectory-level reward search changes reasoning. However, the nuScenes state-of-the-art claim rests on a one-centimeter margin over Drive-R1, and the evaluation protocol gives FactorDrive an input that is a deterministic function of the ground-truth future trajectory. That protocol gap is load-bearing. The paper does not release code or checkpoints at review time, but the reproducibility statement is explicit about planned release.","major_comments":[{"comment":"The nuScenes evaluation is not controlled for the navigation command. The input c_nav is constructed by accumulating approximately 20 m along the expert future trajectory and then labeling it Turn Left/Turn Right/Go Straight based on a 2 m lateral-displacement threshold (Supplementary A.2.1), and Figure 4 shows that this command is part of the inference-time prompt. A 'Turn Left' or 'Turn Right' label directly reveals the dominant lateral maneuver in the next 3-4 s, which can only make the planning task easier than for a model that must infer the maneuver from the scene. The claimed SOTA margin over Drive-R1 is only 0.01 m average L2 (Table 1), and the ablation in Table 4 does not vary c_nav. This is a load-bearing protocol gap: please report nuScenes results with c_nav removed or with an equivalent route input provided to all baselines, and quantify how often the constructed command matches the actual future maneuver.","section":"Section 3.1; Supplementary A.2.1"},{"comment":"No QS-GRPO ablation is reported on nuScenes, so the paper does not demonstrate that the 0.30 m nuScenes result is attributable to the proposed RL method rather than to the navigation-command input. Tables 3 and 5 show the QS-GRPO gain only on NAVSIM (2.6 PDMS over SFT, 0.3 PDMS over standard GRPO). Please add the nuScenes training-stage ablation (SFT vs. SFT+RL), or explicitly limit the QS-GRPO effectiveness claim to NAVSIM.","section":"Section 4.4; Table 3"}],"minor_comments":[{"comment":"The benchmark name is written inconsistently as both 'NAVSIM' and 'NA VSIM' (abstract and tables); please unify.","section":"Throughout"},{"comment":"The paper states that a single complete evaluation run was performed with temperature 0 and top-p 1. Since the margins over prior work are very small, please state explicitly whether the decoding is fully deterministic and whether the baseline numbers are taken from the original papers under the same evaluation protocol; if any stochasticity remains, report multiple runs with standard deviations.","section":"Supplementary D.1"},{"comment":"The PCF-CoT ablation on nuScenes shows that the average L2 error slightly worsens (0.28 m to 0.29 m) while collision rate improves. This is worth discussing in the text: it weakens the claim that grounding reasoning in planning-critical factors directly improves trajectory accuracy on nuScenes.","section":"Table 4"},{"comment":"The human quality check of 120 samples is described, which is good, but for a dataset of over 117,000 samples this sample size yields a wide confidence interval around the reported 90.83% pass rate; please report the per-reviewer agreement and the confidence interval.","section":"Supplementary A.3"},{"comment":"The PDMS formula is only fully defined in the supplementary material; define NC, DAC, EP, TTC, and comfort in the main text where the metric is first used.","section":"Equation (17)"}],"recommendation":"major_revision","confidential_remarks":"I see no reason to suspect fabrication; the issue is a protocol-level control problem in the nuScenes comparison. The paper would become acceptable if the authors rerun nuScenes without the future-derived navigation command, compare baselines under the same input, or explicitly narrow the nuScenes SOTA claim. The NAVSIM results and the QS-GRPO algorithm itself are, on their own, a reasonable contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful new thing in FactorDrive is QS-GRPO: a quality-guided MCTS search over reasoning paths, with a response buffer and confidence-aware group construction, used to post-train a driving VLM. The ablations show it buys a real 2.6 PDMS over SFT and a consistent 0.3 over plain GRPO on NAVSIM. The PCF-CoT dataset is carefully built and the paper is unusually explicit about its pipeline.\n\nThat said, the headline nuScenes SOTA does not hold up as stated. The navigation command is constructed by walking 20 m along the expert future trajectory and labeling it Left/Right/Straight (Supp A.2.1). That command is part of the inference input. Baselines like Drive-R1 don't get that cue, and the claimed margin over Drive-R1 is 0.30 vs 0.31 m average L2. A future-derived route label is exactly the kind of cue that could produce a centimeter. The paper never ablates the command, so the open-loop SOTA claim is not established under a fair protocol. This is a protocol flaw, not a fabrication, and it doesn't sink the NAVSIM results, where the route command comes from the official simulator and the comparison is cleaner.\n\nOther soft spots are minor by comparison: one evaluation run per benchmark (though training uses a fixed seed), no public code or data yet, a 120-sample human audit on a 117k-sample dataset, and a PCF-CoT ablation that shows essentially no L2 gain—its benefit shows up only in collision rate. The QS-GRPO vs GRPO gain is small but directionally consistent.\n\nOverall, the paper is methodologically serious and worth engaging. The integration of search-guided RL with scene-adaptive reasoning is new in the driving-VLA space, and the write-up is honest about its pipeline. It deserves a serious referee. I'd ask the authors to fix the nuScenes protocol before publication: report results with and without the constructed command, and compare to at least one strong baseline given the same command. If the SOTA margin survives that, it's real.","headline":"QS-GRPO is a genuine contribution; the nuScenes SOTA claim is undermined by a future-derived navigation command and needs a protocol fix before it's believable.","tokens_in":34248,"tokens_out":3521,"would_cite":true,"duration_ms":32339,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FactorDrive claims that scene-adaptive chain-of-thought plus search-guided RL post-training — QS-GRPO — makes a VLM driving planner state of the art: 0.30 m average $L_2$, 0.08 % collision, 91.0 PDMS on NAVSIM.","keywords":["end-to-end autonomous driving","vision-language model","chain-of-thought reasoning","planning-critical factors","group relative policy optimization","Monte Carlo tree search","trajectory planning","NAVSIM benchmark"],"falsifier":"Re-run the nuScenes evaluation with the navigation command produced by a map-based routing engine over the recorded route — or with the command removed entirely — and check whether the 0.30 m average $L_2$ and 0.08 % average collision rate survive. If they degrade to baseline levels, the open-loop state-of-the-art claim rests on the expert-derived command rather than on adaptive reasoning or search-guided post-training; a complementary check is to run the post-trained policy in a reactive closed-loop simulator and see whether its safety gains persist when surrounding agents respond to the ego plan.","tokens_in":2551,"feed_emoji":"🚗","tokens_out":3111,"duration_ms":152532,"temperature":0.7,"pith_summary":"FactorDrive attempts to show that an end-to-end driving planner reasons best when it focuses on the few planning-critical factors in each scene — lane geometry, drivable space, traffic controls, critical interacting agents, and ego motion — and when the reasoning path itself is optimized by search. The paper builds PCF-CoT, a chain-of-thought dataset in which reasoning paths vary in composition and depth per scene instead of following a fixed template, and QS-GRPO, a post-training method that uses Monte Carlo tree search with trajectory-level rewards to discover reasoning paths that yield better trajectories, then reinforces them through group-relative policy optimization. Evaluated open-loop on nuScenes the model reports 0.30 m average $L_2$ error and 0.08 % average collision rate; on NAVSIM it reports a 91.0 PDM score, the best in its comparison tables. If the results hold, they indicate that making reasoning scene-adaptive and optimizing it explicitly is a productive direction for VLM-based planning, within the scope the paper itself sets: these are open-loop and non-reactive benchmarks, not proof of safe on-road driving.","feed_headline":"Search-guided reasoning drives planner to 0.30 m trajectory error","feed_subtitle":"FactorDrive sizes its chain-of-thought to each scene and uses MCTS-guided RL to outperform planners on both benchmarks.","key_machinery":"The load-bearing mechanism is the pair formed by the planning-critical-factor reasoning plan and the QS-GRPO search loop. In PCF-CoT every reasoning path opens with a 'Global scene assessment and reasoning plan' that identifies the active PCFs — road geometry, drivable corridors, traffic controls, critical interacting agents, and ego motion — and then decides which reasoning units to activate: Scene spatial constraint reasoning, Key object prediction and interaction reasoning, Counterfactual reasoning (activated only when at least two reasonable actions survive filtering), and Ego motion and trajectory generation reasoning. The path always ends with the trajectory-generation step, so reasoning composition and depth adapt to the scene. In QS-GRPO, Monte Carlo tree search over partial reasoning states uses PUCT selection with old-policy priors and the trajectory-level reward $r(\\tau,\\xi) = \\lambda_{\\mathrm{fmt}} R_{\\mathrm{fmt}} + \\lambda_{\\mathrm{traj}} R_{\\mathrm{traj}}$ to explore and select complete responses; the highest-reward cached response per hard sample is retained in a buffer and re-used, and group construction keeps the best response while adding high-confidence rollouts, so that high-confidence but low-reward paths receive negative group-relative advantage under the GRPO objective with clipping and KL regularization.","core_discovery":"The central claim is that planning-critical-factor-driven adaptive reasoning and quality-guided search-based post-training together produce state-of-the-art end-to-end trajectory planning in a vision-language driving model. FactorDrive first instruction-tunes the Qwen3-VL-8B base model on roughly 1.09 million driving-domain question–answer pairs, then fine-tunes it on PCF-CoT, a chain-of-thought dataset in which each reasoning path is organized around the factors that actually constrain the maneuver, with the composition and depth of reasoning steps selected per scene (paths range from two to five steps). QS-GRPO then post-trains only on hard samples: Monte Carlo tree search over reasoning steps, guided by a trajectory-level reward (format correctness plus an exponential $L_2$/FDE reward on nuScenes, or $L_2$ plus PDMS on NAVSIM), discovers higher-reward reasoning paths; a per-sample buffer caches the best response; and a confidence-aware group mixes the best response with high-confidence rollouts under the clipped GRPO objective with group-relative advantages. The paper reports the lowest average $L_2$ error on nuScenes and the highest PDMS on NAVSIM among the methods compared — 0.30 m and 0.08 % on nuScenes, 91.0 on NAVSIM — with ablations attributing a 2.6-point PDMS gain to QS-GRPO post-training and a 16.7 % relative reduction in average collision rate to PCF-CoT reasoning.","pith_inferences":["Because the nuScenes navigation command is synthesized from roughly 20 m of the expert future trajectory (labeled by a 2 m lateral-displacement threshold), the open-loop comparison implicitly assumes this coarse route cue does not leak the ground-truth turn direction. A map-derived routing command independent of the expert path — or no command at all — would test whether the reported 0.30 m averag","PCF-CoT is annotated by an LLM that is shown the ground-truth trajectory as a teacher-side reference, so the generated reasoning may partly rationalize the expert path rather than cause it. An inference-time intervention that deletes or corrupts individual reasoning units while keeping the final trajectory head would reveal how much of the planning quality actually flows through the reasoning toke","The published path statistics show counterfactual reasoning is activated in only 0.6 % of NAVSIM samples and 1.1 % of nuScenes samples, with 96 % of NAVSIM paths running three or four steps, so the practical adaptivity is mostly a choice of whether to include spatial and interaction reasoning. A curated suite of genuinely ambiguous right-of-way scenes would stress whether the counterfactual unit —","Both benchmarks are non-reactive or open-loop, so neither tests interactive safety against agents that respond to the ego plan. The qualitative before/after QS-GRPO example shows the policy switching from proceeding to yielding for an oncoming sedan; a reactive closed-loop test would show whether that reasoning change persists when the other agent actually reacts."],"forward_implications":["Scene-adaptive reasoning paths (two to five steps, with counterfactual reasoning invoked only when a real trade-off exists) can replace fixed-template and fast–slow reasoning as the default CoT structure for driving VLMs.","Explicitly searching over reasoning paths with trajectory-level rewards finds better plans than direct-rollout GRPO: QS-GRPO beats standard GRPO by 0.3 PDMS points on NAVSIM (90.7 → 91.0).","Grounding CoT in spatial-physical evidence reduces collisions even when it does not reduce $L_2$ error: PCF-CoT cuts average collision rate from 0.12 % to 0.10 % while leaving 1 s and 2 s $L_2$ unchanged.","The recipe transfers across benchmark styles: the same two-stage SFT plus QS-GRPO yields the top average $L_2$ on open-loop nuScenes and the top PDMS on closed-loop-oriented NAVSIM among the compared methods, with camera input only on NAVSIM.","Hard-sample selection concentrates RL compute where it pays: fewer than 5,000 NAVSIM samples (4.8 % of the training split) account for the 2.6-point PDMS gain from post-training."],"supporting_citations":[{"why":"Supplies the Qwen3-VL-8B base VLM that FactorDrive initializes from and fine-tunes in all training stages.","marker":"[1]"},{"why":"The Seed-2.0-Pro annotation model that generates the structured PCF annotations and PCF-CoT reasoning paths.","marker":"[2]"},{"why":"The nuScenes dataset that provides the open-loop planning benchmark, the PCF-CoT nuScenes subset, and the evaluation frames.","marker":"[3]"},{"why":"The NAVSIM non-reactive simulation benchmark and data source for the NAVSIM-PCF training subset and the PDMS evaluation.","marker":"[7]"},{"why":"Supplies the GRPO group-relative policy optimization objective that QS-GRPO builds on.","marker":"[8]"},{"why":"Defines the nuScenes open-loop planning protocol ($L_2$ error and collision rate) used for evaluation.","marker":"[9]"},{"why":"Supplies the MCTS search algorithm and PUCT selection criterion used by QS-GRPO to explore reasoning paths.","marker":"[33]"},{"why":"The search-augmented RL approach that motivates using tree-based search to find higher-reward reasoning responses.","marker":"[42]"},{"why":"AutoDrive-P3 is the strongest baseline FactorDrive must beat on both benchmarks (0.06 % collision on nuScenes, 90.6 PDMS on NAVSIM).","marker":"[45]"},{"why":"AutoVLA supplies the nuScenes navigation-command construction and the PDMS reward component on NAVSIM, and serves as the fast–slow reasoning baseline.","marker":"[49]"}],"fun_headline_variants":["MCTS-search optimizes reasoning paths for SOTA planning","Adaptive CoT per scene yields 0.30 m planner error","FactorDrive: search-guided reasoning trims L2 to 0.30 m","QS-GRPO boosts PDMS via quality-guided reasoning search"],"cache_read_input_tokens":36224,"weakest_assumption_plain":"The premise the open-loop nuScenes result rests on is that the route-level navigation command — constructed by looking roughly 20 m along the expert's future trajectory and labeling it Go Straight, Turn Left, or Turn Right by lateral displacement — is a legitimate route input that does not leak the ground-truth turn direction to the planner; if it leaks, FactorDrive's reported 0.30 m average $L_2$ is partly earned from information its comparison baselines never received.","fun_headline_variants_meta":{"raw":{"variants":["MCTS-search optimizes reasoning paths for SOTA planning","Adaptive CoT per scene yields 0.30 m planner error","FactorDrive: search-guided reasoning trims L2 to 0.30 m","QS-GRPO boosts PDMS via quality-guided reasoning search"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000386,"raw_usage":{"total_tokens":2120,"prompt_tokens":1106,"completion_tokens":1014,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":722,"completion_tokens_details":{"reasoning_tokens":938}},"tokens_in":722,"tokens_out":1014,"duration_ms":9733,"temperature":1.0,"reasoning_tokens":938,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:29:09.302242+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the nuScenes evaluation with the navigation command produced by a map-based routing engine over the recorded route — or with the command removed entirely — and check whether the 0.30 m average $L_2$ and 0.08 % average collision rate survive. If they degrade to baseline levels, the open-loop state-of-the-art claim rests on the expert-derived command rather than on adaptive reasoning or search-guided post-training; a complementary check is to run the post-trained policy in a reactive closed-loop simulator and see whether its safety gains persist when surrounding agents respond to the ego plan.","supporting_citations":[{"cited_title":"Seed2.0 Model Card: Towards in- telligence frontier for real-world complexity.https:// seed.bytedance.com/en/seed2, 2026","cited_arxiv_id":null,"evidence_quote":"The Seed-2.0-Pro annotation model that generates the structured PCF annotations and PCF-CoT reasoning paths."},{"cited_title":"nuscenes: A multi- modal dataset for autonomous driving","cited_arxiv_id":null,"evidence_quote":"The nuScenes dataset that provides the open-loop planning benchmark, the PCF-CoT nuScenes subset, and the evaluation frames."},{"cited_title":"Navsim: Data-driven non-reactive autonomous vehicle simulation and benchmarking.Advances in Neural Information Processing Systems, 37:28706–28719, 2024","cited_arxiv_id":null,"evidence_quote":"The NAVSIM non-reactive simulation benchmark and data source for the NAVSIM-PCF training subset and the PDMS evaluation."},{"cited_title":"Deepseek-r1 incentivizes reasoning in llms through reinforcement learning.Nature, 645(8081):633– 638, 2025","cited_arxiv_id":null,"evidence_quote":"Supplies the GRPO group-relative policy optimization objective that QS-GRPO builds on."},{"cited_title":"St-p3: End-to-end vision-based au- tonomous driving via spatial-temporal feature learning","cited_arxiv_id":null,"evidence_quote":"Defines the nuScenes open-loop planning protocol ($L_2$ error and collision rate) used for evaluation."},{"cited_title":"Mastering the game of go without human knowledge.nature, 550(7676): 354–359, 2017","cited_arxiv_id":null,"evidence_quote":"Supplies the MCTS search algorithm and PUCT selection criterion used by QS-GRPO to explore reasoning paths."},{"cited_title":"Deepsearch: Overcome the bottleneck of reinforcement learning with verifiable rewards via tree-based search","cited_arxiv_id":null,"evidence_quote":"The search-augmented RL approach that motivates using tree-based search to find higher-reward reasoning responses."},{"cited_title":"$autodrive\\text{-}pˆ3$: Unified chain of perception–prediction–planning thought via rein- forcement fine-tuning","cited_arxiv_id":null,"evidence_quote":"AutoDrive-P3 is the strongest baseline FactorDrive must beat on both benchmarks (0.06 % collision on nuScenes, 90.6 PDMS on NAVSIM)."},{"cited_title":"Autovla: A vision- language-action model for end-to-end autonomous driv- ing with adaptive reasoning and reinforcement fine-tuning","cited_arxiv_id":null,"evidence_quote":"AutoVLA supplies the nuScenes navigation-command construction and the PDMS reward component on NAVSIM, and serves as the fast–slow reasoning baseline."}],"review_version":1}