{"id":"80c52ebd-3ef3-4483-912f-a840f76926e0","arxiv_id":"2607.04681","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A learned critic of observation grounding and stepwise coherence, used as dense RL reward, improves VLA planner faithfulness and OOD hazard responsiveness while preserving competitive trajectory accuracy.","lead":"Robot vision-language-action models often write step-by-step reasoning that does not actually drive their actions. A learned critic that scores grounding and coherence, used as RL reward, raises faithfulness and long-tail hazard response in driving planners.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The 1.6x OOD claim rests on Gemini judging both the training reward and the hazard-response metric, so the headline gain may partly measure judge-model agreement rather than true causal faithfulness.","rationale":"The reader correctly flags the necessary-but-not-sufficient gap between pairwise consistency (Eq. 4) and mechanistic faithfulness (Eq. 3) as the weakest assumption. That gap is real, but it is already stated by the authors and is somewhat abstract. The more load-bearing, immediately testable concern is that the same Gemini family both trains the critic and scores the 1.6x OOD result that carries the paper’s strongest claim. This does not invalidate the in-distribution consistency gains (Tables 3–4 still show a clear edge over ADE and ADE-Swap), nor does it make the method worthless; it does mean the headline long-tail number needs an independent judge before it can be treated as solid evidence of improved causal faithfulness. The verdict therefore remains CONDITIONAL, with the same practical caveats the reader already listed (closed-loop, independent judge), only with the OOD metric now identified as the highest-priority place to apply that independent check.","tokens_in":32702,"tokens_out":621,"duration_ms":5394,"concrete_test":"Re-score the identical 66 original/augmented pairs and all model outputs with an independent judge family (e.g., Claude 4 or GPT-4.1) using the same hazard-response rubric, or with a blinded human panel of at least three raters. If the overall success rate for the Pinocchio planner falls below ~1.2x that of Alpamayo (or loses statistical significance), the 1.6x claim is judge-dependent and the central OOD evidence weakens.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper’s strongest empirical claim is that Pinocchio post-training yields a 1.6x overall causal hazard-response rate versus Alpamayo-1.5-10B on a 66-scene synthetic counterfactual set (Table 5, Section 6.3). That metric is produced by Gemini 3.1 Pro using a hazard-response prompt (Appendix 8.6.1). The same Gemini family also supplies the edge-level consistency labels that train Pinocchio (Section 6.1) and the consistency scores reported in Tables 3–4. Because the reward model and the OOD evaluator share the same judge family, the reported 1.6x improvement can be inflated by the policy learning to produce traces that Gemini prefers, rather than by a genuine increase in the causal mediation demanded by Eq. 3. The authors already note that behavioral consistency (Eq. 4) is necessary but not sufficient for mechanistic faithfulness (Section 5.2, Limitations); the Gemini-in-the-loop evaluation of the long-tail claim is the concrete place where that gap most threatens the headline number.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper distinguishes functional from faithful embodied Chain-of-Thought in VLAs and argues that action-level alignment is necessary but insufficient for faithfulness. Using autonomous driving as a testbed, it reports a human study on Alpamayo showing weak coupling between RL trajectory gains and reasoning quality (Section 4, Figure 2). It formalizes mechanistic faithfulness via a first-order Markov DAG (Eqs. 2–3) and a behavioral surrogate as the product of five pairwise semantic-consistency relations (Eq. 4, set C). A VLM critic, Pinocchio, is trained on Gemini-labeled edges and used as a dense GRPO reward (Eq. 7). On held-out DE/US data the method improves overall consistency by ~4% over a VLM-Judge baseline and ~18% over ADE-only RL while remaining competitive on ADE (Tables 3–4). On a 66-scene synthetic counterfactual hazard set it reports 1.6× overall causal hazard response relative to Alpamayo-1.5-10B (Table 5).","tokens_in":33037,"tokens_out":1655,"duration_ms":18077,"significance":"If the results hold under independent evaluation, the work would be a meaningful contribution to embodied reasoning: it elevates faithfulness from a post-hoc interpretability property to an explicit training objective, supplies a structured multi-edge consistency decomposition that is more granular than action–text alignment, and shows that optimizing that objective can improve long-tail responsiveness without large ADE collapse. Strengths include the human pilot of Gemini as judge (Table 1, κ within inter-annotator range on four of five edges), balanced-accuracy validation of Pinocchio (Table 2), controlled GRPO baselines spanning functionality and faithfulness (Section 6.2), and an explicit necessary-but-not-sufficient statement of the behavioral surrogate (Section 5.2, Limitations). The formalization and the dense multi-edge reward design are useful even if the OOD multiplier is revised.","major_comments":[{"comment":"Tables 3–5 and the OOD protocol (Section 6.3, Appendix 8.6.1) use Gemini 3.1 Pro both to label D_faith for critic training (Section 6.1) and to score final consistency and hazard response. Although the human pilot (Table 1) shows Gemini–majority agreement within human–human range on four of five edges, the 1.6× Overall claim in Table 5 is therefore partly a same-judge-family agreement metric. An independent human or held-out non-Gemini judge on at least the OOD set (and ideally a subsample of Tables 3–4) is needed before the headline long-tail claim can be treated as established.","section":"Section 6.3, Table 5; also Tables 3–4"},{"comment":"Proposition 1 and Section 5.2 correctly state that F(τ)=1 (Eq. 4) is necessary but not sufficient for mechanistic faithfulness (Eq. 3): a policy can emit fully consistent traces while computing a via a pathway that bypasses z. The GRPO objective (Eq. 7) maximizes critic log-probabilities, not interventional mediation. The paper’s central claim that faithful traces “contribute to more robust, generalizable” behavior therefore rests on a behavioral surrogate whose gap to Eq. 3 is untested. At minimum, report a simple intervention test (e.g., overwrite z_k and measure change in a, as sketched after Eq. 3) on a subset of rollouts for the final policy versus ADE and VLM-Judge; without this, the mechanistic language in the abstract and conclusion overreaches the evidence.","section":"Section 5.1–5.2, Eqs. (3)–(4), (7); Limitations"},{"comment":"The OOD benchmark has N=66 synthetic inpainted scenes, no ground-truth trajectories, and absolute Overall rates of only 7.6% (Ours) vs 4.5–4.6% (baselines) (Table 5). The 1.6× relative gain is therefore sensitive to small absolute counts and to Gemini’s hazard-response rubric. Report confidence intervals or bootstrap over scenes, and either enlarge the set or add a non-synthetic long-tail subset (e.g., real rare events from the cited WOD-E2E taxonomy) so the robustness claim does not rest solely on AI-generated imagery that none of the models saw in training.","section":"Section 6.3, Table 5"},{"comment":"On DE data, Ours ADE is 4.324 vs ADE baseline 4.169 (~3.7% worse) and vs ADE-Swap 4.196; on US, 3.858 vs 3.734–3.745 (Table 3–4). The abstract’s “maintaining competitive downstream task performance” is fair, but the paper should quantify whether faithfulness gains survive under a constrained multi-objective setting (e.g., Pareto front or λ-sweep on ADE vs r_faith) rather than a single fixed equal-weight composite. Without that, it remains unclear whether the method improves the functionality–faithfulness frontier or merely trades a small amount of ADE for Gemini-aligned consistency.","section":"Section 6.2.2, Tables 3–4, Eq. (7)"}],"minor_comments":[{"comment":"Figure 2’s win-rate visualization is hard to parse (overlapping markers, “pp from 50%”); a tabular breakdown of n and win rates per stratum would make the Section 4 claim easier to verify.","section":"Section 4, Figure 2"},{"comment":"Edge naming is inconsistent across the main text (E1–E5) and Appendix 8.3.2 (image→justification vs image→scene); align labels so readers can map Table 1–4 columns to C without ambiguity.","section":"Section 6.1, Appendix 8.3.2"},{"comment":"The abstract states “improves faithfulness by 4% and 18%”; specify that these are absolute percentage-point gains on overall consistency (e.g., 61.4 vs 57.5 and 43.4 on DE) so they are not read as relative improvements.","section":"Abstract; Tables 3–4"},{"comment":"Limitations correctly flag open-loop evaluation; a short closed-loop note or deferred experiment plan would strengthen the outlook without requiring full simulator results in this revision.","section":"Limitations"},{"comment":"Typographical inconsistencies appear (e.g., “SoT A”, “W ork”, “T races”, “V alidating”); a pass for spacing and OCR-like artifacts would improve polish.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The Gemini-in-the-loop issue is the main reason I chose major_revision rather than minor_revision: the 1.6× OOD number is the paper’s most marketable claim and is the place where judge circularity most threatens the narrative. If the authors can add independent human scoring on the OOD set and a small interventional mediation check, the contribution is solid for a robotics/ML venue. Scope fit for a serious robotics journal is good; novelty relative to Alpamayo-style reasoning–action alignment is real but incremental and should not be oversold as solving mechanistic faithfulness."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful part of this paper is not the abstract claim that “faithfulness matters.” It is a concrete five-edge consistency graph (scene, justification, meta-action, waypoints) plus a frozen VLM critic (Pinocchio) used as dense GRPO reward, with controlled baselines that actually isolate the signal.\n\nWhat is new: they separate functional from faithful reasoning, show via human annotation that Alpamayo-style RL can improve ADE while leaving CoT quality loosely coupled, then train a multi-edge critic and optimize against it. Tables 3–4 are the real result—overall consistency up ~4% vs VLM-Judge and ~18% vs ADE-only, with only modest ADE cost, on both US and DE held-out sets. The human pilot on Gemini labels (Table 1) and Pinocchio’s balanced accuracy (Table 2) are done carefully enough that the training loop is not pure circularity.\n\nSoft spots, in proportion. The authors already say pairwise consistency is necessary but not sufficient for mechanistic faithfulness; that is honest and correct. The stress-test concern about Gemini labeling the critic and scoring the 66-scene OOD set is real: the 1.6x overall hazard-response rate vs Alpamayo is the headline that most depends on the same judge family. I would not hang a paper on that number alone. Open-loop only is a limitation they flag. None of that erases the in-distribution consistency gains against four GRPO variants.\n\nMath and citation pattern look fine—standard DAG factorization, Lagrangian-style reward, proper engagement with LLM faithfulness and Alpamayo-style VLA work. Free parameters (λ weights, GRPO settings) are ordinary.\n\nThis is for people building reasoning VLAs in driving or other long-tail control. Worth a serious referee. I would bring it to reading group and cite the critic-as-dense-reward idea if I am working on embodied CoT. Send it out.","headline":"Solid training recipe for multi-edge faithfulness in driving VLAs; the 1.6x OOD number is real but Gemini-tied, so treat it as directional.","tokens_in":33623,"tokens_out":503,"would_cite":true,"duration_ms":5918,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Faithful reasoning traces—not just functional ones—make embodied VLAs more robust to rare hazards.","keywords":["embodied reasoning","vision-language-action models","faithfulness","chain-of-thought","autonomous driving","reinforcement learning","Pinocchio critic","counterfactual evaluation"],"falsifier":"A controlled intervention that overwrites an intermediate reasoning step (for example, changing the meta-action from lane-keeping to turn-right) and shows that the predicted waypoints still ignore the overwrite and follow the original observation, proving the trace is not load-bearing.","tokens_in":33617,"feed_emoji":"🤖","tokens_out":885,"duration_ms":9707,"temperature":0.7,"pith_summary":"Vision-language-action models often produce chain-of-thought explanations that look helpful but may not actually drive the actions they accompany. This paper separates functional reasoning (does the text help performance?) from faithful reasoning (does the text track the real decision process?). Using autonomous driving as the testbed, a human study shows that trajectory gains from reinforcement learning do not reliably improve reasoning quality. The authors then train a critic, Pinocchio, that scores whether each step of the reasoning is grounded in the scene and coherent with the next step and the final waypoints, and they use those scores as dense rewards. The resulting planner raises overall faithfulness while keeping competitive trajectory error, and on synthetic long-tail hazards it responds with causally aligned reasoning and motion 1.6 times more often than a strong baseline. The claim is that making reasoning load-bearing, rather than a post-hoc rationalization, is a practical route to more trustworthy robot policies.","feed_headline":"Faithful CoT lifts rare-hazard response 1.6× in driving VLAs","feed_subtitle":"A critic of stepwise consistency beats action-only alignment while keeping trajectory accuracy competitive.","key_machinery":"Pinocchio: a learned VLM critic that scores pairwise semantic consistency along the edges of a first-order Markov reasoning graph (observation→scene, scene→justification, justification→meta-action, meta-action→waypoints, justification→waypoints). Its log-probabilities supply a dense, decomposable faithfulness reward inside GRPO post-training.","core_discovery":"State-of-the-art alignment that only requires the final action to match the text is necessary but insufficient for faithfulness; intermediate steps can still be ungrounded or disconnected. Optimizing a learned critic of observation grounding and stepwise coherence during RL post-training improves overall faithfulness by 4% over a VLM-judge alignment baseline and 18% over trajectory-error-only RL on held-out driving data, while remaining competitive on average displacement error, and yields 1.6× better causal hazard response on a synthetic counterfactual set.","pith_inferences":["The same pairwise-consistency critic could be applied at inference time as a runtime filter that aborts or re-samples inconsistent rollouts.","If the necessity gap is large, closed-loop simulators that force reactive recovery will expose residual unfaithfulness that open-loop ADE misses.","Edge-level rewards may transfer to other VLA domains (manipulation, navigation) wherever CoT is structured into perception–justification–action stages."],"forward_implications":["Post-training rewards can and should supervise intermediate reasoning edges, not only final trajectory error or action–text agreement.","Policies trained this way remain competitive on ordinary benchmarks while becoming more responsive when rare hazards appear.","Faithfulness metrics that only check final action consistency will systematically under-detect ungrounded or disconnected CoT.","Synthetic counterfactual scene edits become a practical probe of whether stated justifications actually control behavior."],"fun_headline_variants":["Pinocchio critic lifts VLA CoT faithfulness 4% over alignment","Grounded stepwise CoT yields 1.6× better rare-hazard response","RL for CoT coherence beats action-only alignment in driving VLAs","Faithful intermediate reasoning improves OOD hazard handling 1.6×","Post-training on observation grounding boosts embodied CoT fidelity"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That the product of five pairwise consistency checks is a tight enough stand-in for true causal mediation, so that rewarding those checks actually forces the action to be generated through the reasoning rather than merely rationalized afterward.","fun_headline_variants_meta":{"raw":{"variants":["Pinocchio critic lifts VLA CoT faithfulness 4% over alignment","Grounded stepwise CoT yields 1.6× better rare-hazard response","RL for CoT coherence beats action-only alignment in driving VLAs","Faithful intermediate reasoning improves OOD hazard handling 1.6×","Post-training on observation grounding boosts embodied CoT fidelity"]},"model":"grok-4.5","effort":"low","cost_usd":0.00578,"raw_usage":{"total_tokens":1630,"prompt_tokens":904,"num_sources_used":0,"completion_tokens":98,"cost_in_usd_ticks":57800000,"prompt_tokens_details":{"text_tokens":904,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":628,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":904,"tokens_out":98,"duration_ms":4951,"temperature":1.0,"reasoning_tokens":628,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T15:15:35.621339+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"A controlled intervention that overwrites an intermediate reasoning step (for example, changing the meta-action from lane-keeping to turn-right) and shows that the predicted waypoints still ignore the overwrite and follow the original observation, proving the trace is not load-bearing.","supporting_citations":[],"review_version":1}