{"id":"5cc2c025-60df-4280-8ffe-6fa1055a328f","arxiv_id":"2605.21061","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"By adding future visual state prediction and a dedicated inverse kinematics diffusion network that uses only visual boundary conditions, a 0.5B driving VLA recovers visual grounding and matches 7-8B models on NAVSIM-v2 and nuScenes.","lead":"The paper redesigns driving vision-language-action models by treating trajectory prediction as an inverse kinematics problem that needs both current and future visual states as inputs. This forces the model to use visual features instead of shortcuts from ego status or text commands, letting a small 0.5B model match much larger ones on driving benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Predicted future visual state may be generated via text/ego shortcuts, so the IK network receives ungrounded inputs and the structural fix remains incomplete","rationale":"The reader’s weakest assumption is precisely the load-bearing point; the abstract supplies no additional evidence that would refute an upstream shortcut in the visual-prediction objective, so the CONDITIONAL verdict with LOW confidence is left unchanged.","tokens_in":1749,"tokens_out":316,"duration_ms":37969,"concrete_test":"Ablate ego-status and text inputs to the LLM exclusively during the future-visual-prediction training phase (keep them for the main trajectory head if needed), then re-evaluate closed-loop NAVSIM-v2 L2 error and visual-attention metrics on dynamic turns; if performance drops more than 15 % relative to the reported model, the upstream shortcut concern is confirmed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that next-visual-state prediction plus a visual-only IK network eliminates shortcut reliance. The LLM still receives ego status and text when predicting the future visual scene; nothing in the formulation prevents it from satisfying the prediction loss primarily from those signals (e.g., a “turn left” command directly implies the corresponding future scene). The IK network then operates only on the resulting visual pair, but inherits any upstream non-visual bias. The abstract asserts that “extensive analysis” shows recovered visual exploitation, yet provides no control that isolates whether the visual-prediction head itself uses current visual tokens or simply routes through ego/text.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper argues that Driving VLAs under-exploit visual tokens because the task is structurally ill-posed: trajectory decoding receives only the current visual state and can therefore shortcut through ego status and text commands. The authors reframe the problem as inverse kinematics by adding (1) a next-visual-state prediction objective that supplies dense visual supervision to the LLM and (2) a separate Inverse Kinematics Network (cross-attention conditional diffusion model) that receives only the current and predicted future visual states and outputs the trajectory. They report that this change alone enables a 0.5 B model to reach closed-loop performance on NAVSIM-v2 and nuScenes comparable to 7–8 B VLAs, with the largest gains in dynamic maneuvers, and attribute the improvement to recovered visual grounding.","tokens_in":1917,"tokens_out":654,"duration_ms":32753,"significance":"If the central mechanism is verified, the work offers a lightweight structural remedy for visual grounding in driving VLAs that does not require scaling model size. The inverse-kinematics framing is conceptually clean and the reported efficiency gain (0.5 B vs. 7–8 B) would be practically relevant for deployment. The approach also supplies a concrete testbed for studying shortcut behavior in multimodal planners.","major_comments":[{"comment":"Abstract and analysis section: The claim that the next-visual-state prediction 'suppresses shortcut paths' and that 'extensive analysis' demonstrates recovered visual exploitation is load-bearing for the central thesis. However, because the LLM still receives ego status and text when predicting the future scene, it remains possible that the prediction head satisfies its loss primarily via non-visual signals; the downstream IK network would then inherit any such bias. A control that isolates whether the visual-prediction head actually conditions on current visual tokens (e.g., an ablation that masks visual inputs during prediction and measures degradation) is required to substantiate the structural fix.","section":"Abstract / analysis section"},{"comment":"Experiments section: The manuscript states that the 0.5 B model reaches performance 'comparable' to 7–8 B VLAs on both NAVSIM-v2 and nuScenes. To support this cross-scale claim, the results must be accompanied by full ablation tables, error analysis, and statistical comparisons (including variance across seeds or runs). Without these details the attribution of gains specifically to visual grounding versus other implementation choices cannot be verified.","section":"Experiments section"}],"minor_comments":[{"comment":"The description of the Inverse Kinematics Network should include the precise conditioning variables, diffusion schedule, and training losses so that the visual-only constraint can be reproduced exactly.","section":"Method section"},{"comment":"Figure 1 (or equivalent architecture diagram) would benefit from explicit arrows or annotations distinguishing the information flow into the LLM versus the IK network.","section":"Figure 1"}],"recommendation":"major_revision","confidential_remarks":"The work is a natural fit for a computer-vision or robotics venue that values efficient multimodal planning; the citation pattern appears appropriate and no obvious scope mismatch is evident."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive and detailed feedback. The comments correctly identify areas where additional evidence would strengthen the central claims regarding visual grounding recovery. We address each major point below and commit to revisions that directly respond to the concerns.","responses":[{"response":"We appreciate the referee's identification of this potential ambiguity. Our existing analysis shows that the visual prediction objective leads to higher attention weights on visual tokens and disproportionately larger gains in dynamic scenarios that require visual reasoning. Nevertheless, we agree that a direct isolation experiment is needed. In the revised manuscript we will add an ablation that masks current visual inputs to the prediction head (while retaining ego status and text) and report the resulting degradation in both next-state prediction accuracy and downstream closed-loop planning performance. This will substantiate that the head conditions on visual tokens rather than non-visual shortcuts.","revision_made":"yes","referee_comment":"[Abstract / analysis section] Abstract and analysis section: The claim that the next-visual-state prediction 'suppresses shortcut paths' and that 'extensive analysis' demonstrates recovered visual exploitation is load-bearing for the central thesis. However, because the LLM still receives ego status and text when predicting the future scene, it remains possible that the prediction head satisfies its loss primarily via non-visual signals; the downstream IK network would then inherit any such bias. A control that isolates whether the visual-prediction head actually conditions on current visual tokens (e.g., an ablation that masks visual inputs during prediction and measures degradation) is required to substantiate the structural fix."},{"response":"We concur that expanded experimental reporting is required to support the cross-scale comparison and to isolate the source of the gains. The revised experiments section will include: (i) complete ablation tables showing the incremental contribution of the next-visual-state prediction objective and the separate Inverse Kinematics Network, (ii) error analysis stratified by scenario type (e.g., turns, lane changes, and straight driving), and (iii) statistical comparisons reporting mean performance with standard deviation across at least three independent runs with different random seeds. These additions will allow readers to verify attribution to recovered visual grounding.","revision_made":"yes","referee_comment":"[Experiments section] Experiments section: The manuscript states that the 0.5 B model reaches performance 'comparable' to 7–8 B VLAs on both NAVSIM-v2 and nuScenes. To support this cross-scale claim, the results must be accompanied by full ablation tables, error analysis, and statistical comparisons (including variance across seeds or runs). Without these details the attribution of gains specifically to visual grounding versus other implementation choices cannot be verified."}],"tokens_in":1540,"tokens_out":560,"duration_ms":35640,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing to know is that the authors treat trajectory output as an inverse kinematics problem that needs both a current and a future visual state. They add a next-visual-state prediction loss on the LLM and then decode trajectories with a separate cross-attention conditional diffusion network that receives only those two visual states. This is the concrete change that lets their 0.5B model close the gap to much larger VLAs on the closed-loop benchmarks, with the biggest lift in turning maneuvers where visuals matter most. The idea is straightforward and the reported numbers are the part worth checking first. What the paper does well is name a structural reason why current VLAs can ignore their visual tokens and then supply a fix that is easy to implement on top of existing architectures. The separation of the IK network is a clear way to block direct ego and text shortcuts at decode time, and the emphasis on dynamic scenes shows they looked at where the gains actually appear. The soft spots are around the mechanism. The LLM still sees ego status and text when it predicts the future visual state, so it could meet the prediction loss without using current visual tokens at all. The IK network would then inherit that bias. The abstract mentions extensive analysis that recovers visual exploitation, but without seeing the specific ablations that isolate the visual-prediction head from text and ego routes, the causal story remains plausible rather than locked down. Adding the extra network also brings some complexity, even if the scale reduction is the headline benefit. This paper is for groups working on efficient driving VLAs who want a practical lever for visual grounding without just scaling parameters. A reader who cares about real-time deployment and benchmark numbers will find the results useful to test. I would send it to peer review because the structural proposal is well-defined and the performance claims are falsifiable on public benchmarks.","headline":"The inverse kinematics framing plus future visual prediction and a visual-only diffusion decoder is a clean structural change that lets a 0.5B model reach 7-8B performance on NAVSIM-v2 and nuScenes, but the shortcut-elimination story still needs tighter controls.","tokens_in":2375,"tokens_out":464,"would_cite":false,"duration_ms":42800,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[{"relation":"unclear","rs_module":"IndisputableMonolith/Cost/FunctionalEquation.lean","rs_theorem":"washburn_uniqueness_aczel","paper_passage":"trajectory recovery, when viewed through the lens of inverse kinematics, requires both a current and a future visual state as boundary conditions"},{"relation":"unclear","rs_module":"IndisputableMonolith/Foundation/BranchSelection.lean","rs_theorem":"branch_selection","paper_passage":"the IK Network ... takes only the current and future visual states as input ... suppressing reliance on ego status and textual shortcuts"}],"headline":"Driving VLA shortcut-suppression via next-state prediction + visual-only IK bottleneck is orthogonal to RS forcing","alignment":"orthogonal","rationale":"The paper's core machinery (bias-variance decomposition of trajectory loss, next-visual-state MSE objective, and cross-attention diffusion IK Network conditioned solely on (Vt, V̂t+Δ)) addresses shortcut learning in end-to-end VLAs. This is a standard ML architectural fix for ill-posed regression and has no structural overlap with RS primitives such as the reciprocal cost J(x) = ½(x + x⁻¹) − 1, φ-ladder, 8-tick periodicity, or the reality_from_one_distinction theorem. No RS module (Cost, Foundation, etc.) is echoed or contradicted.","tokens_in":62093,"confidence":"high","tokens_out":327,"duration_ms":12236,"cache_read_input_tokens":32896,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A 0.5B driving VLA matches 7B-8B models by treating trajectory prediction as inverse kinematics that requires future visual states.","keywords":["driving VLA","inverse kinematics","visual grounding","trajectory planning","NAVSIM","nuScenes","vision language models","future state prediction"],"falsifier":"Measure whether trajectory accuracy collapses and shortcut usage rises when the future visual state input is replaced by random noise or mismatched frames while keeping ego status and text available.","tokens_in":2656,"feed_emoji":"🚗","tokens_out":711,"duration_ms":27692,"temperature":0.7,"pith_summary":"The paper argues that existing driving vision-language-action models largely ignore visual tokens because the task of generating trajectories from only current visuals, ego status, and text commands is structurally ill-posed and invites shortcuts. It fixes this by adding a next visual state prediction objective that supplies dense visual supervision and by introducing a separate Inverse Kinematics Network that receives only the current and future visual states to decode the trajectory. With these changes alone, the 0.5B model recovers the ability to exploit visual features and reaches closed-loop trajectory planning performance on NAVSIM-v2 and nuScenes that is comparable to models more than ten times larger. The gains appear strongest in dynamic situations such as turning, indicating that proper visual boundary conditions matter for grounding.","feed_headline":"0.5B driving model matches 7B models by adding future visual states","feed_subtitle":"Inverse kinematics redesign supplies boundary conditions that force visual grounding and close the performance gap on NAVSIM-v2 and nuScenes","key_machinery":"The Inverse Kinematics Network, a cross-attention-based conditional diffusion model that decodes trajectories from current and future visual states alone while ignoring ego status and text.","core_discovery":"Trajectory recovery in driving VLAs requires both a current and a future visual state as boundary conditions; supplying only the current state encourages reliance on ego status and text. By adding a next visual state prediction objective and a cross-attention-based conditional diffusion Inverse Kinematics Network that takes solely the current and future visual states, the model suppresses shortcut paths during decoding. This redesign allows a 0.5B-scale model to recover visual grounding and match the trajectory planning performance of 7B-8B VLAs on NAVSIM-v2 and nuScenes, with the largest improvements in dynamic driving scenarios.","pith_inferences":["The same boundary-condition approach could be tested in other vision-language-action domains where future observations provide natural constraints.","Correcting task formulation this way may reduce the need for ever-larger models in grounded planning tasks.","Closed-loop tests on additional benchmarks or real vehicles would check whether the recovered visual grounding transfers beyond the reported datasets."],"forward_implications":["The model regains the ability to exploit visual features instead of bypassing them.","Trajectory planning performance becomes comparable to much larger models without increasing parameter count.","Gains concentrate in dynamic situations such as turning where visual grounding is most needed.","The structural change reduces dependence on ego status and textual commands during decoding."],"fun_headline_variants":["Future visual states close gap between 0.5B and 7B driving models","Inverse kinematics redesign grounds 0.5B VLA trajectories","Visual boundary conditions suppress ego status shortcuts in VLAs","Next visual prediction enables 0.5B model to match 7B VLAs"],"cache_read_input_tokens":64,"weakest_assumption_plain":"That providing a predicted future visual state together with a visual-only IK network is sufficient to remove the model's reliance on ego-status and text shortcuts.","fun_headline_variants_meta":{"raw":{"variants":["Future visual states close gap between 0.5B and 7B driving models","Inverse kinematics redesign grounds 0.5B VLA trajectories","Visual boundary conditions suppress ego status shortcuts in VLAs","Next visual prediction enables 0.5B model to match 7B VLAs"]},"model":"grok-4.3","cost_usd":0.00846,"raw_usage":{"total_tokens":3774,"prompt_tokens":726,"num_sources_used":0,"completion_tokens":70,"cost_in_usd_ticks":84603000,"prompt_tokens_details":{"text_tokens":726,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2978,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":726,"tokens_out":70,"duration_ms":23981,"temperature":1.0,"reasoning_tokens":2978,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-21T05:29:56.490489+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Measure whether trajectory accuracy collapses and shortcut usage rises when the future visual state input is replaced by random noise or mismatched frames while keeping ego status and text available.","supporting_citations":[],"review_version":1}