{"id":"b78d51e2-49e3-412f-a2bd-389a6fe56f5d","arxiv_id":"2607.06328","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":5,"one_line_summary":"Sparse autoencoders decompose the latent space of end-to-end autonomous driving models into interpretable concepts whose causal influence on trajectory scoring can be measured and intervened upon to improve driving performance.","lead":"The paper applies sparse autoencoder (SAE) dictionary learning to end-to-end autonomous driving models to decompose latent representations into interpretable concepts, then uses circuit analysis to link those concepts to trajectory scoring heads. It shows that ablating specific neurons identified as harmful improves downstream driving metrics without retraining.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Neuron selection and performance evaluation occur on the same NAVSIM split with no held-out validation, making the 0.097 EPDMS gain indistinguishable from test-set overfitting.","rationale":"The reader's weakest_assumption correctly identifies the single most load-bearing concern: the neuron selection and evaluation use the same NAVSIM data with no held-out validation or closed-loop testing. I agree with this assessment. The concern is well-founded for several reinforcing reasons: (1) The search space over 256 neurons × multiple scoring heads is large enough that finding a 3-neuron combination that improves the score on any fixed dataset is not surprising. (2) The magnitude of improvement (surpassing even the original SAE-free model) is suspicious given the minimal intervention. (3) The conservative shift (ego progress drops, compliance scores rise) is a known failure mode of open-loop evaluation. (4) The paper itself acknowledges the correlation-vs-causation problem but does not address it for the specific intervention. The reader's verdict of CONDITIONAL with MODERATE confidence is appropriate. The paper makes genuine contributions to the interpretability methodology (SAE integration, circuit analysis, concept attribution), and the framework is well-structured. But the central intervention claim — that interpretability enables targeted performance improvement — rests entirely on an unvalidated result. The paper would need either cross-validated neuron selection or closed-loop evaluation to elevate the claim from demonstration to validated method. I recommend UNCHANGED because the reader already identified this exact concern and calibrated the verdict accordingly.","tokens_in":16519,"tokens_out":1809,"duration_ms":164891,"concrete_test":"Split the NAVSIM evaluation set into two disjoint halves (e.g., by scenario log). Perform circuit analysis and neuron selection on half A only, then evaluate the ablation of neurons {177, 59, 71} on half B. If the EPDMS improvement on held-out half B is within ±0.02 of the 0.097 gain reported on the full set, the intervention generalizes. If the gain drops substantially or disappears, it is a dataset-specific artifact. Additionally, run the ablated model on the NAVSIM closed-loop pseudo-simulation benchmark (Cao et al. [3]) to verify the improvement persists under reactive evaluation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader correctly identifies the core issue. In §V-F, neurons {177, 59, 71} are selected because they have 'negative influence across multiple scoring heads' as determined by circuit analysis (attribution patching, activation patching, ACDC pruning — all computed over the NAVSIM evaluation data). The EPDMS improvement from ablation is then measured on that same data. This is a textbook selection-evaluation circularity: with 256 active SAE neurons and multiple scoring heads, the search space for beneficial ablation combinations is enormous, and selecting 3 neurons that improve the score on the same benchmark they were identified on provides no evidence of generalization. The concern is amplified by two observations. First, the SAE injection itself degrades EPDMS from 0.524 to 0.496 despite near-perfect reconstruction (cosine similarity 0.9975, explained variance 0.9951), yet ablating just 3 of 256 neurons not only recovers this loss but surpasses the original model by 0.069 — a suspiciously large effect for such a minimal intervention. Second, the paper claims 'improved closed-loop behavior' (§V-F) but evaluates only on NAVSIM, an open-loop benchmark where trajectory proposals are scored against ground truth without environmental reactivity. Open-loop metrics are known to be gameable: a model that simply selects more conservative trajectories can improve compliance scores without genuinely better driving. The ego progress drop of -0.100 (Table II) is consistent with this conservative-shift artifact rather than a genuine correction of erroneous behavior. No code or checkpoints are released, preventing independent verification. The paper's own §IV-E acknowledges that 'patterns learned from the data likely induce strong cross-associations between features and prediction heads that are driven by correlation rather than true causality,' yet the intervention result is presented without addressing this caveat for the specific neurons chosen.","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"This paper introduces a concept-based interpretability framework for end-to-end autonomous driving models. The authors integrate Sparse Autoencoders (SAEs) into the latent space of state-of-the-art models (GTRS, iPAD) to decompose driving behavior into interpretable concepts. They propose a stepwise methodology: selecting a latent space, training SAEs, assigning semantic meaning via activation maximization and attribution, linking concepts to trajectory scoring heads via circuit analysis, and performing targeted interventions by zeroing specific SAE neurons. The central intervention result claims that ablating three SAE neurons ({177, 59, 71}) improves the EPDMS score from 0.496 (SAE-injected baseline) to 0.593, surpassing the original model's 0.524.","tokens_in":16742,"tokens_out":1331,"duration_ms":208712,"significance":"The paper addresses an important gap in end-to-end autonomous driving: the lack of concept-level latent space interpretability. The systematic comparison of SAE variants (TopK, Matryoshka, Archetypal) in Table I is a useful contribution, and the application of circuit discovery to connect SAE features to PDM subscores is methodologically sound and well-motivated. The observation that the backward-facing camera in iPAD appears unused is a concrete, falsifiable finding. The framework is clearly structured and the SAE training and circuit analysis methodology is reasonably specified.","major_comments":[{"comment":"§V-F, Table II: The central intervention result has a selection-evaluation circularity. Neurons {177, 59, 71} are selected post hoc based on circuit analysis (attribution patching, activation patching, ACDC pruning) computed over the NAVSIM evaluation data, and the EPDMS improvement from ablation is then measured on that same data. With 256 active SAE neurons and multiple scoring heads, the search space for beneficial ablation combinations is large, and selecting 3 neurons that improve the score on the same benchmark used to identify them provides no evidence of generalization. The paper needs either a held-out validation split for neuron selection or cross-validation to demonstrate that the 0.097 EPDMS gain is not a dataset-specific artifact.","section":null},{"comment":"§V-F: The paper claims 'improved closed-loop behavior' but evaluates only on NAVSIM, an open-loop benchmark where trajectory proposals are scored against ground truth without environmental reactivity. The ego progress drop of -0.100 (Table II) is consistent with a model that selects more conservative trajectories to game compliance scores, which is a known failure mode of open-loop metrics. The claim of improved closed-loop behavior is not supported by the evaluation; either closed-loop simulation results should be provided, or the claim should be retracted and the result framed strictly as an open-loop improvement.","section":null},{"comment":"§V-F, Table II: The magnitude of the intervention effect is suspiciously large. The SAE injection degrades EPDMS from 0.524 to 0.496 despite near-perfect reconstruction (cosine similarity 0.9975, explained variance 0.9951, Table I), yet ablating just 3 of 256 neurons not only recovers this loss but surpasses the original model by 0.069. The paper should explain why such a minimal intervention produces such a large effect, and whether the 0.496 baseline reflects a systematic degradation from SAE injection that the ablation happens to correct, rather than a genuine correction of erroneous model behavior.","section":null}],"minor_comments":[{"comment":"§III-B, Eq. (1): The ego correlation score is defined as the max over neurons of |r_{i,j}|, but the notation uses index 'n' in r_{n,j} while the max is over 'i'. This should be made consistent.","section":null},{"comment":"§V-A: The text states 'SAEs in language models typically require a high expansion factor, we experience substantially less required neurons for vision-based models.' This claim would benefit from a citation or quantitative comparison to support it.","section":null},{"comment":"Table I: The 'TopK*256' row uses an asterisk that is not explained in the table caption. Presumably this marks the selected SAE configuration, but this should be stated explicitly.","section":null},{"comment":"§V-F: The paper states neurons 59 and 71 'show a close correlation to neuron 177.' It would strengthen the intervention claim to report the actual correlation values and clarify whether 'correlation' refers to activation correlation or influence correlation.","section":null},{"comment":"§V-E: The remark on model construction and interpretability is insightful but reads as a tangent. Consider integrating this observation (that penalty heads predict absence of violation, complicating activation-based attribution) into the methodology section where it can inform the interpretation framework.","section":null},{"comment":"Fig. 2: The caption references 'top' and 'bottom' panels but the figure description could be more precise about what the clustering visualization (top) actually shows (e.g., t-SNE/UMAP projection, dendrogram, etc.).","section":null},{"comment":"§IV-B, Eq. (8): The reanimation loss term is described as encouraging inactive neurons, but the mechanism by which pushing pre-activations of dead neurons leads to reactivation could be stated more precisely.","section":null}],"recommendation":"major_revision","confidential_remarks":"The reader's report and stress-test note correctly identify the selection-evaluation circularity as the primary concern. This is a load-bearing issue: the paper's fourth contribution ('improvements in downstream driving performance without retraining') rests entirely on Table II, which is currently indistinguishable from test-set overfitting. The SAE training and circuit analysis methodology (contributions 1-3) is sound and potentially publishable on its own, but the intervention claim needs either a held-out split or a substantially revised framing. The authors should be given the opportunity to address this, as the fix (splitting data for selection vs. evaluation) is within the manuscript's scope. The closed-loop claim should simply be removed if no closed-loop evaluation is available."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive review. The referee raises three major concerns about the intervention results in Section V-F: (1) selection-evaluation circularity due to neuron selection and evaluation on the same NAVSIM data, (2) an unsupported claim of improved closed-loop behavior when only open-loop evaluation is conducted, and (3) the suspiciously large magnitude of the intervention effect relative to the minimal nature of the ablation. We agree with the substance of all three comments and will revise the manuscript accordingly. Specifically, we will (a) add a held-out validation split for neuron selection and report cross-validated results, (b) retract the 'improved closed-loop behavior' claim and reframe the result strictly as an open-loop improvement, and (c) add analysis explaining the mechanism by which SAE injection introduces systematic degradation that the ablation corrects. No standing objections remain.","responses":[{"response":"The referee is correct that the current evaluation has a selection-evaluation circularity: the circuit analysis used to identify neurons {177, 59, 71} was computed over the NAVSIM evaluation data, and the ablation effect was measured on the same data. We acknowledge that this does not constitute evidence of generalization. We will address this by splitting the NAVSIM evaluation set into a validation split (used for neuron selection via circuit analysis) and a held-out test split (used solely for reporting the final ablation result). We will additionally report the EPDMS change on the held-out split. If the effect does not generalize, we will report this honestly and reframe the contribution accordingly. The circuit analysis methodology itself (attribution patching, activation patching, ACDC pruning) is not changed; only the data split on which selection versus evaluation occurs is corrected.","revision_made":"yes","referee_comment":"§V-F, Table II: The central intervention result has a selection-evaluation circularity. Neurons {177, 59, 71} are selected post hoc based on circuit analysis (attribution patching, activation patching, ACDC pruning) computed over the NAVSIM evaluation data, and the EPDMS improvement from ablation is then measured on that same data. With 256 active SAE neurons and multiple scoring heads, the search space for beneficial ablation combinations is large, and selecting 3 neurons that improve the score on the same benchmark used to identify them provides no evidence of generalization. The paper needs either a held-out validation split for neuron selection or cross-validation to demonstrate that the 0.097 EPDMS gain is not a dataset-specific artifact."},{"response":"The referee is correct. NAVSIM is an open-loop, non-reactive benchmark, and our use of the phrase 'improved closed-loop behavior' in Section V-F is not supported by the evaluation we present. We will retract this claim throughout the manuscript and reframe the result strictly as an open-loop EPDMS improvement. We also agree that the ego progress drop of -0.100 is consistent with the known failure mode of open-loop metrics rewarding overly conservative trajectory selection. We will add an explicit discussion of this limitation, noting that the observed trade-off between compliance scores and ego progress may reflect metric gaming rather than genuine behavioral improvement, and that closed-loop evaluation would be needed to distinguish these explanations. Providing closed-loop simulation results is beyond the scope of what we can add in revision, so we will scope our claims accordingly.","revision_made":"yes","referee_comment":"§V-F: The paper claims 'improved closed-loop behavior' but evaluates only on NAVSIM, an open-loop benchmark where trajectory proposals are scored against ground truth without environmental reactivity. The ego progress drop of -0.100 (Table II) is consistent with a model that selects more conservative trajectories to game compliance scores, which is a known failure mode of open-loop metrics. The claim of improved closed-loop behavior is not supported by the evaluation; either closed-loop simulation results should be provided, or the claim should be retracted and the result framed strictly as an open-loop improvement."},{"response":"The referee raises a valid concern about the plausibility of the effect magnitude. We will add analysis addressing this. Our working hypothesis is that the near-perfect reconstruction metrics (cosine similarity 0.9975, explained variance 0.9951) are aggregate measures that can mask localized, high-impact distortions in specific regions of the latent space that are disproportionately important for particular scoring heads. The SAE injection may introduce small but systematic perturbations in the representations feeding the compliance-related heads (drivable area, driving direction), which are particularly sensitive to certain feature directions. The three ablated neurons may sit at a critical junction where the SAE's reconstruction error concentrates, so their ablation removes a systematic bias introduced by the SAE rather than correcting an original model error. This would mean the 0.496 baseline partly reflects SAE-injection degradation, and the ablation recovers and surpasses the original by removing both the SAE-induced distortion and a pre-existing spurious correlation. We will add this analysis to the manuscript and be explicit that the result may reflect correction of SAE-injection artifacts rather than purely genuine correction of erroneous model behavior. We will also report per-head reconstruction quality to test whether the degradation is concentrated in specific heads.","revision_made":"partial","referee_comment":"§V-F, Table II: The magnitude of the intervention effect is suspiciously large. The SAE injection degrades EPDMS from 0.524 to 0.496 despite near-perfect reconstruction (cosine similarity 0.9975, explained variance 0.9951, Table I), yet ablating just 3 of 256 neurons not only recovers this loss but surpasses the original model by 0.069. The paper should explain why such a minimal intervention produces such a large effect, and whether the 0.496 baseline reflects a systematic degradation from SAE injection that the ablation happens to correct, rather than a genuine correction of erroneous model behavior."}],"tokens_in":16514,"tokens_out":1284,"duration_ms":231966,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"Bottom line: this is the first paper to apply SAE-based concept decomposition with circuit analysis to end-to-end autonomous driving models, and the framework itself is well-constructed. The central intervention result — that zeroing three SAE neurons improves EPDMS from 0.496 to 0.593 — is the part that needs scrutiny, and the reader's concern about it is correct and important enough to require a revision before publication.","headline":"First application of SAE-based concept decomposition to end-to-end autonomous driving models; the intervention result is promising but unvalidated due to selection-evaluation circularity.","tokens_in":17371,"tokens_out":162,"would_cite":false,"duration_ms":119791,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Zeroing three neurons makes a self-driving model safer","keywords":["sparse autoencoder","end-to-end autonomous driving","interpretability","circuit analysis","concept-level intervention","trajectory scoring","model steering","mechanistic interpretability"],"falsifier":"If the same three-neuron ablation fails to produce comparable improvements on a different driving benchmark or in closed-loop simulation, the gain would be a dataset-specific artifact rather than a genuine correction of model behavior.","tokens_in":16593,"feed_emoji":"","tokens_out":779,"duration_ms":198225,"temperature":0.7,"pith_summary":"This paper claims that the internal decision-making of end-to-end autonomous driving models can be decomposed into human-interpretable concepts using Sparse Autoencoders, and that targeted suppression of specific concept-level neurons identified through circuit analysis can correct erroneous driving behavior without retraining. The authors integrate an SAE into the latent space of the GTRS driving model, decompose its trajectory-scoring logic into semantically meaningful feature directions (such as 'car ahead', 'intersections', 'lane markings'), and trace how these features causally influence individual prediction heads for collision avoidance, drivable-area compliance, traffic-light compliance, and ego progress. They then identify three SAE neurons (numbered 177, 59, and 71) that exert negative influence across multiple scoring heads, zero their activations at inference time, and report that the model's driving score (EPDMS) rises from 0.496 to 0.593, surpassing the original model's 0.524. The central mechanism is the SAE-mediated decomposition of an opaque latent space into monosemantic, intervenable concept directions that can be selectively silenced to remove learned spurious correlations and shift the model toward more conservative, compliant trajectory choices.","feed_headline":"Zeroing three neurons makes a self-driving model safer","feed_subtitle":"Sparse autoencoders decompose a driving model's latent space into interpretable concepts, and silencing just three of them improves driving ","key_machinery":"SAE","core_discovery":"The paper's central object is the SAE-injected latent space sitting directly before a driving model's trajectory-scoring module. By training a Sparse Autoencoder on this representation, the authors decompose the model's entangled internal features into sparse, monosemantic concept directions. Circuit analysis then maps these concepts to individual scoring heads, revealing which latent features drive which aspects of the driving decision. The core discovery is that zeroing just three of these concept-level neurons — identified as having broad negative influence across multiple heads — produces a measurable improvement in driving performance, shifting the model from risky, progress-maximizing,","pith_inferences":["The three-neuron ablation is selected and evaluated on the same benchmark (NAVSIM), so the reported gain could partly reflect overfitting to that dataset's specific scenario distribution rather than a generalizable correction of model behavior.","The finding that the backward-facing camera has no measurable influence on predictions suggests the model may be ignoring potentially useful input modalities, raising questions about whether other cameras or sensors are similarly underutilized.","The observation that penalty heads (e.g., traffic-light compliance) score high by default when no violation exists implies that attribution-based interpretability methods may systematically miss the most safety-critical failure modes, since those occur when the model fails to detect a hazard rather than when it actively processes one.","If the concept-level intervention approach generalizes, it could enable rapid post-deployment safety patches for autonomous driving models — correcting discovered failure modes through neuron-level edits rather than full retraining cycles."],"forward_implications":[],"fun_headline_variants":["Zeroing three latent directions improves autonomous driving","Suppressing three sparse concepts fixes risky driving models","Concept-level interventions improve end-to-end driving performance","Sparse autoencoders isolate and correct risky driving behaviors","Editing three SAE features improves autonomous driving safety"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The three neurons are selected based on circuit analysis performed on the same NAVSIM evaluation set used to measure the improvement, with no held-out test set or closed-loop simulation to confirm that zeroing them generalizes beyond that benchmark.","fun_headline_variants_meta":{"raw":{"variants":["Zeroing three latent directions improves autonomous driving","Suppressing three sparse concepts fixes risky driving models","Concept-level interventions improve end-to-end driving performance","Sparse autoencoders isolate and correct risky driving behaviors","Editing three SAE features improves autonomous driving safety"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":923,"prompt_tokens":458,"completion_tokens":465,"prompt_tokens_details":null},"tokens_in":458,"tokens_out":465,"duration_ms":36315,"temperature":1.0,"reasoning_tokens":461,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-08T09:30:59.205504+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If the same three-neuron ablation fails to produce comparable improvements on a different driving benchmark or in closed-loop simulation, the gain would be a dataset-specific artifact rather than a genuine correction of model behavior.","supporting_citations":[],"review_version":1}