{"id":"22712edf-f9ff-4b98-8fb3-a8f6c84a88f8","arxiv_id":"2509.22015","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Concept-SAE adds dual-supervised concept tokens and unsupervised free tokens to sparse autoencoders, claiming faithful, localized concept representations that support causal intervention and robustness diagnosis.","lead":"A new training scheme augments sparse autoencoders for vision models with concept tokens that are supervised to predict whether a user-provided concept is present and where it appears, plus free tokens for everything else. The authors show the concept tokens localize better than a concept-embedding baseline and use them to edit predictions and find fragile layers under adversarial attacks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Causal-correction claims rest on an untested counterfactual assumption: replacing real activations with aggregator reconstructions is treated as a valid concept intervention, with no identity-intervention or reconstruction-fidelity control reported.","rationale":"The reader's weakest assumption and my reading converge on the same point: the paper's headline contribution is active causal probing, but the causal claims depend on the counterfactual validity of aggregator-based feature substitution. That validity is never demonstrated. The minimal control is to substitute the reconstructed feature without any concept-score edit and check whether the model's prediction is preserved; if it is not, every score-editing effect is contaminated by reconstruction error. This is a concrete, missing experiment rather than a philosophical objection. I give credit for what is independently supported: the localization ratio in Table 1 is a real quantitative comparison against CEM, the qualitative reconstructions show spatial localization, the code link is a concrete artifact, and Appendix E's irrelevant-concept entropy results suggest the tokenizer does not fire randomly. These support RQ1-level claims of localization and disentanglement. They do not, however, validate the counterfactual interventions in RQ2, and the RQ3 finetuning results would also benefit from non-targeted baselines. Because the paper is a preprint with a plausible method and a missing decisive control, the appropriate disposition is conditional acceptance contingent on that control, which matches the reader's CONDITIONAL verdict. No verdict change is needed.","tokens_in":14919,"tokens_out":4775,"duration_ms":44051,"concrete_test":"Run an identity-intervention control on a held-out CelebA/ImageNet set. For each layer used in Fig. 7, compare (a) model prediction on original h, (b) model prediction when h is replaced by A_concept(T_concept(h)) with no score edits, and (c) model prediction when random concept scores are zeroed. Report per-layer top-1 agreement and mean logit distance. If (b) causes a material accuracy drop or logit shift relative to (a), then the score-editing effects in Fig. 7 are confounded by reconstruction artifacts and the causal-correction claim fails. Additionally, fit a simple density estimate (e.g., Mahalanobis distance) on real activations and measure whether edited-score reconstructions are outliers; if they are easily separable, the intervention is out-of-distribution. The identity control alone is the decisive minimal experiment.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the intervention protocol in RQ2 (§4.2, Fig. 7): the original activation h is replaced by A_concept(T_concept(h)) after manually editing concept scores. For this to demonstrate a causal link between an internal concept and the prediction, the reconstructed feature must lie on the target model's activation manifold and be processed as though the input concept had actually changed. Section 3.3 trains the aggregator only with an MSE reconstruction loss (Eq. 4); no quantitative reconstruction accuracy on held-out data is reported (only qualitative examples in Figs. 4, 5, 12, 13), there is no control where h is replaced by the unmodified reconstruction, and there is no check that edited-score features are in-distribution. Without these, the prediction flips in Fig. 7 could be artifacts of a lossy or out-of-distribution aggregator rather than evidence about the model's concept usage. The same gap weakens the abstract's 'validated fidelity' claim and the RQ2 diagnosis. RQ3's layer ranking via JS distance is less affected because it measures score shifts on real adversarial inputs, but its claimed link between JS distance and finetuning benefit also lacks a random-layer baseline. Section 5 acknowledges label noise from ClipSeg/GPT-4o but does not address this counterfactual-validity gap, which is independent of label quality.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Concept-SAE, a method to augment sparse autoencoders with concept tokens that are supervised by VLM-generated existence scores and segmentation masks, while retaining free tokens trained with the usual SAE objective. The authors claim that this dual-supervision and staged training yields concept tokens that are faithful, spatially localized, and disentangled from the residual subspace, enabling causal intervention on predictions and layer-wise localization of adversarial vulnerabilities. Experiments on CelebA and ImageNet compare localization against Concept Embedding Models, report concept-score entropy for correct/incorrect/adversarial examples, and measure adversarial finetuning accuracy after targeting layers selected by JS distance.","tokens_in":15301,"tokens_out":5102,"duration_ms":42693,"significance":"If validated, Concept-SAE would be a valuable contribution to interpretability by providing externally anchored concept tokens that can be manipulated directly. The idea of using both existence labels and segmentation masks as dual supervision is a sensible way to ground token semantics, and the external anchor via GPT-4o and ClipSeg avoids purely self-referential concept labeling. The release of code supports reproducibility. However, the current evidence does not yet substantiate the 'validated fidelity' and 'causal' claims because key controls—reconstruction accuracy, identity-intervention baseline, random-layer baseline, and statistical significance—are missing.","major_comments":[{"comment":"The causal-correction experiment replaces the original activation h with A_concept(T_concept(h)) after editing concept scores, but no quantitative reconstruction fidelity for the aggregator is reported (only qualitative examples in Figs. 4, 5, 12, 13), and there is no control condition in which h is replaced by the unmodified reconstruction. Without such a control, and without a check that edited-score inputs remain on the target model's activation manifold, the prediction flips in Fig. 7 could be artifacts of a lossy or out-of-distribution aggregator rather than evidence about the model's concept usage. This missing control is load-bearing for the abstract's 'validated fidelity' claim and for RQ2's conclusion; §5 acknowledges label noise but does not address this counterfactual-validity gap.","section":"§3.3, Eq. (4); §4.2, Fig. 7"},{"comment":"The claim that layers with larger JS distance are more vulnerable is supported only by the correlation between the JS distance column and the after-finetune accuracy column. The experiment lacks a random-layer baseline: any fine-tuning of a single layer might improve adversarial accuracy by itself, and depth or other confounds could explain the pattern. The table also reports no error bars or significance tests, and only FGSM attacks are used, so the 'systematically localizing adversarial vulnerabilities' conclusion is not yet established.","section":"Table 3, RQ3"},{"comment":"The entropy differences between correct and incorrect predictions are extremely small (e.g., +0.024 to +0.036 for CelebA) and the paper reports no variance, significance tests, or alternative baselines. The conclusion that 'uncertain concept activations are a key factor of model failures' is not supported by these unreplicated point estimates.","section":"Table 2, RQ2"},{"comment":"The localization comparison against CEM is limited to three shallow layers of ResNet-18 on CelebA, with no error bars and no evaluation on other models or datasets. The claim of consistently superior concept localization is therefore not established beyond this narrow configuration.","section":"Table 1, RQ1"}],"minor_comments":[{"comment":"The word 'distangled' should be 'disentangled'.","section":"Table 1 caption"},{"comment":"The notation λ1, λ2, λ3 is reused with different numerical weightings across the three loss functions; consider using distinct symbols (e.g., α, β, γ) to avoid confusion.","section":"Eqs. 3, 4, and 5"},{"comment":"The number and dimension of concept tokens are not reported; only the free-token count (36) is given. This information is needed to interpret the capacity of the concept module.","section":"§3.2-3.4"},{"comment":"The statement that interventions on deeper ViT layers are more effective is based on Fig. 7, which is a set of qualitative examples; quantitative aggregate statistics would strengthen the claim.","section":"§4.2, RQ2"},{"comment":"The paper provides the full prompt used for ImageNet concept-label generation but not the corresponding CelebA prompt, which makes the external anchor for the key experiments less reproducible.","section":"Appendix C"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claims are plausible but the evidence is not yet sufficient for the strength of the claims. The authors should be required to add the missing controls—reconstruction fidelity metrics, an identity-intervention baseline, random-layer baselines, and significance testing—before publication. The framing of the paper might also be adjusted to present Concept-SAE as a promising new method with preliminary evidence rather than as a fully validated causal instrument."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Concept-SAE is a real architectural contribution: it marries supervised concept tokens (existence + spatial mask) with free SAE tokens in a reconstruction objective, and the external grounding via GPT-4o and ClipSeg avoids the circularity that plagues self-labeled feature interpretability. The localization result in Table 1 is the strongest evidence—the ratio gap versus CEM is sizable and consistent across shallow layers. Free tokens remaining interpretable is a nice check that the supervised branch doesn't gut the SAE's discovery function. I'd take the method seriously as a way to make SAE features more actionable.\n\nThe soft spots are where the paper pushes beyond interface quality. RQ2's causal correction assumes that A_concept(T_concept(h)) with edited scores is a valid counterfactual feature. That assumption is untested: no held-out reconstruction accuracy, no identity-intervention control (replace h with the unmodified reconstruction and see if prediction is preserved), and no in-distribution check for edited score vectors. Without these, the prediction flips in Fig. 7 could be aggregator artifacts. This is a load-bearing gap, not a cosmetic one, and the limitations section doesn't acknowledge it. RQ3 is less affected because it measures score shifts on real inputs, but the claimed link between JS distance and finetuning benefit needs a random-layer baseline and error bars; currently only FGSM and no significance tests. Tables 2 and 3 also lack variance.\n\nThe paper would benefit from: quantitative reconstruction metrics (e.g., MSE on held-out features, feature-space cosine), an identity-intervention control, a random-layer baseline for Table 3, and error bars on all main tables. I don't see evidence of a circular or dishonest methodology—the external anchor is genuinely independent—but the causal claims are currently overstated. A reproducible pipeline with a commit hash would also help.\n\nWho is this for? Researchers building concept-based tools for vision model debugging and robustness auditing. It deserves a serious referee; the core idea is sound enough that a major revision with the missing controls could make it a solid contribution.","headline":"Novel concept/free token architecture worth engaging, but the causal-intervention and vulnerability-localization claims need controls before they can be trusted.","tokens_in":15695,"tokens_out":2081,"would_cite":true,"duration_ms":18559,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that supervising SAE tokens with both existence scores and spatial masks makes model features faithful, disentangled, and causally intervenable, turning sparse autoencoders from passive dictionaries into active probes of…","keywords":["sparse autoencoders","mechanistic interpretability","concept bottleneck models","concept embeddings","causal intervention","adversarial robustness","concept localization"],"falsifier":"Compute the reconstruction error of the concept aggregator on held-out images and compare downstream accuracy when hidden features are replaced by concept-plus-free reconstructions against a standard SAE reconstruction baseline; if the intervention effect vanishes or predictions change for unrelated concepts, the causal corrections are artifacts of distribution shift rather than evidence of concept control.","tokens_in":14700,"feed_emoji":"🧠","tokens_out":5266,"duration_ms":46765,"temperature":0.7,"pith_summary":"Concept-SAE tries to upgrade sparse autoencoders (SAEs) from passive observational tools into instruments for active, causal probing of vision models. It splits a model's activation subspace into concept tokens, supervised by both a binary existence score and a spatial segmentation mask, and free tokens, which behave like a standard SAE and capture residual information. The paper argues that this dual supervision makes concept tokens faithful, spatially grounded, and cleanly separated from residual features, and that this validated interface enables three concrete diagnostics: detecting adversarial samples, causally editing predictions by changing concept scores, and localizing which layers are most vulnerable to attack. If the claim holds, interpretability moves from correlational feature inspection to testable interventions on the model's internal concepts.","feed_headline":"Dual-supervised tokens turn sparse autoencoders into causal probes","feed_subtitle":"Concept-SAE lets users edit concept scores to change predictions and pinpoints layers vulnerable to attack.","key_machinery":"The machinery is a tokenizer–aggregator pair. The Concept Tokenizer projects intermediate feature maps into concept-specific embeddings and predicts, for each concept, a binary existence score and a spatial segmentation mask under MSE losses plus an L1 sparsity penalty. The Concept Aggregator, trained after the tokenizer is frozen, multiplies the predicted score and segmentation, fuses them through an MLP, and reconstructs the original feature map; a KL-divergence term aligns the aggregator's per-concept channel distribution with the tokenizer's so that edits stay localized. A parallel Free Tokenizer and Free Aggregator, trained jointly without supervision, reconstruct the residual information. The invertible reconstruction is what turns an edited concept score into a counterfactual feature that can be substituted into the model.","core_discovery":"Concept-SAE's central claim is that a hybrid disentanglement strategy—anchoring some SAE tokens to user-defined concepts through dual supervision on concept existence and spatial localization—forges concept representations that are faithful and clean enough to support causal claims. The paper reports that on CelebA and ImageNet, across ResNet-18 and ViT models, concept tokens reconstruct only concept-relevant image regions and achieve higher localization ratios than a concept-embedding baseline. It further reports that concept-score entropy rises on incorrectly predicted and adversarially perturbed inputs, that editing concept scores and feeding the reconstructed feature back into the model changes predictions in the intended direction, and that layers showing large Jensen–Shannon distance between clean and adversarial concept-score distributions are the ones whose fine-tuning yields the largest robustness gains.","pith_inferences":["A direct extension the paper leaves implicit: the same tokenizer–aggregator interface could be transferred to language models by replacing spatial masks with token-span masks, turning 'localized concept' supervision into span supervision.","Concept-score entropy and Jensen–Shannon distance could be reused as a lightweight, training-free out-of-distribution or adversarial detector; the paper reports the signal but does not frame it as a standalone detector.","The free-token pool is a natural place to grow the concept set: tokens with consistently high activation on coherent image clusters could be promoted into supervised concept tokens, letting the concept vocabulary expand from the model's own structure.","Because concept fidelity inherits the quality of the external label sources, improvements in foundation-model segmentation and vision-language grounding should directly improve Concept-SAE's faithfulness, even for concepts that are hard to localize today."],"forward_implications":["Concept scores become a diagnostic signal: if entropy rises systematically on misclassified and adversarial inputs, score distributions alone can flag unreliable or attacked inputs.","Direct intervention becomes possible: users can change a concept score, reconstruct the feature, and observe a corresponding change in the model's prediction, enabling causal correction of errors.","Vulnerability localization guides defense: fine-tuning layers with the largest clean-versus-adversarial concept-score shift yields larger robustness gains than fine-tuning other layers.","Free tokens preserve the exploratory role of standard SAEs, so constraining a subset of tokens to user-defined concepts does not have to sacrifice open-ended feature discovery."],"supporting_citations":[{"why":"Supplies the Concept Bottleneck Model baseline, whose restrictive bottleneck the paper argues is unsuitable for intermediate-layer analysis.","marker":"Koh et al., 2020"},{"why":"Supplies the concept-embedding model (CEM) baseline that Concept-SAE is compared against on localization and disentanglement.","marker":"Espinosa Zarlenga et al., 2022"},{"why":"Provides the intervention-aware concept embedding line of work that motivates the need for direct supervision on concept values and locations.","marker":"Espinosa Zarlenga et al., 2023"},{"why":"Represents the standard SAE scaling and evaluation practice that relies on passive, post-hoc inspection of latent tokens.","marker":"Gao et al., 2024"},{"why":"Supplies the segmentation model used to generate the spatial masks that supervise concept token localization.","marker":"Lüddecke & Ecker, 2022"},{"why":"Provides the FGSM attack used to generate adversarial samples for the vulnerability localization experiments.","marker":"Goodfellow et al., 2014"},{"why":"Supplies the Jensen–Shannon divergence measure used to quantify concept-score distribution shift between clean and adversarial samples.","marker":"Lin, 2002"},{"why":"Supplies the CelebA dataset with face attributes and gender labels used in the main experiments.","marker":"Liu et al., 2015"},{"why":"Supplies the ImageNet-1k dataset used for the 1000-class object recognition experiments.","marker":"Deng et al., 2009"}],"fun_headline_variants":["Concept-SAE: steerable concept probes for SAEs","Select concepts, steer SAEs: Concept-SAE","Concept-SAE: dual-supervised concept control","Edit scores, flip predictions: Concept-SAE"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim leans on the assumption that a feature reconstructed from edited concept scores lies on the target model's feature manifold, so the downstream model responds as it would to an input whose concept actually changed; the paper reports only qualitative reconstruction examples rather than measuring this counterfactual validity directly.","fun_headline_variants_meta":{"raw":{"variants":["Concept-SAE: steerable concept probes for SAEs","Select concepts, steer SAEs: Concept-SAE","Concept-SAE: dual-supervised concept control","Edit scores, flip predictions: Concept-SAE"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000271,"raw_usage":{"total_tokens":1628,"prompt_tokens":944,"completion_tokens":684,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":622}},"tokens_in":560,"tokens_out":684,"duration_ms":6590,"temperature":1.0,"reasoning_tokens":622,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:44:15.952645+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the reconstruction error of the concept aggregator on held-out images and compare downstream accuracy when hidden features are replaced by concept-plus-free reconstructions against a standard SAE reconstruction baseline; if the intervention effect vanishes or predictions change for unrelated concepts, the causal corrections are artifacts of distribution shift rather than evidence of concept control.","supporting_citations":[],"review_version":2}