{"id":"38234a14-5a95-40dd-9b08-9c79e7acc0fe","arxiv_id":"2411.16725","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Diffusion model features, when decoded with k-sparse autoencoders, reveal interpretable visual concepts, and a lightweight classifier on the best layer (up_ft1 at t=25) beats prior diffusion-based classifiers on fine-grained benchmarks.","lead":"This paper applies k-sparse autoencoders to diffusion model features to find interpretable visual concepts, and shows that simple classifiers on those features transfer well to image tasks. It maps which layers and denoising steps hold the most useful semantic information across several diffusion architectures.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The quantitative interpretability story (Table 2) rests on σ_label, an average standard deviation over categorical class labels; no label-to-number mapping is specified anywhere, so the metric is undefined as written and the layer/timestep rankings it produces are not reproducible or semantically…","rationale":"The paper pursues two intertwined claims: (1) mechanistic interpretation, that k-sparse autoencoders trained on intermediate diffusion features isolate monosemantic visual concepts whose granularity varies with layer, timestep, architecture, and pretraining data; and (2) transfer utility, that a lightweight probe (Diff-C) on the best layer (up_ft1 at t=25) beats prior diffusion-feature classifiers. For claim (1), the paper needs a valid quantitative measure of semantic granularity; the instrument chosen is σ_label (Sec. 4.2). The concern, which the reader also identified, is that σ_label is an average standard deviation over categorical class labels, and no label-to-number mapping is specified in the main text or the supplementary material. This is an internal definitional gap, not a disagreement with consensus: the Table 2 numbers cannot be reproduced from the description, and under ordering-dependent implementations the rankings can shift with arbitrary alphabetical adjacency. I add two supporting observations: the reported layer differences (8.59 vs 9.48 vs 9.90) are small relative to the ~10.7 standard deviation of uniform-random labels over 37 classes and carry no error bars; and the metric conflates class purity with monosemanticity, since coherent non-class features such as 'grass background' are penalized. I also verified that the metric is genuinely load-bearing: Sec. 4.2 explicitly says the authors primarily relied on σ_label because GPT-4o predictions are noisy, and Table 2 with the surrounding text (Secs. 4.3-4.5) uses it for every layer, timestep, architecture, and model comparison; the table caption and supplementary Sec. G similarly flag GPT-4o as noisy, leaving σ_label as the only quantitative instrument for the interpretability claims. What keeps this from being a rejection: the random-neuron qualitative visualizations and, more importantly, the Diff-C accuracy results in Figs. 4 and 6, which are computed without σ_label and reproduce the layer and timestep trends, plus the Table 5 transfer gains, which stand or fall on the Diff-C numbers and the quoted [33] baselines rather than on σ_label. Those transfer comparisons have secondary weaknesses (single runs without error bars; baselines from a different backbone and protocol, though a trained-feature baseline is also provided), but they do not change the verdict. I agree with the reader's weakest_assumption and recommend the verdict remain CONDITIONAL: the authors should specify the label mapping, demonstrate robustness of Table 2 to label ordering, or replace σ_label with a mapping-free measure, and then re-report the layer and timestep findings.","tokens_in":93,"tokens_out":15369,"duration_ms":195590,"concrete_test":"Recompute Table 2a on Oxford-IIIT Pet from the same per-neuron top-10 image lists, under three label orderings: (i) the default alphabetical class indices, (ii) reverse-alphabetical indices, and (iii) ten random permutations of the 37 class indices; additionally recompute with a mapping-free purity measure such as the mean over neurons of the fraction of top-10 images sharing the modal class, or the mean pairwise cosine distance of CLIP text embeddings of the class names.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2 defines σ_label as 'the average standard deviation in the class labels (σ_label) of the top-10 most highly activating images among the top 1000 most highly activating features of the learned k-SAE.' Class labels are categorical strings (e.g., 'Havanese' vs 'Keeshond'), and a standard deviation is only defined after mapping labels to numbers; neither the main text nor the supplementary material specifies this mapping. Under the most natural implementation, the dataset's integer class ids, which for Oxford-IIIT Pet are assigned alphabetically, the metric's scale is set by arbitrary alphabetical adjacency: breeds with adjacent names contribute a small distance while semantically similar but alphabetically distant breeds contribute a large one. The entire interpretability narrative of Secs. 4.3-4.5 (granularity across layers, timesteps, architectures, model variants) is calibrated with this quantity, and the paper explicitly states in Sec. 4.2 that 'GPT-4o predictions can be noisy, hence we primarily relied on label purity (σ_label) for accurate quantification,' so σ_label is load-bearing, not auxiliary. The reported between-layer differences are also small on the arbitrary scale (8.59, 9.48, 9.67, 9.90 in Table 2a, near the ~10.7 standard deviation of a uniform-random label over 37 classes), with no error bars. A second, independent defect: even with a fixed ordering, σ_label measures class purity, not monosemantic granularity, since a neuron firing on 'grass background' across 20 breeds is semantically monosemantic yet scores maximally impure. What survives independently is the qualitative evidence and the Diff-C transfer results (Figs. 4, 6; Table 5), which do not use σ_label and corroborate the layer/timestep trends; but the paper's quantitative claim to mechanistically map representation granularity across model states is unsupported until σ_label is defined, validated, or replaced.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies how visual semantic information is organized inside frozen diffusion models. It trains k-sparse autoencoders on intermediate activations from different layers and timesteps, and quantifies the \"granularity\" of the discovered features with a label-purity metric (σ_label) defined as the average standard deviation of class labels for the top-activating images. The paper also introduces Diff-C, a lightweight convolutional classifier applied to diffusion features, and reports transfer-learning results on Oxford-IIIT Pet, FGVC-Aircraft, Caltech-101, and a LLaVA visual-reasoning setup. The main claims are that semantic granularity varies nonlinearly across layers and timesteps, that this variation depends on architecture and pretraining, and that Diff-C on the up_ft1 layer at t=25 outperforms previous diffusion-feature classifiers, with large gains on FGVC-Aircraft.","tokens_in":19237,"tokens_out":6744,"duration_ms":61730,"significance":"If the claims hold, the paper would be a useful contribution to diffusion-model interpretability and to the study of diffusion features as visual representations. The k-SAE analysis across multiple architectures and datasets, the qualitative visualizations, and the lightweight Diff-C probe are valuable and largely independent of the specific metric used. The transfer-learning results, especially on FGVC-Aircraft, are suggestive and worth pursuing. The main quantitative interpretability metric is currently under-specified, and the comparison to prior work conflates supervised and zero-shot settings, so the significance is conditional on fixing these issues. The paper also contains a clear quantitative overstatement about inference speedup.","major_comments":[{"comment":"σ_label is not well-defined as written. Class labels are categorical strings, and a standard deviation over strings requires a label-to-number mapping that is never specified. If the mapping is the dataset's integer class ids, the metric's scale depends on an arbitrary ordering of the class names; for Oxford-IIIT Pet the ordering is alphabetical, so alphabetically adjacent breeds contribute small distances while semantically similar but alphabetically distant breeds contribute large distances. All layer- and timestep-granularity conclusions in Secs. 4.3-4.5 and Table 2 rest on this metric, and the paper explicitly states in Sec. 4.2 that GPT-4o predictions can be noisy and that it \"primarily relied on label purity (σ_label) for accurate quantification.\" Please replace σ_label with a label-encoding-independent measure such as the entropy of the top-activating label distribution, report error bars or bootstrap intervals, and re-run the analyses. The reported between-layer differences on Oxford-IIIT Pet (8.59, 9.48, 9.67, 9.90) are also small relative to the approximate uniform-random baseline of about 10.7 for 37 classes, and no variance is reported.","section":"Sec. 4.2, Table 2"},{"comment":"The claimed state-of-the-art improvement over [33] is based on a comparison between a supervised classifier and zero-shot classifiers. Diff-C is trained on target-dataset labels, whereas the Diffusion Classifier of [33] is a zero-shot text-conditioned likelihood classifier. The +1.39% on Oxford-IIIT Pet and +39.03% on FGVC-Aircraft are therefore not apples-to-apples, and the phrase \"significantly better than the best reported numbers in [33]\" overstates the comparison. Please benchmark Diff-C against standard supervised transfer baselines (e.g., linear probes on CLIP and DINOv2 features, or fine-tuned classifiers) or explicitly reframe the contribution as a study of simple probes on diffusion features relative to zero-shot baselines. The comparison also confounds model choice (SD-2.0 vs SD-1.5) and text conditioning (prompt vs empty prompt).","section":"Sec. 4.7, Table 5"},{"comment":"The claimed \"4 orders of magnitude inference speedup\" is not supported by the reported timings. The paper states that the diffusion classifier from [33] takes about 24 sec/sample and Diff-C takes about 0.13 sec/sample, which is a factor of about 185, i.e., roughly two orders of magnitude, not four. Please correct this statement in the abstract and in Sec. 4.7.","section":"Abstract, Sec. 4.7"},{"comment":"The k-SAE is trained on activations from the same target dataset used to compute σ_label, and no held-out split is described for the interpretability evaluation. Because the k-SAE has seen those activations during training, the top-activating images may partly reflect memorization of the training set rather than generalizable semantic structure. The Diff-C transfer results are on held-out test splits and are independent, but the interpretability claims in Secs. 4.3-4.5 would be materially strengthened by reporting σ_label on a validation split or by cross-validating the feature-selection procedure.","section":"Sec. 4.3, Sec. 4.5"}],"minor_comments":[{"comment":"The text says features are extracted at t = {25, 100, 200, 300, 400, 500}, but Table 2b lists t = 0 and t = 500 and omits t = 400; please reconcile the listed timesteps.","section":"Sec. 4.4, Table 2b"},{"comment":"The parameter counts in Table 5 are inconsistent with the text: the paper describes Diff-C as a 40M-parameter model, but Table 5 lists \"Diff-C (upft1)\" with 800M parameters, likely mixing the frozen diffusion backbone with the trained classifier. Please clarify what parameter count is being reported and how it affects the lightweight-claim.","section":"Sec. 4.7, Table 5"},{"comment":"The TopK function is not formally defined, and k is described only as empirically set to 32. A short definition and a sensitivity analysis over k would improve reproducibility.","section":"Sec. 3.2, Eq. (3)"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the journal's scope and the core idea is interesting. The main concerns are technical and fixable: the σ_label metric needs to be replaced or precisely defined, the SOTA comparison needs to be reframed as supervised-versus-zero-shot or supplemented with supervised baselines, and the speedup claim needs correction. I did not identify any citation or novelty concern beyond the comparison issue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the transfer-learning half of this paper is solid and practically useful: a light classifier (Diff-C) on up_ft1 features at t=25 beats the best reported diffusion-feature classifier by a large margin on FGVC-Aircraft and a smaller but real margin on Oxford-IIIT Pet, at a fraction of the inference cost. That result does not depend on the interpretability analysis and should survive scrutiny. Second, the paper's quantitative interpretability story is built on σ_label, and σ_label as defined is not a metric. It is the average standard deviation of class labels, but class labels are strings; no mapping to numbers is specified anywhere. The layer and timestep rankings in Table 2, and hence the central claims about granularity, are not reproducible from the text.\n\nWhat is genuinely new: a systematic k-SAE sweep across layers, timesteps, architectures, and checkpoints of diffusion models, with qualitative neuron visualizations that are often striking. The monotonic layer trend (up_ft1 best for fine-grained, bottleneck best for coarse) is independently corroborated by the Diff-C accuracy curves in Figs. 4 and 6, which do not use σ_label. That triangulation is the paper's best evidence and I take it seriously. The comparison to DiT and DeepFloyd-IF is also informative.\n\nWhere it is soft. The σ_label issue is not a minor footnote; the paper explicitly says they relied primarily on it because GPT-4o is noisy. Even if you fix the missing mapping, standard deviation over integer class ids is an arbitrary measure of impurity: Oxford-IIIT Pet ids are assigned alphabetically, so adjacent breeds look 'pure' and semantically similar but alphabetically distant breeds look 'impure.' And as the stress-test notes, σ_label measures class purity, not monosemanticity: a neuron firing on grass background across many breeds is semantically clean but scores maximally impure. No error bars or multiple seeds anywhere; the between-layer differences in Table 2 are small relative to the ~10.7 std of a uniform label distribution. The SOTA comparison in Table 5 imports numbers from prior work rather than re-running those baselines; that is common but should be labeled as such.\n\nWho should read it: anyone working on diffusion feature interpretability or using diffusion features as frozen representations. The Diff-C recipe and the layer/timestep selection are worth having even if the interpretability metric gets replaced. This paper deserves a serious referee; I would send it to review but with a clear request to define or replace σ_label, add error bars, and either re-run the baselines or clearly state they did not. The core empirical contributions can survive that revision.","headline":"Useful transfer-learning recipe and a genuinely systematic k-SAE analysis, but the quantitative interpretability metric is undefined as written and should not be used to rank layers until it is fixed.","tokens_in":19840,"tokens_out":2432,"would_cite":true,"duration_ms":22256,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68T45"],"pacs":[],"model":"deepseek-v4-flash","headline":"Sparse autoencoders on diffusion features isolate monosemantic visual concepts, and a lightweight probe on the best layer beats prior diffusion-feature classifiers.","keywords":["diffusion models","interpretability","k-sparse autoencoders","monosemantic features","representation learning","transfer learning","feature granularity","Diff-C"],"falsifier":"Recompute σ_label with an explicit, justified label-to-number mapping, or replace it with a categorical purity measure such as normalized mutual information, on the same k-SAE activations; if the ordering of layers and timesteps by purity changes materially, the granularity story loses its quantitative support.","tokens_in":18718,"feed_emoji":"🔍","tokens_out":12107,"duration_ms":88225,"temperature":0.7,"pith_summary":"The paper claims that diffusion models, trained only to denoise images, nonetheless organize their internal activations into discrete, interpretable visual concepts, and that k-sparse autoencoders (k-SAE) can surface these concepts. It further claims that a lightweight convolutional classifier, Diff-C, placed on the single best layer and timestep (up_ft1 at t=25) outperforms all prior diffusion-feature classifiers on Oxford-IIIT Pet and FGVC-Aircraft, with a roughly four-order-of-magnitude inference speedup over the strongest baseline. The motivation is that if generative models encode semantic structure in a layer-specific way, interpretation is directly reusable: the same features that explain neurons also transfer to classification and visual reasoning. The analysis also maps how granularity shifts with layer depth, denoising timestep, architecture, and text conditioning, tying each observation to measurable transfer performance.","feed_headline":"Lightweight probe on diffusion features beats prior classifiers by 39%","feed_subtitle":"A frozen diffusion layer yields a classifier that beats prior methods on fine-grained benchmarks.","key_machinery":"The central mechanism is the k-sparse autoencoder (k-SAE), a network whose hidden layer retains only the top k activations (here k=32) and is trained to reconstruct spatially-pooled diffusion activations from a chosen layer and timestep. The top-k constraint forces each hidden unit to specialize, so the decoder's feature directions become candidate monosemantic visual concepts. The paper pairs this with Diff-C, a four-layer convolutional probe with global pooling and a classification head, which measures whether the same frozen diffusion features transfer to downstream tasks. Together they support the inference that the concepts found by the SAE are real properties of the diffusion representation, not artifacts of the dictionary learning.","core_discovery":"The central discovery is that a pre-trained diffusion model's intermediate features are not a monolithic representation but a depth-dependent hierarchy of visual concepts, from coarse composition at the bottleneck to breed-level details at up_ft1 and back to texture-like patterns at up_ft2. Using k=32 sparse autoencoders, the authors isolate neurons that fire on images sharing a single property, such as a specific dog breed or a grass background, and quantify this purity with a label-purity metric. Consistent with the visualizations, Diff-C achieves 88.69% on Oxford-IIIT Pet and 65.07% on FGVC-Aircraft using up_ft1 features at t=25 with an empty prompt, improving on the best reported diffusion-feature classifier numbers by +1.39 and +39.03 percentage points respectively. The paper also finds that coarse-grained datasets such as Caltech-101 and ImageNet are better served by bottleneck features, that earlier timesteps favor fine-grained tasks while t=200 works for Caltech-101, that SD 1.5 captures more class-specific information than SD 2.1, and that the middle block of the transformer-based DiT plays the role that up_ft1 plays in the U-Net. These results are taken as evidence that diffusion models store semantically meaningful, transferable visual features that can be retrieved without distillation or task-specific losses.","pith_inferences":["If the label-purity metric is a valid proxy for semantic granularity, the same measurement could be used to pick the best layer and timestep per dataset without training any classifier, an automated extension the paper does not pursue.","The observation that class-specific information peaks in the middle of the network in both U-Net and DiT hints that semantic abstraction follows a similar depth-dependent curve in other generative backbones, a pattern worth testing on newer architectures.","If k-SAE neurons genuinely isolate monosemantic concepts, then activating or suppressing specific neurons should produce targeted semantic edits in generated images, a direct causal test the paper does not run."],"forward_implications":["Diffusion features can be used as off-the-shelf representations for classification without distillation, extra losses, or synthetic data generation, because Diff-C trains only a small probe on frozen features.","Feature selection can be guided by interpretability: choosing up_ft1 at t=25 for fine-grained tasks and bottleneck features for coarse tasks yields better transfer than using a fixed layer.","The layer-granularity tradeoff generalizes across backbones but shifts location: the U-Net's sweet spot is up_ft1, while the transformer-based DiT's sweet spot is its middle block.","Diffusion features complement CLIP in multimodal reasoning, improving the LLaVA-Bench relative score from 56.6 to 59.9 when interleaved, whereas DINOv2 features do not."],"supporting_citations":[{"why":"Defines the k-sparse autoencoder with top-k activation that the paper trains on diffusion features.","marker":"[39]"},{"why":"Motivates dictionary learning for monosemanticity and supplies the pre-encoder bias and expansion-factor conventions used in the k-SAE setup.","marker":"[11]"},{"why":"Shows sparse autoencoders recover interpretable features in language models, the evidence base for applying them to diffusion models.","marker":"[16]"},{"why":"Provides Stable Diffusion 1.5, the primary U-Net model whose bottleneck and up_ft layers are analyzed and probed by Diff-C.","marker":"[51]"},{"why":"Provides the transformer-based DiT architecture used to compare how semantic information is distributed in non-convolutional diffusion backbones.","marker":"[46]"},{"why":"Is the diffusion-feature classifier baseline Diff-C improves upon by +1.39% on Oxford-IIIT Pet and +39.03% on FGVC-Aircraft.","marker":"[33]"},{"why":"Supplies prior evidence that diffusion features transfer to segmentation, supporting the claim that these representations are not generation-only.","marker":"[9]"},{"why":"Provides the LLaVA framework and LLaVA-Bench evaluation used to show diffusion features complement CLIP in visual reasoning.","marker":"[34]"}],"fun_headline_variants":["Diffusion features are interpretable and beat prior classifiers","Layer-wise semantic hierarchy in diffusion models enables SOTA","Unlocking diffusion features: k-SAEs expose monosemantic neurons","Revelio: Interpreting diffusion models with sparse autoencoders","Diffusion mid-layer features win by 39% on fine-grained tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The quantitative granularity results depend on σ_label, a purity metric computed as the average standard deviation of class labels among a neuron's top-activating images, but class labels are category names and the paper never specifies how those names are mapped to numbers before taking the standard deviation.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion features are interpretable and beat prior classifiers","Layer-wise semantic hierarchy in diffusion models enables SOTA","Unlocking diffusion features: k-SAEs expose monosemantic neurons","Revelio: Interpreting diffusion models with sparse autoencoders","Diffusion mid-layer features win by 39% on fine-grained tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000391,"raw_usage":{"total_tokens":2061,"prompt_tokens":952,"completion_tokens":1109,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":1023}},"tokens_in":568,"tokens_out":1109,"duration_ms":8878,"temperature":1.0,"reasoning_tokens":1023,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:17:00.806792+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute σ_label with an explicit, justified label-to-number mapping, or replace it with a categorical purity measure such as normalized mutual information, on the same k-SAE activations; if the ordering of layers and timesteps by purity changes materially, the granularity story loses its quantitative support.","supporting_citations":[{"cited_title":"Bricken, A","cited_arxiv_id":null,"evidence_quote":"Motivates dictionary learning for monosemanticity and supplies the pre-encoder bias and expansion-factor conventions used in the k-SAE setup."},{"cited_title":"Radford, J","cited_arxiv_id":null,"evidence_quote":"Provides Stable Diffusion 1.5, the primary U-Net model whose bottleneck and up_ft layers are analyzed and probed by Diff-C."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is the diffusion-feature classifier baseline Diff-C improves upon by +1.39% on Oxford-IIIT Pet and +39.03% on FGVC-Aircraft."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the LLaVA framework and LLaVA-Bench evaluation used to show diffusion features complement CLIP in visual reasoning."}],"review_version":1}