{"id":"b333637d-074f-4d04-a62e-2b8308d1ab64","arxiv_id":"2502.03032","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Cosine similarity between sparse autoencoder features across layers and modules builds flow graphs that explain feature evolution and enable multi-layer steering of language model generation.","lead":"This paper maps how AI 'features' (internal concepts) change as they travel through a language model's layers, using the similarity of learned feature directions rather than extra data. It then shows these maps can be used to steer the model's output toward or away from topics like physics or weddings.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Steering gains may reflect larger total intervention and manual feature selection rather than flow-graph structure; no equal-strength or random-feature control is reported.","rationale":"The reader's weakest assumption points to geometric similarity implying causal correspondence. That concern is real: Section 5.2 reports only about 65% successful deactivations for top-1 matching, and Section C.2 admits that deactivating all matched predecessors is often required to deactivate the target. However, the paper does provide a direct causal check (deactivation) and an active baseline (random top-5) for the matching step, so the matching assumption is at least partially grounded. The steering claim is less protected. The comparison in Figure 10 and Appendix B is single-layer versus cumulative, and cumulative applies coefficients across many layers; the total steering magnitude is therefore not matched. The feature sets are manually curated, and no random or direct-similarity control is run. Without such controls, the 'flow graphs improve steering' result is confounded by intervention strength and manual selection. This is the most load-bearing weakness because the paper's third contribution and parts of the abstract rest on it. A single controlled experiment can settle whether the graph structure, rather than the larger perturbation or manual curation, drives the steering improvement. If the flow-graph condition fails against the controls, the steering claim would need to be rejected while the interpretability contribution might still stand. Given the reader's CONDITIONAL verdict and this additional unmet condition, I would keep the verdict CONDITIONAL and require the controls before full acceptance.","tokens_in":23148,"tokens_out":7370,"duration_ms":67344,"concrete_test":"Re-run the steering experiments with three matched controls: (1) cumulative steering over the same number of randomly selected SAE features at the same layers, with the same total L1 coefficient budget as the flow-graph condition; (2) cumulative steering over features chosen by highest direct cosine similarity to the target feature, but not connected through intermediate flow-graph edges; (3) single-layer steering whose total L1 coefficient equals the cumulative total. If the flow-graph condition does not beat all three on Behavioral x Coherence (with multiple random seeds and confidence intervals), the steering claim cannot be attributed to flow-graph structure.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.3 and Appendix B compare single-layer steering against cumulative steering over layers 0..l, but cumulative applies the steering/rescaling coefficient at every layer in the path, so the total perturbation grows with depth (e.g., exponential decay with alpha = -0.05 sums to roughly 20x the per-layer coefficient), while the single-layer baseline perturbs only one layer. Figure 10's cumulative advantage is therefore confounded with total intervention strength. The cumulative feature set is also manually curated: features are added to the flow graph only if their Neuronpedia interpretation 'remains consistent and relevant' (Appendix B), and coefficients/thresholds are tuned on a small generated batch. No control steers an equal number of randomly chosen SAE features at the same layers, or features matched to the target by direct cosine similarity without intermediate flow-graph edges. Thus the central claim that flow graphs 'can improve the quality of model steering' (contribution 3) is not yet supported; the observed improvement could come from steering more features, or from manually selecting features that are semantically on-theme, rather than from the graph structure.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a data-free method for matching sparse autoencoder (SAE) features across consecutive layers and modules of a transformer by taking the cosine similarity between decoder columns. The matches are composed into 'flow graphs' that are meant to show how features originate, propagate, or vanish through the residual stream, MLP, and attention. The authors validate the matching with co-activation statistics, test causal dependence by deactivating matched predecessor features in hidden states, and use the resulting graphs to steer generation toward or away from target topics. They claim the first multi-layer steering via SAE features and position the method as a causal, cross-layer interpretability and control tool.","tokens_in":23266,"tokens_out":7235,"duration_ms":60509,"significance":"The matching step is simple, parameter-free, and requires no activation data, which is a real practical advantage over correlation-based matching. The deactivation experiments are a genuine attempt at causal validation: top-1 cosine matching beats random selection and is comparable to Pearson correlation and to an exhaustive-search ceiling (Table 1). If the steering results were controlled for total intervention strength and feature-selection bias, the paper would provide a useful cross-layer interpretability and control method. As it stands, the steering evidence is confounded, so the paper's third contribution — that flow graphs improve steering — is not yet established.","major_comments":[{"comment":"The comparison between cumulative and single-layer steering is confounded by total intervention magnitude. In the cumulative strategy, a rescaling or activation coefficient is applied at every layer from 0 to l, so with exponential decay alpha = -0.05 the coefficients sum to roughly 15 times the layer-0 coefficient, whereas the single-layer baseline perturbs one layer only. The cumulative feature set is also manually curated: Appendix B states that features are added only if their Neuronpedia interpretation 'remains consistent and relevant', and thresholds t(R)=0.5, t(M)=0.15, alpha, and s are tuned on a small generated batch. Without a control that steers the same number of randomly selected features at the same layers, or features matched by direct cosine similarity without intermediate flow-graph edges, Figure 10 cannot be read as evidence that the flow-graph structure itself improves steering.","section":"Section 5.3, Figure 10, Appendix B.2"},{"comment":"The causal-precursor assumption is only partially validated. Top-1 cosine deactivation succeeds in 65% of cases, and the exhaustive search achieves only 73%, meaning that for about a third of target features the matched predecessor is not a sufficient causal parent on its own. Section C.2 further states that in most cases deactivating all predecessors is required to deactivate the target. The paper should report feature-level diagnostics — for example, the distribution of activation change, success rates by layer and module, and the fraction of target features whose top-1 match is active when the target is active — to bound how often a flow-graph edge corresponds to the main causal pathway. The 'From nowhere' group can be roughly half of features in early layers (Figure 5), so the claim that flow graphs describe how features 'originate, propagate, or vanish' needs an explicit statement of coverage and failure modes.","section":"Section 5.2, Table 1, Appendix C.2"},{"comment":"The attention branch of the flow graph rests on SAEs whose quality the authors themselves doubt. Section 5.1 says the rarity of attention groups 'could stem from our SAE training rather than an inherent property of Gemma', and Appendix D says 'our attention SAEs for Gemma were perhaps not trained well enough'. Because attention is one of the three module types used in both the group-distribution analysis and the steering procedure, the paper should report attention-SAE quality metrics (reconstruction loss, l0, or downstream interpretability) and show that the main conclusions are robust either to excluding attention or to using higher-quality attention SAEs.","section":"Section 5.1 and Appendix D"},{"comment":"The group-separability validation (Mann-Whitney U tests summarized in Figure 4) shows that score distributions differ between groups, but it does not establish that the top-1 cosine match is the correct individual predecessor. A feature-level evaluation is needed — for example, comparing the cosine-ranked predecessor against the Pearson-correlation-ranked predecessor per target feature and measuring agreement and activation correlation, in addition to group-level comparisons. This would directly test the load-bearing assumption that geometric alignment of independently trained SAE spaces implies semantic and causal correspondence.","section":"Section 3.2 and Section 5.1"}],"minor_comments":[{"comment":"The cross-reference 'Figure 15(a)' should be 'Figure 17(a)' for the plot of the number of selected features, and the caption 'Score is a total metric measured as Behavioral × Cumulative' appears to be a typo for 'Behavioral × Coherence'.","section":"Appendix C.3"},{"comment":"The text refers to a green line for the initial-feature baseline, but the caption only describes 'One-layer' and 'Cumulative'; please add the color or label to the legend.","section":"Figure 10"},{"comment":"The description 'topk(·) zeroes out values below the kth order statistic' should specify that the threshold is the k-th largest value and how ties are handled.","section":"Section 3.2"},{"comment":"The variable s* is used in the linear-scaling formula before it is introduced; please define it at first use (it is later set to 1).","section":"Appendix B.2"},{"comment":"The 'From nowhere' category is described as evidence of high-entropy early processing, but it could also reflect matching failure; the text should make explicit that both interpretations are possible, as Appendix C.1 acknowledges.","section":"Section 5.1 and Appendix C.1"}],"recommendation":"major_revision","confidential_remarks":"The novelty claim of being the first to use multi-layer SAE steering seems plausible, but it depends entirely on the steering experiment. If the confounding identified in Section 5.3 cannot be resolved, the contribution reduces to a data-free cross-layer matching tool, which is still interesting but substantially narrower. The authors' own acknowledgments in Section C.2 and Appendix D already flag the two weakest points: incomplete causal coverage and questionable attention-SAE quality; these need to be addressed directly rather than in appendices."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a legitimate extension of Balagansky/Dunefsky-style feature matching, generalized from residual-only to MLP/attention/residual and packaged as flow graphs. The strongest part is the deactivation experiment: top-1 cosine matching beats random, matches Pearson correlation, and an exhaustive-search bound sits at 73% success (Table 1). That is real causal evidence, not just correlation, and the authors are honest that the matching itself is an incremental extension of prior work.\n\nWhat is genuinely new: simultaneous matching across three module types per layer, the \"From nowhere\" group analysis, and the use of the resulting graphs to pick multi-layer steering targets. The three-part layer partition is suggestive, and the Llama Scope replication attempt adds a bit of generality. The paper is readable, the appendix is thorough, and they do not oversell the matching novelty.\n\nThe soft spots are concentrated in the steering section. Cumulative steering applies the rescaling coefficient at every layer in the path, so the total perturbation grows with depth; the single-layer baseline perturbs only one layer. That means Figure 10's cumulative advantage is confounded with total intervention strength. In addition, the features added to the graph are manually filtered via Neuronpedia interpretations (Appendix B), and coefficients/thresholds are tuned on a small generated batch. There is no control steering an equal number of random features at the same layers, or features matched directly by cosine without intermediate graph edges. So contribution 3, the steering improvement, is not yet supported as an effect of the graph structure. The \"From nowhere\" group is also large in early layers, which the authors acknowledge but which undercuts the strong framing of \"how features originate, propagate, or vanish.\" The lack of error bars and absence of code or data artifacts are minor-to-moderate issues.\n\nDespite the steering weakness, the matching/deactivation core is solid and useful to anyone working on SAE feature correspondence or cross-layer interpretability. I would send this to a serious referee. My recommendation: conditional acceptance, with the authors asked to release code, report variance, and add a random-feature or equal-intervention-strength steering control. If the steering claim survives that control, the paper becomes a genuinely useful tool rather than a promising one.","headline":"Honest, well-scoped extension of SAE feature matching to multi-module flow graphs; the deactivation evidence is real, but the steering gains are confounded and need controls before the paper's headline claim holds.","tokens_in":23894,"tokens_out":1543,"would_cite":true,"duration_ms":16049,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68T50"],"pacs":[],"model":"deepseek-v4-flash","headline":"A data-free cosine-similarity match between sparse-autoencoder directions traces how features are born, transformed, or carried through a transformer, and the resulting flow graph improves model steering by targeting multiple features…","keywords":["sparse autoencoders","feature flow","cross-layer interpretability","cosine similarity matching","model steering","mechanistic interpretability","feature circuits","LLM interpretability"],"falsifier":"Measure the deactivation success of top-1 cosine matches on a set of features whose true causal predecessors are known by construction, for example a synthetic transformer where layer $L{+}1$ features are defined as sparse linear combinations of layer $L$ features. If the top-1 cosine predecessor is not the planted one, or deactivating it rarely removes the target feature, the geometric-correspondence claim fails.","tokens_in":22856,"feed_emoji":"🗺️","tokens_out":6402,"duration_ms":53490,"temperature":0.7,"pith_summary":"This paper tries to establish that the inner geometry of a large language model can be mapped as a flow of concepts without running any data through the model at inference time. The key move is to measure cosine similarity between the decoder directions learned by sparse autoencoders trained at different layers and modules, then chain these matches backward through the network to build a flow graph for any chosen feature. The authors claim these graphs describe how features originate, persist, transform, or vanish, and that they double as practical steering handles: suppressing or amplifying a whole set of causally related features across layers controls generation themes better than editing one feature at one layer. If correct, this gives a lightweight, interpretable, and partially causal control layer for LLMs that needs no activation statistics.","feed_headline":"Cosine similarity maps how LLM features are born and flow","feed_subtitle":"A data-free graph of sparse-autoencoder features shows where concepts come from, and lets you steer them across many layers.","key_machinery":"The central object is the cosine-similarity matching operator between SAE decoder matrices, $T^{(A\\to B)} = I_{x>0}\\big(\\mathrm{top}_k\\big({W^{(A)}_{\\mathrm{dec}}}^\\top W^{(B)}_{\\mathrm{dec}}\\big)\\big)$, which maps features trained at position $A$ to the most similar features at position $B$ under unit-norm columns. For each residual feature at layer $L$, the authors compare its embedding against the SAE decoder columns of the previous residual, the MLP output, and the attention output, assigning the feature to groups ('From RES', 'From MLP', 'From ATT', and their combinations) according to which of those matched predecessors is active; composing these backward matches over consecutive layers yields a flow graph. The same matched directions are used for steering by rescaling hidden states, $h \\leftarrow h + a(r-1)v$, which subtracts or amplifies the feature direction. The flow graph is what carries the argument: it converts pairwise geometric matches into a causal-looking computational pathway that can be validated by deactivation and then used for intervention.","core_discovery":"The paper argues that the per-layer feature dictionaries learned by sparse autoencoders are compatible enough, layer to layer and module to module, that a pure geometry check—maximum cosine similarity between decoder columns—can stand in for activation statistics and reveal the computational ancestry of a feature. Matching backward from a residual-stream feature at layer $L$ to the residual, MLP, and attention SAEs at layer $L-1$, and composing these matches over consecutive layers, produces a flow graph showing a feature persisting unchanged, being refined by the MLP, or appearing for the first time (features with no active predecessor are labeled 'From nowhere'). The authors then treat the matched predecessors as intervention targets: removing a predecessor's decoder direction from the hidden state is expected to remove the target feature, and doing this at several layers at once steers generation thematically. Their deactivation results (top-1 cosine matches succeed about 65% of the time, on par with Pearson-correlation matching and below an exhaustive-search ceiling of 73%) and their steering results (cumulative, multi-layer interventions beat single-layer steering at small rescaling coefficients) are offered as evidence that these geometric flow graphs describe real causal structure rather than mere visual similarity.","pith_inferences":["An immediate testable extension is to replace the many-to-one top-1 match with the many-to-many top-k match everywhere and compare deactivation success, since the paper uses top-5 only to define 'inactive' rather than to steer.","Because the 'From nowhere' group is large in early layers (up to roughly half of features) and the attention SAEs were trained by the authors themselves, the flow graphs are likely to get denser and more accurate as SAE quality improves; the qualitative three-part layer segmentation may partly reflect under-trained attention features.","If the geometric-correspondence assumption is right, the same flow graphs should transfer to other diagnostics such as targeted ablation or circuit pruning, and should agree with crosscoder-style shared-feature decompositions where those are available."],"forward_implications":["A feature's causal predecessors can be identified from SAE weights alone, without collecting activation statistics, and this matches the accuracy of Pearson-correlation-based matching.","Deactivating the single most similar predecessor is informative for causal analysis: it deactivates the target feature roughly 65% of the time, and random selection among the top-5 candidates performs markedly worse.","Intervening on features across several layers (cumulative steering) outperforms single-layer steering at small steering coefficients and reduces sensitivity to the rescaling hyperparameter.","Flow graphs explain steering side effects: the 'London' feature steers toward fashion themes because its earlier-layer ancestors already carry fashion-related semantics."],"supporting_citations":[{"why":"Supplies the one-to-one inter-layer feature matching baseline and residual-stream feature dynamics that this work extends to MLP and attention modules.","marker":"(Balagansky et al., 2024)"},{"why":"Defines the transcoder-based transition operator between layer feature spaces that the cosine-similarity approach is compared against.","marker":"(Dunefsky et al., 2024)"},{"why":"Provides the activation-correlation-based matching methodology that the data-free cosine matching is validated against.","marker":"(Balcells et al., 2024)"},{"why":"Supplies the Gemma Scope sparse autoencoder pack used for the residual and MLP features in the main experiments.","marker":"(Lieberum et al., 2024)"},{"why":"Defines the JumpReLU activation used by the Gemma Scope SAEs that the flow graphs are built on.","marker":"(Rajamanoharan et al., 2024)"},{"why":"Provides the single-layer SAE-feature steering baseline and the initial theme features that flow graphs are built from for activation steering.","marker":"(Chalnev et al., 2024)"},{"why":"Supplies the Llama Scope SAE pack used to check that the feature-group distributions transfer to another model family.","marker":"(He et al., 2024)"}],"fun_headline_variants":["Map, trace, and steer LLM features with pure cosine similarity","From nowhere to steer: cross-layer feature flow made visible","Data-free cosine maps expose LLM feature ancestry and control","See feature birth, refine, or vanish—then steer by cosine match","Causal cross-layer feature graphs: interpret and manipulate LLMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that when a decoder column of one SAE is the most cosine-similar to a column of another SAE, the two columns name the same underlying concept and the earlier one causes the later one; the paper's own tests show this link fails for a large share of features, with deactivation succeeding only about 65% of the time and many features having no matched active predecessor at all.","fun_headline_variants_meta":{"raw":{"variants":["Map, trace, and steer LLM features with pure cosine similarity","From nowhere to steer: cross-layer feature flow made visible","Data-free cosine maps expose LLM feature ancestry and control","See feature birth, refine, or vanish—then steer by cosine match","Causal cross-layer feature graphs: interpret and manipulate LLMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000267,"raw_usage":{"total_tokens":1596,"prompt_tokens":909,"completion_tokens":687,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":600}},"tokens_in":525,"tokens_out":687,"duration_ms":7580,"temperature":1.0,"reasoning_tokens":600,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T10:06:51.318714+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the deactivation success of top-1 cosine matches on a set of features whose true causal predecessors are known by construction, for example a synthetic transformer where layer $L{+}1$ features are defined as sparse linear combinations of layer $L$ features. If the top-1 cosine predecessor is not the planted one, or deactivating it rarely removes the target feature, the geometric-correspondence claim fails.","supporting_citations":[{"cited_title":"Mechanistic permutability: Match features across layers","cited_arxiv_id":null,"evidence_quote":"Supplies the one-to-one inter-layer feature matching baseline and residual-stream feature dynamics that this work extends to MLP and attention modules."}],"review_version":1}