{"id":"df358c78-3b29-4ebf-8a11-cf5a01b21b79","arxiv_id":"2608.10537","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Feature Nonlocality, the entropy of a feature's gradient-based influence across input positions, correlates with whether an SAE feature is token-level or context-dependent and can be used to audit and select features.","lead":"This paper introduces Feature Nonlocality (FNL), a number that measures how much context an AI model's internal feature actually draws on. The authors show that FNL can tell whether a feature is driven by a single word or by the whole surrounding meaning, and they use it to audit jailbreak defenses and to select features for steering.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Gradient-based FNL is not validated against causal attribution, and the BOS-exclusion convention manufactures the zero-FNL jailbreak features by construction.","rationale":"The paper is genuinely careful: cross-corpus stability is measured against a split-half ceiling (Table S1), external pipelines are reproduced nearly exactly, steering hyperparameters are documented with selection criteria (Table S5), and the steering result is honestly labelled proof-of-concept with bootstrap intervals in Table S4. The correlational evidence for an abstraction axis is credible. My concern is not that the empirical correlations are absent; it is that their interpretation as 'contextual reach' rests entirely on identifying squared-gradient entropy with causal influence. The token-injection recovery measure is itself an input-perturbation measure, so a negative correlation with FNL may reflect two views of the same local sensitivity rather than a validated link to abstraction. The BOS convention is a concrete instance where the metric's support choice creates the phenomenon reported. The paper itself acknowledges that the operational choices in Eqs. (5)-(6) are not unique, which further motivates an intervention-based validation before the central claim is accepted as stated. This does not destroy the contribution, but it should be tested with causal patching or a comparable attribution baseline. The reader's conditional verdict is the right level, so no verdict change is needed.","tokens_in":21884,"tokens_out":5408,"duration_ms":51395,"concrete_test":"Recompute the FNL ranking of the 100 Ma et al. reasoning features using an intervention-based influence distribution: for each firing event, patch the embedding at each prefix position t to the mean/zero embedding, record Δz_a, normalize |Δz_a| over positions, and compute the same entropy; compare ranks with gradient-FNL (Spearman and TD/CD AUC). If Spearman < ~0.5 or AUC drops below 0.6, the gradient measure is not a faithful causal witness and the abstraction claim needs re-grounding. Secondary: recompute jailbreak-feature FNL without the BOS exclusion; if the 21/25 zeros become nonzero, that finding is convention-driven.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Definition 1/Eq. (4) identifies the squared gradient norm ||∂z_a(T,P)/∂x_t||^2 with the per-position causal influence of prefix token t on feature activation. This is a local linear sensitivity, not an intervention. In a residual-stream transformer these gradients are dense and can be dominated by token-embedding norms, LayerNorm scaling, and attention-sink/BOS effects, none of which is 'contextual reading.' The validation against token-injection recovery and paraphrase robustness is therefore at risk of being partly definitional: a feature whose activation is easily moved by local token perturbations will have concentrated gradients (low FNL), and a feature insensitive to single-token changes will have diffuse gradients (high FNL). That is close to the proxy itself, not an independent confirmation of abstraction. The risk is not purely hypothetical. The supplementary 'Beginning-of-sequence convention' explicitly excludes the window-initial/attention-sink position from the influence support. Consequently any feature whose activation is driven by position-0 has FNL identically zero by construction. This is exactly how 21 of the 25 CC-Delta jailbreak-mitigation features in Table 3 are classified. The paper presents this as a striking mechanistic finding, but it is partly a consequence of the metric's support convention; no causal-patching or attribution baseline is reported anywhere to show that gradient-entropy tracks intervention-based influence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Feature Nonlocality (FNL), defined as the entropy of the normalized per-position squared-gradient influence on an SAE feature's activation at the final token position. The authors claim that FNL is an LLM-independent, label-free correlational witness of the semantic abstractness of SAE features. They validate FNL against two existing proxy metrics: token-injection activation recovery (Spearman rho between -0.39 and -0.46 across layers/dictionaries) and paraphrase robustness (rho = 0.27), and they report that FNL separates token-driven from context-dependent features with AUC 0.73-0.84. Two downstream applications are presented: an audit of jailbreak-mitigation features, where 21 of 25 effective features are reported to have zero FNL and are characterized as positional/BOS indicators, and a proof-of-concept steering study on DeepSeek-R1-Distill-Llama-8B in which steering high-FNL features improves MATH-500 accuracy by 4.6 points over the unsteered baseline, outperforming low-FNL and random envelopes. The paper includes substantial reproducibility details, including seeds, hyperparameter tables, and bootstrap confidence intervals.","tokens_in":22152,"tokens_out":5108,"duration_ms":46075,"significance":"If the central claim is supported, FNL would be a useful and economical tool for feature selection and for diagnosing whether an SAE feature is a surface-level token detector or a genuinely context-dependent representation, with potential value in safety auditing and interpretability pipelines. The paper is careful in several ways: the cross-dataset stability analysis uses split-half reliability ceilings (Table S1), the steering experiment is explicitly labeled a proof-of-concept, and the code repository includes fixed seeds and detailed run configurations. However, the strength of the contribution depends on whether the gradient-based influence measure faithfully tracks causal influence, and on whether the reported jailbreak-audit result is partially manufactured by the metric's support convention. The correlation magnitudes are modest, and the steering gains are model-specific and tuned on held-out subsets, so the central claims should be treated as promising but not fully established.","major_comments":[{"comment":"The paper identifies the squared gradient norm ||dz_a(T,P)/dx_t||^2 with the per-position causal influence of input token t on feature activation. This is a local linear sensitivity measure, not an intervention-based causal quantity. The validation against token-injection recovery is at risk of being partly definitional: a feature whose activation is easily moved by local token perturbations will have concentrated gradients (low FNL), and a feature insensitive to single-token changes will have diffuse gradients (high FNL). The paper does not report a causal-patching, activation-patching, or alternative attribution baseline to show that FNL tracks intervention-based influence. To support the central claim that FNL measures 'contextual reach,' please add a comparison with a causal attribution method (e.g., activation patching across prefix positions) and report the correlation between FNL and intervention-based nonlocality, as well as a control that accounts for gradient concentration at the firing token itself.","section":"Method (Definition 1, Eq. (4))"},{"comment":"The supplementary 'Beginning-of-sequence convention' states that the window-initial position, which hosts the attention sink, is excluded from the influence support. As a result, any feature whose activation is driven by position 0 has FNL identically zero by construction. This is exactly how 21 of the 25 CC-Delta jailbreak-mitigation features in Table 3 are classified. The paper presents this as a striking mechanistic finding ('most effective features are positional features with low FNL'), but the zero-FNL classification is partly an artifact of the support convention. The steering decomposition in Table 3 provides independent evidence for the positional nature of these features, but the FNL-based characterization does not. Please re-compute FNL for these features with the initial position included (or with a BOS token in the window) and report whether the 21 features still have low FNL; otherwise, make explicit that the zero-nonlocality result is a direct consequence of the exclusion convention.","section":"Supplementary: Beginning-of-sequence convention; Table 3"},{"comment":"The main-text steering claim rests on a comparison of high-FNL, low-FNL, and random envelopes in Table 4. The steering gain gamma is selected per condition on held-out MATH-500 subsets ('we perform pilot sweeps over steering strengths and postselect the best-performing gamma for a held-out subset of MATH-500'). Because the high, low, and random arms may be tuned to different strengths, the reported +4.6 vs +3.8 vs +3.6 differences could reflect steering-strength differences rather than FNL itself. The supplementary bootstrap intervals in Table S4 partially mitigate this, but the main text should either report the gamma values used for each arm in Table 4 or state that the strengths were matched across arms. This point is load-bearing for the steering application, even though the paper labels the experiment as a proof of concept.","section":"FNL-Guided Steering; Table 4"}],"minor_comments":[{"comment":"The sentence 'The correlation bewteen FNL and semantic abstractness will be discussed later in Sec. .' contains an undefined cross-reference; please fill in the correct section number.","section":"Method: Feature Nonlocality"},{"comment":"The text states that 'All figures and tables in this work are reproducible in CPU minutes,' while the supplementary reproducibility section reports on the order of 1,000 GPU-hours of runtime. Please reconcile these statements, or specify that the plotting/analysis code for figures runs in CPU minutes while the underlying gradient and inference computations require GPU-hours.","section":"Data and Code Availability"},{"comment":"The abstract and introduction describe FNL as 'LLM-independent,' but the validation pipeline uses LLM-generated paraphrases and LLM judges in several places. The metric itself is LLM-free, so consider using 'LLM-free' or 'computed without LLM calls' to avoid ambiguity.","section":"Abstract and Introduction"},{"comment":"In Table 3, the all-25-feature direction (held-out safety 0.887) is outperformed by the 21-feature positional subset (0.911) even though the all-25 direction has a larger norm (||v||=3.19 vs 2.04). The text says the positional subset 'accounts for most of the improvement,' but the relative ordering suggests that adding the four content features may actually reduce held-out safety. A brief explanation of this non-monotonicity would help the reader interpret the decomposition.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a candidate for publication after substantial revision. The core idea is interesting and the empirical work is unusually well documented, but the two main reservations—the lack of causal validation for the gradient-based influence measure and the BOS-exclusion convention that partially manufactures the jailbreak-audit finding—directly affect the central claims. The steering experiment is honestly labeled as a proof of concept, and the cross-model stability analyses are a strength. I would recommend that the editor require the authors to add an attribution/patching baseline and to re-analyze the jailbreak features under an alternative convention before the paper is accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this paper is worth reading. It introduces Feature Nonlocality (FNL), the entropy of the normalized per-position squared-gradient influence on an SAE feature's activation, and shows it tracks two established proxies for semantic abstraction—token-injection recovery (Spearman −0.39 to −0.46) and paraphrase robustness (+0.27)—with AUC 0.73–0.84 separating token-driven from context-dependent features. It also finds that most effective jailbreak-mitigation features are positional, and that steering high-FNL features on DeepSeek-R1-Distill-Llama-8B gives a modest MATH-500 gain.\n\nWhat's new is the metric itself, and the validation is genuinely independent: FNL is gradient-based, the proxies come from Ma et al.'s pipeline, and the steering is downstream. The paper is unusually thorough on reproducibility—fixed seeds, bootstrap CIs in the supplement, hyperparameter bookkeeping, and full re-runs of the reference pipelines. The supplementary 'Beginning-of-sequence convention' note is a good example of that honesty.\n\nThe soft spots are real but manageable. First, FNL is a local linear sensitivity, not an intervention-based attribution. The paper never compares it to activation patching or causal tracing. Because low FNL (concentrated gradients) is conceptually close to high token-injection recovery, part of the correlation could be definitional. A single causal baseline would settle this. Second, the BOS exclusion worry: the stress-test claims it manufactures the zero-FNL jailbreak features. Having read the supplementary note, I think that overstates it. Excluding the window-initial position from the influence support does not change the entropy for a feature whose influence is entirely at position 0—that entropy is zero either way. The finding that the features are positional is real, not an artifact. The main text still ought to flag the convention explicitly, but it doesn't break the claim. Third, the steering gains are small (high 0.911 vs low 0.903 vs random 0.901 on MATH-500) and partly tuned on held-out problems; the supplement's CIs show the high-low contrast is significant but tiny. The authors frame it as proof-of-concept, which is fair.\n\nWho is this for? Anyone building or using SAE feature selection for interpretability or steering. It deserves a serious referee. I'd accept it with the request that the authors add a causal-attribution comparison and state the BOS convention's role in the main text.","headline":"The FNL metric is a useful, label-free tool for ranking SAE features by abstraction; the paper is honest and reproducible, with the main caveats being no causal-attribution baseline and a BOS convention that should be flagged.","tokens_in":22664,"tokens_out":7097,"would_cite":true,"duration_ms":53233,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A gradient-based entropy called Feature Nonlocality is claimed to measure the semantic abstractness of a model's internal features without labels or LLM judges.","keywords":["Feature Nonlocality","sparse autoencoders","mechanistic interpretability","semantic abstraction","gradient influence","contextual reach","jailbreak mitigation","activation steering"],"falsifier":"Find a feature with high FNL whose activation is nevertheless entirely driven by a single token: patching or ablating all other prefix tokens leaves the activation unchanged, while removing that one token silences it. That would show the gradient entropy can be large without any real contextual dependence, breaking the claimed link between FNL and semantic abstraction.","tokens_in":21656,"feed_emoji":"🧠","tokens_out":8517,"duration_ms":70143,"temperature":0.7,"pith_summary":"The paper introduces Feature Nonlocality (FNL), defined as the entropy of the normalized per-position gradient influence that an SAE feature's activation receives from each input token. Its claim is that FNL is an LLM-independent, label-free, correlational witness of a feature's semantic abstraction level: token-level and positional features score low, while context-dependent reasoning features score high. The paper supports this with correlations against token-injection recovery and paraphrase robustness, with pair classification AUC of 0.73–0.84, and with two downstream case studies: auditing jailbreak-mitigation features and steering high-FNL features for math reasoning. If true, FNL gives interpretability researchers a cheap, quantitative axis for deciding whether a feature genuinely encodes a concept or merely reads a lexical cue.","feed_headline":"Feature Nonlocality: where a feature reads reveals how abstract it is","feed_subtitle":"Gradient entropy separates token-level cues from context-driven reasoning without labels or LLM judges.","key_machinery":"The load-bearing object is Feature Nonlocality (FNL). Given an SAE feature $a$ firing at the last token of a prompt $P$ with context window $T$, the paper defines the per-position influence of token $t$ as $J_a(t,T,P) = \\|\\partial z_a(T,P)/\\partial x_t\\|^2$, the squared norm of the gradient of the feature's activation with respect to the token embedding. Normalizing these influences to a probability distribution $p_a^{(P)}(t)$ and taking the Shannon entropy $H(a,P) = -\\sum_{t=1}^T p_a^{(P)}(t)\\log_2 p_a^{(P)}(t)$ yields the per-prompt nonlocality; dataset-level FNL averages this over firing events. The entropic form is what lets a single-token feature score near zero while a feature that reads evidence from everywhere across the window scores several bits; the paper also shows that FNL ranking survives across corpora, rises with layer depth before saturating, and tracks decoder-direction geometry rather than semantic similarity.","core_discovery":"The central discovery is that the contextual reach of an SAE feature can be read off from a single backward pass: for each firing event, normalize the squared gradient norm of the feature's activation with respect to every prefix token to a probability distribution, take its Shannon entropy, and average over firing events. The paper argues that this quantity, Feature Nonlocality, tracks the level of semantic abstraction: single-token detectors such as a \"Robert\" indicator sit near zero bits, while an epistemic-uncertainty marker spreads influence across a passage and reaches about five bits on a 128-token window. Empirically FNL correlates negatively with token-injection recovery (Spearman about −0.39 to −0.46), positively with paraphrase robustness (0.27), and separates token-driven from context-dependent reasoning features with 73–84% AUC. On the audit side, the paper finds that most effective jailbreak-mitigation features are positional features with zero FNL rather than content-recognizing features, and on the intervention side, steering a high-FNL envelope improves MATH-500 accuracy by 4.6 points over unsteered DeepSeek-R1-Distill-Llama-8B, though the authors present the steering result as a proof of concept with model-specific gains.","pith_inferences":["A natural validation would be to compare FNL against activation patching or token-ablation ground truth on a handful of features; if gradients overstate the influence of positions that patching shows are irrelevant, FNL would need reweighting before it is used as an audit tool.","The jailbreak-mitigation audit implies a concrete safety lesson not drawn by the paper: defenses that work by low-FNL positional features may fail as soon as the attack moves the harmful request out of the beginning-of-sequence position, so wrapper defenses should be tested at varied positions and formats.","Because FNL is defined for any linear subspace of the residual stream, not just SAE features, it could serve as a common abstraction axis across interpretability methods such as probes, steering vectors, and circuit analyses, letting results from different toolkits be compared on the same scale.","A dictionary-wide FNL scan plus a small steering experiment could become a standard screening step: compute FNL for all features, audit the tails, and use the high-FNL envelope as a candidate feature set, with the low-FNL tail as a natural control for confounding positional effects."],"forward_implications":["Feature selection for interpretability audits can be done with gradients alone, removing the need for curated contrastive datasets and LLM-generated feature descriptions when the goal is to separate token-level from context-level features.","A steered feature's behavioral effectiveness no longer implies conceptual content: a feature that defends against jailbreaks can do so by marking a positional artifact, so intervention results should be audited with a contextual-reach measure before drawing mechanistic conclusions.","FNL gives a continuous axis of abstraction that is stable across corpora, so a single dictionary-wide ranking could support cross-task and cross-model comparisons of where semantic processing happens in a network.","In the paper's proof of concept, a high-FNL envelope beat low-FNL and random envelopes on MATH-500 for the reasoning-distilled model, suggesting that contextual-reach ranking could be a viable label-free selection criterion for reasoning interventions, although the paper notes the gains are model-specific."],"supporting_citations":[{"why":"Supplies the token-injection false-positive and paraphrase false-negative proxy measures of feature abstraction that FNL is validated against.","marker":"Ma et al. (2026)"},{"why":"Provides the reasoning SAE dictionary and ReasonScore token-cue selector used as a comparison and representative-feature baseline.","marker":"Galichin et al. (2026)"},{"why":"Its CC-Delta contrastive selection produces the jailbreak-mitigation features whose FNL audit is a downstream application.","marker":"Assogba et al. (2026)"},{"why":"Supplies the LlamaScope SAEs used for the DeepSeek-R1-Distill-Llama-8B experiments.","marker":"He et al. (2024)"},{"why":"Supplies the GemmaScope SAEs used in the initial stability, depth, and geometry experiments.","marker":"Lieberum et al. (2024)"}],"fun_headline_variants":["Feature Nonlocality: a label-free metric for SAE abstraction","One backward pass quantifies SAE feature abstractness","Token-spread entropy reveals whether a feature is contextual or token","FNL: rank SAE features by abstraction without labels","Contextual reach of a feature predicts its abstraction level"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the squared gradient of a feature's activation with respect to each input token faithfully measures that token's causal contribution to the feature firing, so the entropy of the resulting distribution genuinely reflects how much context the feature reads.","fun_headline_variants_meta":{"raw":{"variants":["Feature Nonlocality: a label-free metric for SAE abstraction","One backward pass quantifies SAE feature abstractness","Token-spread entropy reveals whether a feature is contextual or token","FNL: rank SAE features by abstraction without labels","Contextual reach of a feature predicts its abstraction level"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000558,"raw_usage":{"total_tokens":2728,"prompt_tokens":1095,"completion_tokens":1633,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":711,"completion_tokens_details":{"reasoning_tokens":1551}},"tokens_in":711,"tokens_out":1633,"duration_ms":12101,"temperature":1.0,"reasoning_tokens":1551,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:16:49.197684+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Find a feature with high FNL whose activation is nevertheless entirely driven by a single token: patching or ablating all other prefix tokens leaves the activation unchanged, while removing that one token silences it. That would show the gradient entropy can be large without any real contextual dependence, breaking the claimed link between FNL and semantic abstraction.","supporting_citations":[],"review_version":1}