{"id":"c6b4e3f5-f0d5-4eb0-a499-6505430b3d3c","arxiv_id":"2509.04479","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"Rare-token specialists in GPT-2 XL and Pythia are spatially scattered and accessed by generic attention, not modular clusters or dedicated routing.","lead":"An interpretability study reports that transformers handle rare tokens with a distributed set of specialized neurons that are not clustered and are not selectively routed to by attention. The findings, if replicated, support training-driven differentiation over modular architecture in large language models.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'no clustering, no routing' conclusion rests on low-power negative tests: Table 1 modularity is run on 15–20 neuron subgraphs and Table 2 never compares rare vs common routing. A full-layer community enrichment test and a positive control are needed before the central claim is supported.","rationale":"The paper attempts a clear empirical argument: rare-token processing is supported by plateau neurons that are functionally distributed and accessed by nonselective attention. I read this in good faith; the three analyses are complementary and the limitations section is candid. The reader's weakest assumption was that the plateau class is meaningful and stable; that is indeed important. But I see the more load-bearing vulnerability in the negative results: even if the plateau neurons are real, Table 1's modularity test is run on an induced subgraph of only 15–20 neurons, and no positive control establishes that Louvain at that scale can detect modular structure if present. The paper's own control groups are random sets of the same size, which calibrates the null but not the sensitivity. The attention result has a related issue: single-head ablations are small for both max-impact and random heads, and the all-head ablation is large, but this pattern is exactly what a distributed many-head routing scheme would also produce. Without a rare-vs-common comparison or a direct measure of attention mass directed at plateau neurons, the 'no routing' conclusion is not uniquely identified. These concerns do not require rejecting the paper; they require a specific additional analysis. If the full-layer community enrichment test and a synthetic positive control are added, the central claim can be evaluated. I therefore keep the reader's CONDITIONAL verdict, adding this concrete condition to the list. My agreement is partial because the reader emphasized plateau-classification artifact as weakest, whereas I locate the decisive gap in the statistical power and construct validity of the negative results, though the reader did list this as a related premise.","tokens_in":5192,"tokens_out":11149,"duration_ms":146718,"concrete_test":"Run the existing A.1–A.3 pipeline on the full final-MLP correlation graph (all neurons) for GPT-2 XL and Pythia-410M. Detect communities and test whether the 15–20 plateau neurons are overrepresented in any community against 100 random size-matched controls (permutation/hypergeometric). If they are significantly enriched, the Table 1 null is an artifact of the plateau-only induced subgraph and the distributed claim fails. To calibrate the n=15–20 scale, also plant a synthetic two-community structure with 10+10 nodes and within-block correlation ≥0.4 and run the exact pipeline; if Q is not recovered above random controls, the reported null is uninformative regardless of the real data.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing condition for the title claim is that the null modularity and null routing results are true negatives, not merely low-power failures to detect structure. This condition is not established. Table 1 and Appendix A.1 compute Louvain modularity on graphs containing only the 15–20 plateau neurons and compare against random sets of the same size. At that sample size, modularity is known to be low and high-variance even under genuine community structure, and no positive control is reported to show that the exact pipeline would recover a planted two-block structure at n=15–20. The reported Q values (0.03–0.11 vs 0.04–0.09, p=0.42–0.84) are therefore consistent with an uninformative test. A proper test of the claim that plateau neurons are 'spatially distributed' is to ask whether they are enriched in any community of the full final-MLP correlation graph; that analysis is absent. The routing null has the same gap: Table 2 contrasts single-head vs all-head ablation effects but does not compare rare vs common tokens, so it cannot distinguish 'no routing' from 'many-head routing.' The paper's own limitation statement (correlational, no developmental trajectories) further undercuts the causal 'training-driven' wording, but the immediate blocker is that the central negative evidence is underpowered as reported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper asks how transformers organize rare-token processing in the final MLP layer. Using influence ablations, correlation-based community detection, and attention-head ablations in GPT-2 XL and Pythia-410M, it claims (i) rare tokens require a plateau of specialist neurons beyond the power-law regime used by common tokens, (ii) plateau neurons are spatially distributed rather than modularly clustered, and (iii) attention does not selectively route rare tokens to these neurons. The authors conclude that rare-token specialization is distributed and training-driven.","tokens_in":5584,"tokens_out":5146,"duration_ms":66482,"significance":"If the central claims were fully supported, the paper would be a useful contribution to mechanistic interpretability: it would quantify capacity allocation for rare tokens and provide evidence against modular/MoE-style architectures. The use of two model families and multiple complementary analyses is appropriate. However, the current manuscript does not yet establish the load-bearing negative results: the plateau class is defined by the same deviations used as evidence, and the null clustering/routing tests lack positive controls and direct rare-vs-common comparisons. The significance is therefore conditional on substantial additional validation.","major_comments":[{"comment":"The plateau class is constructed from the same deviations that are later reported as evidence. Section 2 defines plateau neurons as those whose influence 'significantly exceeded the fit'; Section 3 then reports δ>0.5 in the top 15–20 neurons as evidence of a dual regime. This is partly definitional. Provide an out-of-sample or threshold-independent validation, and report the fitting procedure (rank range, estimator, goodness-of-fit on rare tokens) so the residual is interpretable.","section":"§2, §3, Eq. (1)"},{"comment":"Token frequency definitions are inconsistent. Section 2 says rare/common tokens are split at the 50th percentile of C4; Appendix A.6 says rare tokens appear <100 times and common >10,000 times in OpenWebText. Also, Appendix A.1 uses 1,000 contexts for correlation graphs, while A.6 says 20 contexts per token pair. The analyses are not reproducible and the rare/common contrast may depend on which definition is used. Please specify the actual corpus, thresholds, and context counts.","section":"Appendix A.6 vs §2"},{"comment":"The null clustering result is underpowered. Louvain modularity is computed on graphs of only 15–20 plateau neurons, and no positive control is reported to show that the same pipeline recovers a planted two-block structure at that size. Reported Q values of 0.03–0.11 vs. controls 0.04–0.09 are consistent with an uninformative test. In addition, the claim that plateau neurons are 'spatially distributed' should be tested by community enrichment in the full final-MLP correlation graph, not only by modularity on the plateau subgraph.","section":"Table 1, A.1, A.4"},{"comment":"The routing claim lacks a rare-vs-common comparison. Single-head versus all-head ablation cannot distinguish 'no routing' from 'many-head distributed routing'; all-head ablation removes all attention, not specifically routing to plateau neurons. Report ablation effects separately for rare and common tokens, and test whether rare tokens' attention is preferentially directed to plateau neurons, beyond the global correlation r=0.89.","section":"Table 2, §3"},{"comment":"The abstract and conclusion say specialization is 'training-driven differentiation,' but the Limitations section states all findings are correlational with no developmental trajectories. This overstates the causal claim. Also, Section 1 describes a 'three-regime' influence pattern, while Section 3 concludes a 'dual-regime structure'; please clarify the terminology.","section":"Abstract, Discussion, §3"}],"minor_comments":[{"comment":"Figure 1 has no error bars or statistical comparison between rare and common tail behavior; adding per-token bootstrap CIs would strengthen the visual claim.","section":"Figure 1"},{"comment":"The activation patching method is cited as 'patching [?]' with an unresolved placeholder; please complete the reference or remove the placeholder.","section":"A.5"},{"comment":"The 'single head (max impact)' row selects the largest effect among many heads; the reported p-values should be adjusted for multiple comparisons or the selection procedure must be described.","section":"Table 2"},{"comment":"Model versions, layer counts, and attention head counts are not specified; include exact model identifiers (e.g., GPT-2 XL, Pythia-410M) and the layers used in analyses.","section":"§2"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is the claim that plateau neurons are spatially distributed and accessed without selective attention routing. That's a real empirical statement, and the paper is honest enough to frame it as a descriptive negative-organizational result. The authors apply standard methods (Louvain, spectral clustering, head ablation) to two model families, and the direction of the findings is consistent: rare tokens recruit extra high-influence neurons, those neurons don't form obvious clusters, and individual attention heads matter little. If that holds, it's a useful counterpoint to modularity-flavored interpretations.\n\nBut the load-bearing parts are weaker than the abstract suggests. First, the plateau class is defined by residuals from a power law fitted to the same influence data, and then those residuals are presented as evidence of a dual regime. That's circular unless the plateau is validated by an independent criterion—activation patterns, causal effects, whatever. Second, the token frequency definitions conflict: Section 2 says the 50th percentile of C4, Appendix A.6 says <100 vs >10,000 in OpenWebText. That's not minor; it changes what 'rare' means. Third, the negative clustering result is run on 15–20 plateau neurons. At that sample size, modularity is low and high-variance even with real structure, and no positive control is reported to show the pipeline would recover a known two-block layout. The right test is enrichment of plateau neurons in communities of the full correlation graph. The routing test has the same problem: single-head vs all-head ablations don't compare rare vs common tokens, so 'no preferential routing' isn't distinguished from 'every head routes a bit.'\n\nThe paper admits its findings are correlational and that it doesn't study developmental trajectories. With no code or data, the empirical claims are hard to check. These are addressable, not fatal: align the token definitions, add positive controls and a full-graph enrichment analysis, report artifacts, and soften the causal 'training-driven' language.\n\nWho gets value from this? Researchers working on sparse interpretability or rare-token mechanisms, especially anyone who might otherwise over-read mixture-of-experts style routing stories. It deserves a serious referee—conditional accept rather than desk reject—because the question is interesting and the evidence, once strengthened, could matter.","headline":"A plausible negative result about rare-token neurons, but the central 'no clustering, no routing' claim is undersupported by low-power tests and a partly circular neuron definition.","tokens_in":6011,"tokens_out":1469,"would_cite":false,"duration_ms":21449,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Rare-token processing in transformers is a distributed, training-driven phenomenon: plateau neurons provide extra capacity beyond the power-law regime, form no spatial clusters, and receive no selective attention routing.","keywords":["rare tokens","transformer interpretability","plateau neurons","modularity","attention routing","distributed specialization","neuron influence","GPT-2 / Pythia"],"falsifier":"Re-run the influence analysis using an alternative attribution method, such as activation patching or integrated gradients, on the same models and contexts. If the 15–20 plateau neurons are not consistently reproduced, the dual-regime claim fails. Also, scan across all attention heads for one whose ablation reduces rare-token plateau activation by more than 20 percentage points above random heads while common-token effects stay small; finding such a head would overturn the no-routing claim.","tokens_in":5086,"feed_emoji":"🧠","tokens_out":5037,"duration_ms":59506,"temperature":0.7,"pith_summary":"This paper asks how transformer language models make room for rare tokens. It claims that rare-token processing in the final MLP layer is carried by a small set of high-influence plateau neurons that common tokens do not engage, on top of the ordinary power-law decay of influence. It then claims these plateau neurons are spread across the layer rather than packed into clusters, and that attention treats rare and common tokens alike, with no specialized routing. If true, the finding overturns the modular picture of specialized circuits for rare tokens and shows that training-driven differentiation inside a shared substrate is enough.","feed_headline":"Rare-token skills live in scattered neurons, no special routing","feed_subtitle":"Plateau neurons sit outside the power-law regime, yet they form no clusters and get no dedicated attention.","key_machinery":"The load-bearing object is the plateau neuron: a neuron in the final MLP layer whose mean-ablation influence exceeds the fitted power-law curve by a threshold (δ > 0.5), identifying 15–20 specialist neurons per rare token. Around this, the paper assembles two test instruments: signed-modularity community detection (Louvain and spectral) on activation-correlation graphs to test spatial clustering, and attention-head ablation with control ablations to test selective routing.","core_discovery":"The central claim is that rare token specialization in GPT-2 XL and Pythia is distributed: (1) rare tokens recruit 15–20 plateau neurons whose influence deviates strongly from a power-law fit, while common tokens follow a clean power law; (2) the plateau neurons show no community structure, with modularity matching random controls; (3) ablating single attention heads barely changes plateau activation while ablating all heads in a layer drops it by 42–45 percent, implying no head is dedicated to these neurons. The authors conclude that transformers achieve adaptive capacity allocation for rare tokens through parameter-level differentiation, not architectural modules or routing.","pith_inferences":["Editorial inference: if plateau classification is reproduced with alternative influence measures, the same power-law residual diagnostic could serve as a cheap, model-wide probe for rare-token capacity without running full ablations.","Editorial inference: the null clustering result is strongest for the final MLP layer and nearby attention; earlier layers might still show modular structure, which would make the distributed conclusion layer-specific rather than universal.","Editorial inference: the paper leaves implicit that tracking plateau neurons across training checkpoints could reveal when the dual-regime structure emerges, offering a direct test of the 'training-driven differentiation' account.","Editorial inference: if the distributed view generalizes, interpretability methods that assign one concept to one neuron will systematically under-explain rare-token behavior, because the mechanism is spread over many weakly connected units."],"forward_implications":["Rare and common tokens are not processed by the same regime; rare tokens demand extra high-influence neurons beyond the power-law scaling that suffices for common tokens.","Because plateau neurons form no clusters, searching for sparse modular circuits for rare-token features may be misdirected.","Because no single attention head matters, interventions on individual heads will not disrupt rare-token processing much, while whole-layer interventions will.","Distributed specialization suggests robust, flexible capacity allocation: redundant, context-sensitive access to specialist neurons.","The paper argues that mixture-of-experts style routing may be unnecessary for rare-token processing because universal attention already integrates specialists."],"supporting_citations":[{"why":"Supplies the plateau-neuron / three-regime influence finding that this paper extends; defines the specialization phenomenon under study.","marker":"[7]"},{"why":"Provides the Louvain community-detection algorithm used to test whether plateau neurons cluster.","marker":"[2]"},{"why":"Provides the C4 corpus from which the rare and common token samples are drawn.","marker":"[10]"},{"why":"Documents the long-tail prediction problem that motivates why rare-token processing matters.","marker":"[5]"},{"why":"Establishes feed-forward layers as key-value memories, grounding the choice to analyze the final MLP layer.","marker":"[4]"},{"why":"Represents the modular, routing-based alternative (mixture-of-experts) that the no-routing result challenges.","marker":"[11]"},{"why":"Supplies complementary-learning-systems theory, the source of the modular vs. distributed hypotheses being tested.","marker":"[9]"},{"why":"Provides the parallel distributed processing perspective underlying the distributed-specialization interpretation.","marker":"[8]"}],"fun_headline_variants":["No clusters, no routing: rare tokens rely on scattered neurons","Distributed neurons do rare tokens, no modules or routing","Transformers split rare-token work across scattered, unrouted specialists","Rare tokens get 20 scattered neurons, no clusters, no dedicated heads","Plateau neurons: scattered, unclustered, and ignored by attention"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The central claim collapses if the 'plateau neurons' singled out by thresholding the residual from a fitted power law are an artifact of that fitting procedure rather than a stable functional class.","fun_headline_variants_meta":{"raw":{"variants":["No clusters, no routing: rare tokens rely on scattered neurons","Distributed neurons do rare tokens, no modules or routing","Transformers split rare-token work across scattered, unrouted specialists","Rare tokens get 20 scattered neurons, no clusters, no dedicated heads","Plateau neurons: scattered, unclustered, and ignored by attention"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00025,"raw_usage":{"total_tokens":1348,"prompt_tokens":659,"completion_tokens":689,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":403,"completion_tokens_details":{"reasoning_tokens":607}},"tokens_in":403,"tokens_out":689,"duration_ms":7230,"temperature":1.0,"reasoning_tokens":607,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T13:23:22.632637+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the influence analysis using an alternative attribution method, such as activation patching or integrated gradients, on the same models and contexts. If the 15–20 plateau neurons are not consistently reproduced, the dual-regime claim fails. Also, scan across all attention heads for one whose ablation reduces rare-token plateau activation by more than 20 percentage points above random heads while common-token effects stay small; finding such a head would overturn the no-routing claim.","supporting_citations":[{"cited_title":"Emergent Specialization: Rare Token Neurons in Language Models","cited_arxiv_id":"2505.12822","evidence_quote":"Supplies the plateau-neuron / three-regime influence finding that this paper extends; defines the specialization phenomenon under study."},{"cited_title":"Kandpal, H","cited_arxiv_id":null,"evidence_quote":"Documents the long-tail prediction problem that motivates why rare-token processing matters."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes feed-forward layers as key-value memories, grounding the choice to analyze the final MLP layer."},{"cited_title":"Shazeer, A","cited_arxiv_id":null,"evidence_quote":"Represents the modular, routing-based alternative (mixture-of-experts) that the no-routing result challenges."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies complementary-learning-systems theory, the source of the modular vs. distributed hypotheses being tested."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the parallel distributed processing perspective underlying the distributed-specialization interpretation."}],"review_version":1}