{"id":"5848faa4-fe8b-4dbc-a7ed-0ac248d94065","arxiv_id":"2501.06196","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"The paper names three components of a neuron's aggregation function as cognitive factors and reports near-unity correlations in GPT-2XL, but the effects are largely true by construction.","lead":"This paper claims that artificial neurons in GPT-2XL form categories through three factors that mirror the parts of their basic math: input activations, connection weights, and the summation of weighted inputs. It is a candidate framework for making neural network internals speak the language of human psychology, but the evidence is mostly a restatement of that math.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The near-unity correlations behind effects X, W, and Sigma are computed on the same top-100/top-10 subsets that define the variables, and no null baseline is provided, so they may be selection or tautology artifacts rather than evidence for three cognitive factors.","rationale":"The paper's central claim requires that the three terms of the aggregation function be separable cognitive causes of category content. For that to hold, the observed correlations must be informative about trained network structure rather than inevitable consequences of computing y from w and x and then selecting on y. That condition is not secured. Sections 4.2 and 4.3 fix the top-10 precursor and top-100 core-token sets, and Section 5 reports correlations only on those sets. Because y_j is the aggregation function, rank correlations between y_j and functions of the same x_i and w_i are expected to be high even without any semantic category structure. The paper's own Section 5.4 labels the multiple regression illustrative due to failed normality and suspected collinearity, and no code or data are provided for audit. These are internal support problems, not mere disagreement with consensus. The reader's rejection is therefore reasonable. However, the concern is empirically testable: a random-weight or shuffled-token null, together with variations of the subset sizes, would show whether the reported effect sizes are substantive or artifacts. If the correlations survive the null, the rejection should be reconsidered; if they collapse, the central claim is unsupported.","tokens_in":17071,"tokens_out":5320,"duration_ms":55426,"concrete_test":"Run a null-model control on the same GPT-2XL activations: randomly permute the layer-0 to layer-1 weight matrices (or shuffle token identities) while preserving marginal distributions, then recompute the Spearman correlations for Tables 1, 2, and 4 plus the Section 5.4 regression over, say, 100 shuffles. If near-unity rho persists under the null, the results are properties of the aggregation formula and subset-selection procedure, not learned categorical segmentation. If it collapses, the top-10 weighted-precursor pathway is genuinely informative. As a second check, recompute with all 6,400 precursors or with a random 10-precursor subset, and with 500 core-tokens instead of 100; large drops in rho would show the Section 4.2-4.3 cutoffs are load-bearing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 5.4 is that the three 'mathematico-cognitive factors'—precursor activations (X), connection weights (W), and the additive combination (Sigma)—drive categorical segmentation. The empirical support consists of Spearman correlations near .94-.999 computed on the same quantities that define the selection: for each destination neuron, the 100 highest-activation tokens ('core-tokens') and the 10 highest-weight precursor neurons (Sections 4.2-4.3). This makes the correlations partly tautological. Since a neuron's activation is literally the aggregation function y_j = Sigma_i w_ij x_ij + b_j, rank-ordering tokens by y_j and then correlating that rank with a partial, unweighted, top-10 subset of the same x_i and w_i is not an independent test of a cognitive mechanism. The paper provides no baseline or null model. Under random weights, or even random token activations with the same marginal distributions, high Spearman rho may persist because the same formula generates the dependent variable and the predictors. Additionally, the restriction in Section 5.1 to core-tokens active in at least one of the 10 precursor neurons conditions on the predictor, which can induce positive association. The multiple regression in Section 5.4 is explicitly labelled illustrative because normality and collinearity assumptions fail, so the only quantitative evidence for the three factors is these unbenchmarked rank correlations. The claim that three independent factors 'drive' segmentation is therefore not established; the observations are consistent with the weaker, architecturally trivial fact that y depends on w and x.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes that the categorical segmentation performed by formal neurons in GPT-2XL is driven by three 'mathematico-cognitive factors' embedded in the aggregation function Σ(w_ij x_ij)+a: categorical priming (effect X, the activation values of precursor neurons), categorical attention (effect W, the connection weights), and categorical phasing (effect Σ, the additive combination). For each of 6,400 layer-1 destination neurons, the authors consider the 100 most activated tokens ('core-tokens') and the 10 highest-weight layer-0 precursor neurons, and they report Spearman correlations between destination-neuron activation ranks and precursor activation or weight variables, along with a multiple regression. They interpret the near-unity correlations as evidence that these three factors drive categorical segmentation. The study is explicitly exploratory and limited to the first two layers of GPT-2XL.","tokens_in":17430,"tokens_out":4356,"duration_ms":41405,"significance":"If the central claim were established, it would offer a bridge between mechanistic interpretability and cognitive-psychological constructs, and the qualitative examples of 'categorical complementation' and 'categorical phasing' are suggestive for hypothesis generation. However, the empirical support is largely tautological: the destination neuron's activation is, by definition, the weighted sum of precursor activations, so correlations between that activation and functions of those same precursor activations and weights are expected to be high even under a null model. The paper is transparent about the illustrative status of its multiple regression, but it does not supply a null baseline, does not test the independence of the three factors, and conditions on a heavily filtered subset. The result, as presented, does not establish three distinct cognitive factors, and the manuscript needs major revision or re-framing before it can support its title claim.","major_comments":[{"comment":"The correlations are largely tautological because the dependent variable is constructed from the same quantities as the predictors. For each token t, the destination-neuron activation is defined as y_j(t) = Σ_i w_ij x_ij(t) + a_j (Section 3). The 'effect X' variable is a sum of a subset of the x_ij(t), the 'effect W' variable is a function of the corresponding w_ij, and the 'effect Σ' variable is a count of non-zero contributions in that same sum. A Spearman correlation between the rank of y_j(t) and an unweighted, truncated version of its own defining terms is not an independent test of a cognitive mechanism; near-unity ρ values would likely persist under random or independent terms. The manuscript acknowledges in §5.1 that activation is 'directly a function' of precursor activations, but it does not provide any null model or baseline (e.g., random weights, shuffled tokens) to show that the observed correlations exceed what the algebraic form alone would produce. These correlations therefore cannot, by themselves, support the existence of three separable factors.","section":"§5.1–5.3, Tables 1, 2, 4, 5"},{"comment":"The analysis is computed on a heavily filtered subset, and the filtering conditions on the predictor. For each destination neuron, only the 100 highest-activation tokens (core-tokens) are retained, only the 10 highest-weight precursor neurons are considered, and in the priming and attention analyses only core-tokens 'activated in at least one precursor neuron' are included. Because the core-tokens are selected by high destination activation and then restricted to tokens that are also active in the selected precursors, positive association can be induced even if the variables are independent. The paper does not report how many tokens are excluded by this condition, nor does it rerun the analyses on the full set of tokens or on a random subset of precursor neurons. Without such controls, the high correlations in Tables 1, 2, 4, and 5 may be selection artifacts.","section":"§4.3 and §5.1"},{"comment":"The only multivariate analysis, the multiple regression on activation rank as a function of the three factors, is explicitly labeled 'illustrative only' because the normality and collinearity assumptions are violated. The standardized coefficients (s-a1 = .86, s-a2 = .56, s-a3 = .65) are therefore not reliable estimates of the relative contributions of the three factors. Since the three variables are constructed from the same aggregation function and are 'likely highly correlated' (as the authors note), the regression cannot establish that priming, attention, and phasing are independent or comparable drivers of categorical segmentation. The paper's central claim in §5.4 that the three factors 'drive the categorical segmentation that neurons perform' is not supported by the quantitative evidence presented.","section":"§5.4, Table 7"},{"comment":"The three proposed factors are not separable empirical causes; they are components of an algebraic identity. Effect X is the set of precursor activations x_ij, effect W is the set of weights w_ij, and effect Σ is the summation operation itself. Since y_j = Σ(w_ij x_ij)+a by definition, saying that X, W, and Σ 'drive' y_j is like saying that the factors of a product determine the product. The cognitive interpretation of these algebraic components may be a useful expository frame, but the empirical tests do not distinguish between the claim that the aggregation function determines the activation (a tautology) and the claim that specific cognitive mechanisms with distinct psychological analogues are operative. The paper needs to articulate what would falsify the three-factor model, e.g., by showing that a neuron whose activation is a nonlinear function of precursors, or whose category is not captured by the top-10/top-100 cutoffs, still exhibits the same 'factors.'","section":"§5.1–5.3, definitions of X, W, Σ"}],"minor_comments":[{"comment":"The phrase 'tokens with the highest average activation values' is ambiguous: the paper does not state over what corpus or context set the average is taken, nor whether activations are averaged over all occurrences of the token. This should be specified for reproducibility.","section":"§4.2"},{"comment":"The text refers to 'Figures 10 to 12' to illustrate residual outliers, but only Figure 10 appears in the manuscript; Figures 11 and 12 are missing.","section":"§5.4"},{"comment":"The reference list contains multiple entries that are unrelated to the content of the paper (e.g., [33] FEL2024 conference, [34] quantum computing, [36] materials science, [44] psoriasis), and several references are duplicated ([51] and [52] are the same Nadeau entry; [56] and [57] are both self-citations). These should be corrected or removed.","section":"Bibliography"},{"comment":"The third effect is introduced as 'effect P' in Section 3 but later denoted 'effect Σ' in Section 5.3; the notation should be made consistent throughout.","section":"§3 and §5.3"},{"comment":"The example neurons are described as 'control neurons,' but no criterion for selecting these examples is given. State how the control neurons were chosen and whether they are representative or illustrative.","section":"Figures 2, 4, 7, 10"},{"comment":"The abstract and title make a broad claim about 'How Do Artificial Intelligences Think?' but the study only examines two layers of GPT-2XL. The conclusions should be explicitly limited to this architecture and layer range.","section":"General"}],"recommendation":"reject","confidential_remarks":"The paper's central quantitative evidence is circular because the predictors are components of the definition of the outcome. The authors are honest about the limitations of the regression, but the manuscript does not supply the missing null baselines or alternative analyses that could make the correlations informative. In addition, the reference list contains many entries that appear irrelevant to the content, which suggests that the bibliography was not carefully curated; the editors may wish to ask the authors to verify all citations if the paper is resubmitted. The fit with q-bio.NC is also questionable, as the work is primarily an interpretability study of a large language model rather than a neuroscience contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read the paper. The headline is that the three 'mathematico-cognitive factors' are just the three ingredients of the linear aggregation function y = Σ(wx)+a: effect X is x, effect W is w, effect Σ is the sum. The authors basically admit this in Sections 5.1–5.3, where they note that activation is directly a function of precursor activations and connection weights. Calling the resulting correlations a test of a new cognitive mechanism doesn't hold up.\n\nWhat is genuinely there: the qualitative taxonomy in Section 5.2.2 (categorical complementation types: intra-lexical, sub-lexical, peri-lexical, para-lexical, graphemic, etc.) and Section 5.3.2 (categorical intersections) is a useful way to describe how groups of GPT-2 layer-0 neurons jointly contribute to layer-1 categories. It's anecdotal, but it's the kind of example that could seed a more systematic study. The paper is also honest enough to flag the multiple regression as illustrative because normality and collinearity assumptions fail.\n\nThe soft spots are not minor. The Spearman correlations (ρ ≈ .94 to .999) are computed on the same top-100/top-10 subsets that define the variables. The dependent variable is the rank of y, and the predictors are unweighted sums of subsets of the x and w that enter y. Under those conditions, high correlation is expected even if the model had no semantic structure resembling human priming, attention, or phasing. There's no null baseline, no random-weight control, and no code or data to re-run with different cutoffs. The conditioning on 'core-tokens activated in at least one precursor' makes it worse. So the quantitative evidence does not support the claim that three distinct cognitive factors drive segmentation. It supports the weaker, architecturally trivial fact that y depends on w and x.\n\nThe reference list also has entries that don't support the statements they're attached to, which is a red flag for preparation quality.\n\nWho gets value from this paper? Someone mining hand-picked examples of neuron behavior in early GPT-2 layers might find the complementation/intersection vocabulary mildly useful. The quantitative claims and the 'genetic factors' framing should not be taken seriously without code and a proper null model.\n\nRecommendation: this deserves a desk reject, not a referee. If a future version includes reproducible code, a random-weight baseline, and a clear statement that the three factors are a terminological repackaging of the aggregation function rather than discovered mechanisms, it could be a modest empirical note for an interpretability workshop.","headline":"The three 'factors' are just the terms of the weighted-sum aggregation function, and the near-unity correlations that 'confirm' them are largely built into the top-100/top-10 selection procedure; only the qualitative taxonomy adds something worth keeping.","tokens_in":17987,"tokens_out":3382,"would_cite":false,"duration_ms":31079,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a formal neuron's categorical segmentation of the token universe in a large language model is set by three 'mathematico-cognitive factors' that live inside the neuron's aggregation function.","keywords":["mechanistic interpretability","categorical segmentation","synthetic neurons","aggregation function","categorical priming","categorical attention","categorical phasing","GPT-2XL"],"falsifier":"Recompute the same Spearman correlations using all 6,400 precursor neurons of each layer-1 destination neuron (or a random sample of the non-top-10 precursors) and the full token vocabulary instead of the top 100 core-tokens; if the correlations fall well below the reported values, the three-factor story is an artifact of subset selection. A second check is to ablate the top-10 connection weights to zero and observe whether the activation-rank changes match the exact weights-predicted ordering.","tokens_in":16862,"feed_emoji":"🧠","tokens_out":6319,"duration_ms":58285,"temperature":0.7,"pith_summary":"This paper claims that a formal neuron's categorical segmentation of the token universe in a large language model is set by three 'mathematico-cognitive factors' that live inside the neuron's aggregation function. The factors are categorical priming (the activation levels arriving from precursor neurons), categorical attention (the connection weights attached to those activations), and categorical phasing (the additive summing that co-activates a token across several precursors). The authors test the claim on the first two layers of GPT-2XL, correlating token activation ranks with each factor; they report Spearman correlations of .94 for priming, .999 for attention, and .976 to .989 for phasing, alongside qualitative examples of how precursor categories contribute sub-dimensions to successor categories. If the claim is right, neuron-level semantics can be read directly off the weighted-sum components, and synthetic categories become analyzable with the same vocabulary used for human attention, priming, and conceptual combination.","feed_headline":"Three factors in a weighted sum drive neuron categories in GPT-2XL","feed_subtitle":"Precursor activations, connection weights, and additive co-activation predict token rankings almost perfectly in layer 1.","key_machinery":"The central object is the aggregation function $\\sum_{i,j}(w_{i,j}x_{i,j}) + a$ that every formal neuron applies to its inputs, where $x_{i,j}$ are the activation values arriving from precursor neurons, $w_{i,j}$ are the connection weights, and $a$ is a bias. The argument works by identifying each algebraic component with a cognitive factor: the precursor activations give priming, the weights give attention, and the additive summation gives phasing. Around this identity the paper builds its operational observables: for each of 6,400 destination neurons in layer 1 it takes the 10 precursor neurons with the largest connection weights and the 100 tokens with the highest average activation ('core-tokens'), then measures rank correlations between token activation and each factor, and performs a multiple linear regression of activation rank on the three factor variables.","core_discovery":"The central claim, stated in the paper's overview of its three factors, is that the aggregation function $\\sum(w_{ij}x_{ij})+a$ genetically shapes the categorical segmentation performed by synthetic neurons. Effect X (categorical priming) is the propagation of precursor activation values into the destination neuron; effect W (categorical attention) is the gating and extraction performed by connection weights; effect $\\Sigma$ (categorical phasing) is the additive combination by which a token already core to several precursors arrives with a strengthened activation at the destination. The paper reports that each factor alone shows an extremely strong ordinal relationship with core-token activation rank in layer 1, and that a multiple regression with the three factors yields standardized coefficients $0.86$, $0.56$, and $0.65$ respectively. It concludes that these factors, together with the activation function, constitute the genetic variables of categorical segmentation, operating through categorical complementation and categorical intersection.","pith_inferences":["A direct test of the causal reading would be to ablate the top-10 weights or feed in shuffled precursor activations and check whether destination activation rankings shift exactly as the three factors predict; the paper's correlations alone do not establish causation.","Because the destination activation is literally a weighted sum of precursor activations, the near-unity correlations may be partly tautological; an independent test would need to compare against a null model that uses random subsets of the remaining 6,390 precursors.","The same three-factor lens could be applied to attention heads, biases, layer norms, and residual connections, which the paper excludes; if those components matter, the factor list may need extension.","The qualitative complementation examples (e.g., 'manager' plus 'Wenger', 'Klopp') suggest that neuron categories could be used as a probe of statistical category structure in language; one could test whether human category judgments match these synthetic sub-dimensions."],"forward_implications":["If the three factors are the generative variables of categorical segmentation, then the semantic content of a neuron can be predicted from a small window of its inputs: the top precursor weights and precursor activations, without probing deeper network states.","Because connection weights act as selective extractors of precursor sub-dimensions, interpretability tools can focus on high-weight connections to explain why a particular token enters a neuron's category.","Destination categories are compositional: their core-token extensions are assembled by complementation and intersection from precursor categories, so category structure can be studied sub-dimension by sub-dimension.","The same factor decomposition should transfer to other transformer layers and models built on the same weighted-sum aggregation, since the identity is a structural property of the architecture.","The qualitative examples imply that apparently 'alien' or non-human categories can arise from ordinary statistical co-activation, not from a separate mechanism."],"supporting_citations":[{"why":"Supplies the GPT-2XL parameter details and neuron activation values that the analysis uses as its data.","marker":"[11]"},{"why":"Provides the linear-decomposition approach for tracing a neuron's activation back to its inputs, which the three-factor split extends.","marker":"[76]"},{"why":"Defines fuzzy logic and α-cut extensions, the basis for treating a neuron's most-activated tokens as the category's extension.","marker":"[84]"},{"why":"Provides the generalized context model in which weighted attention stretches representational space, used to interpret the activation-range result for attention.","marker":"[54]"},{"why":"Supplies the neuron-interpretation methodology that the paper positions itself within.","marker":"[32]"},{"why":"Provides the human syntactic-priming model that motivates the transposition of priming to synthetic neurons.","marker":"[83]"}],"fun_headline_variants":["Three math factors predict GPT-2XL neuron categories","How weighted sums create categorical thought in artificial neurons","Priming, attention, and phasing: keys to AI category formation","Algebraic operations behind AI's categorical segmentation","GPT-2XL layer 1: three factors almost perfectly rank tokens"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The analysis assumes that the only inputs that matter for a neuron's category are its 10 strongest incoming connections and the 100 tokens that already activate it most, so the near-perfect correlations could be an artifact of looking only inside that preselected window.","fun_headline_variants_meta":{"raw":{"variants":["Three math factors predict GPT-2XL neuron categories","How weighted sums create categorical thought in artificial neurons","Priming, attention, and phasing: keys to AI category formation","Algebraic operations behind AI's categorical segmentation","GPT-2XL layer 1: three factors almost perfectly rank tokens"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000785,"raw_usage":{"total_tokens":3398,"prompt_tokens":816,"completion_tokens":2582,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":432,"completion_tokens_details":{"reasoning_tokens":2501}},"tokens_in":432,"tokens_out":2582,"duration_ms":21666,"temperature":1.0,"reasoning_tokens":2501,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:45:50.439623+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the same Spearman correlations using all 6,400 precursor neurons of each layer-1 destination neuron (or a random sample of the non-top-10 precursors) and the full token vocabulary instead of the top 100 core-tokens; if the correlations fall well below the reported values, the three-factor story is an artifact of subset selection. A second check is to ablate the top-10 connection weights to zero and observe whether the activation-rank changes match the exact weights-predicted ordering.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the generalized context model in which weighted attention stretches representational space, used to interpret the activation-range result for attention."}],"review_version":1}