Pith. sign in

REVIEW 4 major objections 5 minor 208 references

Polysemy of Synthetic Neurons Towards a New Type of Explanatory Categorical Vector Spaces

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A neuron in a language model may be a small categorical vector space of its own.

desk verdict The main empirical claim is likely built into the construction; the paper's geometric hypothesis is clear but unsupported as presented. read the letter →

arxiv 2505.07831 v1 pith:WTCXODM3 submitted 2025-04-30 cs.CL cs.AIcs.LG

Michael Pichat , William Pogrund , Paloma Pichat , Judicael Poumay , Armanouche Gasparian , Samuel Demarchi , Martin Corbet , Alois Georgeon
show 1 more author
Michael Veillet-Guillem
This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords polysemanticneuronsinterpretabilitycategoricalvectorspacenon-orthogonalbasisactivationintra-neuronalattentionGPT2-XLsuperposition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a geometric account of why a single neuron in a language model can respond to many different kinds of input. Rather than treating a neuron only as one feature direction embedded in a layer-wide space, it models each neuron in layer n as a categorical vector space whose non-orthogonal basis consists of categorical sub-dimensions clipped from strongly connected neurons in layer n-1. Within that space, the paper claims, a token's activation level is a positive function of its coordinates on those sub-dimensions, so the most strongly activated tokens sit at the intersection of several sub-dimensions and are less polysemous. If true, this gives a directly testable way to locate a less-polysemous 'critical' zone inside each neuron, and it complements rather than contradicts the usual superposition explanation of polysemy.

What carries the argument

The load-bearing object is the taken-cluster, defined as the set of tokens simultaneously among the 100 most-activated tokens of a layer-1 neuron and among the 100 most-activated tokens of one of its ten most strongly connected layer-0 precursor neurons; each taken-cluster is treated as the extensional realization of one principal categorical sub-dimension. Dimensional proximity, the mean cosine similarity between a token and a taken-cluster in GPT2-XL's input-embedding space, operationalizes the token's coordinate on that sub-dimension, and averaging these coordinates across sub-dimensions yields the value compared with activation rank. The mechanism claimed to generate this geometry is categorical clipping, driven by three effects: categorical priming (the x effect), inter-neuronal categorical attention (the w effect), and categorical phasing (the sum effect). Together they are said to make highly activated tokens fall at the intersection of several sub-dimensions, producing categorical convergence and reduced polysemy at high activation levels.

What would settle it

Recompute the activation-versus-proximity monotonicity after replacing each taken-cluster with a frequency-matched random set of tokens drawn from the same layer-0 neuron's core tokens; if the positive difference d > 0 and Kendall's tau around .8 persist unchanged, then token frequency or embedding geometry, rather than categorical sub-dimensional membership, explains the ordering.

Watch

Extended reading notes

Core claim

The central claim is that the polysemantic thought category of a neuron in layer n can be modeled as a categorical vector space with a non-orthogonal basis, where each basis element is a categorical sub-dimension clipped from the category of a strongly connected precursor neuron in layer n-1. Operationally, each principal categorical sub-dimension is represented by a taken-cluster: the intersection of the layer-1 neuron's 100 most-activated core-tokens with the 100 core-tokens of one of its ten strongest layer-0 precursor neurons. For each core-token, its coordinate on a sub-dimension is measured as the mean cosine similarity, in GPT2-XL's input-embedding space, between that token and the taken-cluster tokens. The paper reports that 78.01% of 614 qualifying layer-1 neurons show a positive difference between the mean dimensional proximity of their most activated and least activated core-tokens, with a mean difference of .0274 and a Cliff's delta of .2407, and that across 612 neurons with exactly three sub-dimensions, Kendall's tau between activation rank and mean proximity ranges from .78 to .85. It concludes that activation magnitude is a positive, monotonic function of a token's coordinates in the neuron's categorical vector space, with the highest activations marking the intersection of several sub-dimensions and thus a less-polysemous critical zone.

Load-bearing premise

The argument assumes that a 'taken-cluster' (the intersection of a layer-1 neuron's 100 most-activated tokens with a layer-0 neuron's 100 most-activated tokens) is the true extensional realization of a categorical sub-dimension and that GPT2-XL input-embedding cosine similarity measures a token's coordinate on that sub-dimension, so if those intersections are artifacts of token frequency or embedding geometry, the monotonicity result would not establish the proposed vector space.

Editorial extensions

If this is right

  • Activation rank can serve as a cheap, local readout of categorical membership: the most strongly activated tokens of a neuron are its least polysemous ones, concentrated near the intersection of its sub-dimensions.
  • The intra-neuronal critical zone identified this way becomes the natural source for categorical clipping into the next layer, giving a concrete mechanism for how categories are refined layer by layer.
  • The non-orthogonal basis description reconciles the standard superposition view with the observation that neurons look monosemantic at their top tokens: both are true at different activation scales.
  • Dimensional proximity scores computed from taken-clusters offer an interpretability tool that explains individual neuron behavior without training supervised probes.
  • The monotonic activation-proximity relationship gives a quantitative handle on the abstract notion of intra-neuronal attention, turning it from a qualitative description into a measurable ordering of tokens.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if the monotonicity holds, a purely activation-based token ranking could be used to select intervention targets for steering a model's output, without needing sparse autoencoders or separately trained concept detectors.
  • Beyond the paper: the same construction should be testable on later layers, since the paper's mechanism predicts that the critical zone of a layer-n neuron should become the source of layer-(n+1) sub-dimensions, so the monotonic relationship should reappear at each layer transition.
  • Beyond the paper: replacing the fixed input embeddings with contextual or layer-specific embeddings would test whether the categorical vector space is an intrinsic feature of the neuron or an artifact of the observation frame, since the paper's operationalization uses only GPT2-XL's input-embedding space.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an alternative geometric account of polysemy in GPT2-XL neurons, in which a layer-n neuron is interpreted as a categorical vector space whose non-orthogonal basis consists of categorical sub-dimensions clipped from its strongly connected layer n-1 precursors. The central empirical claim, tested in Section 4.1, is that a token's activation level is a positive, monotonic function of its dimensional proximity to those categorical sub-dimensions, operationalized as mean GPT2-XL embedding cosine similarity to the neuron's taken-clusters (intersections of the target neuron's 100 most-activated tokens with the 100 most-activated tokens of each of its ten strongest layer-0 precursors). The paper reports group comparisons (Table 1), Kendall correlations on an average neuron (Table 2), and a PCA of sub-dimension proximities (Section 4.2), interpreting the results as evidence for a structured intra-neuronal space with a 'critical' less-polysemous zone.

Significance. If the central claim were established, the proposed intra-neuronal categorical vector space would be a conceptually novel and potentially useful explainability construct, and the paper does formulate a clearly falsifiable monotonicity hypothesis. The work also has strengths: it builds on publicly available GPT2-XL data, explicitly contrasts its account with the superposition framework rather than dismissing it, computes effect sizes (Cliff's delta) alongside p-values, and makes its operationalization concrete enough to be criticized and improved. However, the empirical core is currently undermined by a constructional coupling between the activation ordering and the proximity measure, and by statistical analyses that pool across neurons without accounting for neuron-level variability. The paper is more persuasive as a theoretical proposal than as a validated empirical finding.

major comments (4)
  1. [§4.1, Tables 1–2] The main association is partly constructed by the measurement design. A taken-cluster K_{i,p} is defined as the intersection of the target neuron's 100 most-activated core-tokens with a precursor's 100 most-activated core-tokens, and s_ijk averages cosine similarity between token j and all members of K_{i,p}. If j belongs to K_{i,p}, its self-similarity contributes a perfect score of 1, and cluster membership is itself determined by high activation of j in the target neuron. High-activation tokens are therefore disproportionately included in the clusters and have inflated proximity scores, so the reported d > 0 in 78% of neurons and Kendall tau ≈ 0.78–0.85 can arise even if embedding geometry carries no categorical sub-dimension structure. The paper's note that averaging is over n < 10 sub-dimensions does not address this confound. Please provide a permutation baseline or a hold-out evaluation that removes self-similarity (e.g., leave-one-out cosine) and decouples cluster definition from the target neuron's activation ranking.
  2. [§4.1, Table 2] The Kendall tau coefficients are computed on a single 'average neuron' formed by pooling 612 neurons after rank-alignment, and the reported p-values treat these 100 rank points as independent observations. This is a pseudoreplicated analysis: it does not characterize the distribution of neuron-level correlations that the hypothesis is about, and it can mask substantial heterogeneity. Please report the distribution of neuron-level Kendall tau values with confidence intervals (or a mixed-effects/hierarchical analysis), rather than a single pooled coefficient.
  3. [§3.2, §4.1, §4.2] The analyses use different inclusion thresholds across the three results without justification: n = 614 neurons in Table 1 (at least three taken-clusters of at least six tokens each and a cumulative union of at least 40 tokens), n = 612 neurons in Table 2 (exactly three taken-clusters), and n = 463 neurons in Section 4.2 (exactly four taken-clusters). The paper does not show that the central monotonicity result is robust to these choices, and the ad hoc thresholds make the reported effect sizes difficult to interpret. A sensitivity analysis or a principled selection rule is needed.
  4. [§3.2, §4.1, §4.2] The operationalization of a 'categorical sub-dimension' as a taken-cluster is assumed rather than validated. The paper's geometric interpretation requires that these intersections of top-100 activation sets correspond to semantically coherent sub-dimensions, but no evidence is provided that they are more coherent than random token sets matched for frequency or embedding norms, nor is there a control condition comparing the observed embeddings to a null model (e.g., shuffled cluster labels or random embeddings). Without such a control, the PCA structure in Section 4.2 and the monotonicity in Section 4.1 could reflect generic properties of token-frequency or embedding geometry rather than the proposed categorical vector space.
minor comments (5)
  1. [§4.1, Tables 1–2] Please define the symbols αmin, αmax, σmin, σmax, δ, and Δc in the table captions or text; they are used without definition.
  2. [§4.1, Table 2] Report p-values as p < 0.0001 rather than p = .0000, which implies a zero probability.
  3. [§1.5, near 'Le détourage catégoriel'] The French paragraph followed by its English translation is redundant in a journal article; retaining only one language would improve readability.
  4. [Graph n°1 caption] The caption says 'Comparison of mean activations between categorical clusters from hierarchical classification on tokens' embeddings (layer 0)', but the text says the graph is produced from the same data as Table 2 (layer 1, taken-clusters); please clarify the mismatch.
  5. [Bibliography] Several references appear off-topic or misattributed (e.g., [1] on quantum code generation, [5] and [12] are used inconsistently), and there is a large proportion of self-citations to unpublished preprints; please review the bibliography for relevance and accuracy.

Circularity Check

2 steps flagged · score 6.0 of 10

Central activation–proximity result is partly constructed: categorical sub-dimensions are defined from the neuron's own top-activated token sets, and the proximity score includes self-similarity for taken-tokens.

  1. self definitional [Section 1.5, footnote 1; Section 4.1 (definition of taken-clusters and dimensional proximity)]
    "We define a 'taken-token' as a core-token of a neuron in layer n-1 that is also a core-token of its associated neuron in layer n. A categorical sub-dimension extracted by a neuron in layer n from a neuron in layer n-1 is therefore, by definition, composed of taken-tokens in terms of its extension."

    The categorical sub-dimension is by definition composed of taken-tokens, i.e., tokens that are among the target neuron's 100 highest-activation core-tokens. The dimensional proximity score s_ijk is then the mean GPT2-XL embedding cosine between token j and all tokens in the taken-cluster K_{i,p}=C_i∩C_p. If j itself lies in K_{i,p}, the average includes cos(e_j,e_j)=1, so the 'coordinate' is inflated by self-similarity. More generally, cluster membership is defined using the same activation ordering that the paper then claims is a positive function of these coordinates.

  2. fitted input called prediction [Section 4.1, Tables 1 and 2]
    "These results are consistent with our hypothesis : for a given neuron, the higher the activation of a token, the more it is categorically linked to the principal categorical sub-dimensions associated with that neuron. In other words, the magnitude of a token’s activation is a positive function of the coordinates of that token within the categorical vector space constituted by the neuron’s principal categorical sub-dimensions."

    The 'coordinates' in this claim are exactly the s_ijk scores defined from taken-clusters K_{i,p}=C_i∩C_p, where C_i is the set of the 100 tokens that most activate the target neuron. A token's coordinate therefore already encodes whether the token belongs to the same top-activation set used to define the sub-dimension. The reported Kendall tau values of 0.78–0.85 and the 78% positive d statistic thus largely measure the coupling between activation rank and cluster membership plus self-similarity, rather than an independent geometric property of a pre-existing categorical vector space. Without a control removing self-similarity or using held-out sub-dimension definitions, the 'prediction' is partly a restatement of the construction.

full rationale

The paper's central empirical claim is that token activation level is a positive, monotonic function of a token's coordinates in an intra-neuronal categorical vector space. That claim is not wholly tautological, because the proximity scores use GPT2-XL input embeddings, an external source of semantic similarity, and the monotonicity could in principle have failed. However, the operationalization makes the test partially circular. The categorical sub-dimensions are defined as taken-clusters, i.e., intersections of the target neuron's top-100 activated core-tokens with the top-100 core-tokens of its strongest L0 precursors. The dimensional proximity score averages cosine similarity to all tokens in that cluster, including the token itself whenever the token is a taken-token. Thus the outcome variable is built from the same activation ordering that serves as the predictor. Moreover, because the target activation is an affine function of the L0 activations, high-activation tokens are disproportionately likely to lie in these clusters even under a null model with no semantic sub-dimension structure, so the reported effects can arise from the construction itself. The paper does not report any control for self-similarity, random-embedding baselines, or held-out cluster definitions. This is a partial but substantial circularity: the geometrical interpretation is not independently established by Tables 1 and 2. I do not additionally penalize the heavy self-citation of prior Pichat et al. work, because the paper transparently labels the vector-space framework as a postulate and attempts a fresh empirical test; the circularity lies in the measurement construction, not in the citation chain. Score 6 reflects that one or more central 'predictions' reduce substantially by construction, while some independent embedding-based content remains.

Assumptions & free parameters 2 free parameters · 4 assumptions · 3 invented entities

The central claim rests on four unverified domain assumptions: taken-clusters are genuine semantic sub-dimensions, embedding cosine is a valid coordinate, the averaged 'mean neuron' is a valid statistical unit, and the aggregation function's effects are causal extraction operators. The free parameters are the neuron inclusion thresholds and the 100-token cutoff, neither of which is robustly varied. The proposed entities are all internal to the same measurement pipeline, so they lack independent evidence.

free parameters (2)
  • Neuron inclusion thresholds = >=3 taken-clusters, >=6 tokens per cluster, >=40 tokens total (Table 1); exactly 3 clusters (Table 2); exactly 4…
    Thresholds vary between analyses and are not independently justified; they determine which neurons are retained (614, 612, 463).
  • Core-token cutoff = 100 tokens per neuron
    Inherited from OpenAI neuron-explainer data [12]; no robustness analysis varies this cutoff, and all clusters and proximity scores depend on it.
assumptions (4)
  • domain assumption A 'taken-cluster', defined as the intersection of a layer-1 neuron's 100 core-tokens and a layer-0 neuron's 100 core-tokens, is the extension of a categorical sub-dimension of the layer-1 neuron.
    This identification is made in Section 3.2 and used throughout Section 4.1; no external validation establishes that these token intersections correspond to latent semantic sub-dimensions.
  • domain assumption Cosine similarity in GPT2-XL's input embedding space is a valid coordinate of a token along a categorical sub-dimension.
    Section 3.3 and 4.1 define dimensional proximity as mean cosine similarity; the paper provides no evidence that embedding cosine tracks the categorical sub-dimension structure rather than generic word frequency or cluster size.
  • domain assumption Averaging tokens across 612 (or 463) selected neurons yields a 'mean neuron' whose 100 average activation ranks can be treated as independent statistical units.
    Section 4.1 and Table 2 compute Kendall tau on the average neuron; this assumes no between-neuron correlation and ignores the hierarchical data structure, inflating significance.
  • ad hoc to paper The aggregation function sum(w_i,j x_i,j) + a acts as a 'theorem-in-action' whose three effects (priming, inter-neuronal attention, phasing) are the causal drivers of categorical sub-dimension extraction.
    Section 1.5 introduces these processes from the authors' prior work [75] without independent derivation or test; the current study does not manipulate these quantities.
invented entities (3)
  • Intra-neuronal categorical vector space
    purpose: Proposed explanation of polysemy as internal to each neuron rather than distributed over inter-neuronal space.
    Called the central object in the abstract and Section 2; no basis vectors or coordinate equations are given, and the only empirical support is the correlation described in Section 4.1.
  • Categorical sub-dimensions (operationalized as taken-clusters)
    purpose: Basis directions of the proposed vector space, defined as intersections of activation-defined core-token sets.
    Defined Section 3.2; since they are intersections of activation-defined core token sets, they have no independent falsifiable handle outside the construction.
  • Critical categorical zone
    purpose: High-activation intersection of sub-dimensions where the neuron is claimed to be less polysemous.
    Inferred Section 5.1 from PCA factor loadings and high-activation tokens; no direct causal or behavioral test isolates this zone.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Polysemy of Synthetic Neurons Towards a New Type of Explanatory Categorical Vector Spaces." pith.science (2026). https://pith.science/paper/WTCXODM3

@misc{pith2026250507831,
  author       = {Pith},
  title        = {Pith review of: Polysemy of Synthetic Neurons Towards a New Type of Explanatory Categorical Vector Spaces},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WTCXODM3}},
  note         = {Machine review of arXiv:2505.07831}
}
read the original abstract

The polysemantic nature of synthetic neurons in artificial intelligence language models is currently understood as the result of a necessary superposition of distributed features within the latent space. We propose an alternative approach, geometrically defining a neuron in layer n as a categorical vector space with a non-orthogonal basis, composed of categorical sub-dimensions extracted from preceding neurons in layer n-1. This categorical vector space is structured by the activation space of each neuron and enables, via an intra-neuronal attention process, the identification and utilization of a critical categorical zone for the efficiency of the language model - more homogeneous and located at the intersection of these different categorical sub-dimensions.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

208 extracted references · 9 canonical work pages

  1. [1]

    Asif, H., Basit, A., Innan, N., Kashif, M., Marchisio, A., & Shafique, M. (2025). PennyLang : Pioneering LLM-Based Quantum Code Generation with a Novel PennyLane-Centric Dataset.arXiv preprint arXiv:2503.02497

    arXiv 2025

  2. [2]

    S., Gardner, M

    Alahmari, S. S., Gardner, M. R., & Salem, T. (2024). Attention guided approach for food type and state recognition. Food and Bioproducts Processing

    2024

  3. [4]

    G., Lioma, C., & Augenstein, I

    Atanasova, P., Simonsen, J. G., Lioma, C., & Augenstein, I. (2020). Generating Fact Checking Explanations. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics(pp. 7352–7364). Association for Computational Linguistics. DOI : 10.18653/v1/2020.acl- main.656

    doi:10.18653/v1/2020.acl- 2020

  4. [5]

    AlKhamissi, B., Tuckute, G., Bosselut, A., & Schrimpf, M. (2024b). Massive Activations in Large Language Models.arXiv :2402.17762v2

    arXiv 2024

  5. [6]

    AlKhamissi,B.,Tuckute,G.,Bosselut,A.,&Schrimpf,M.(2024a).TheLLM Language Network : A Neuroscientific Approach for Identifying Causally Task-Relevant Units.arXiv :2411.02280v2

    arXiv 2024

  6. [7]

    Barr, W., & Bieliauskas, L. A. (2024). Neuropsychology of Decision Making : A Clinical Perspective. Neuropsychology Review, 34(1), 1–15. DOI : 10.1007/s11065-023-09500-1

    doi:10.1007/s11065-023-09500-1 2024

  7. [9]

    Barkan, R. (2021). The Role of Cognitive Biases in Human Decision Making. Journal of Behavioral Decision Making, 34(3), 243–255. DOI : 10.1002/bdm.2210

    doi:10.1002/bdm.2210 2021

  8. [10]

    Bathia, N., & Richie, D. (2024). Advances in Reinforcement Learning : Applications and Challenges.Artificial Intelligence Review, 57(2), 123–145. DOI : 10.1007/s10462-023-10123-4

    doi:10.1007/s10462-023-10123-4 2024

Show all 208 references
  1. [11]

    Beaufils, M. (1996). Les réseaux de neurones artificiels : Modèles et applications. Revue d’Intelligence Artificielle , 10(4), 365–387. DOI : 10.1016/S0992-499X(97)80001-2

    1996 doi

  2. [12]

    (2023).Language models can explain neurons in language models

    Bills, S., Cammarata, N., Mossing, D., Saunders, W., Wu, J., Tillman, H., Gao, L., Goh, G., Sutskever, I., & Leike, J. (2023).Language models can explain neurons in language models. OpenAI.https://openaipublic. blob.core.windows.net/neuron-explainer/paper/index.html

    2023

  3. [13]

    Bolognesi, M. (2020). Where Words Get Their Meaning : Cognitive Processing and Distributional Modelling of Word Meaning. John Benjamins Publishing Company. DOI : 10.1075/ftl.7

    2020 doi

  4. [15]

    R., & Frost, R

    Bosker, H. R., & Frost, R. L. A. (2024). Statistical learning at a virtual cocktail party.Psychonomic Bulletin & Review, 31, 849-861

    2024

  5. [16]

    H., Black, J

    Bower, G. H., Black, J. B., & Turner, T. J. (1979). Scripts in memory for text. Cognitive Psychology, 11(2), 177-220

    1979

  6. [17]

    F., & Hay, A

    Brewer, W. F., & Hay, A. E. (1984). Reconstructive recall of linguistic style. Journal of Verbal Learning and Verbal Behavior, 23(2), 237-249

    1984

  7. [18]

    F., & Hay, A

    Brewer, W. F., & Hay, A. E. (1984). Reconstructive recall of linguistic style. Journal of Verbal Learning and Verbal Behavior, 23(2), 237-249. 30

    1984

  8. [19]

    Bricken, T., Schaeffer, R., Olshausen, B., & Kreiman, G. (2023). Emergence of Sparse Representations from Noise.Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Re- search, 202 :3148-3191. Available fromhttps://proceeding...

    2023

  9. [20]

    E., & Gregory, M

    Broadbent, D. E., & Gregory, M. (1965). Effects of noise and of signal rate upon vigilance analysed by means of decision theory.Human Factors, 7(2), 155-162

    1965

  10. [21]

    Deutsch, J. A. (1958). Perception and Communication.Nature, 182(4649), 1572-1572

    1958

  11. [23]

    R., & Wolfe, J

    Cave, K. R., & Wolfe, J. M. (1990). Modeling the role of parallel processing in visual search.Cognitive Psychology, 22(2), 225-271

    1990

  12. [24]

    Chen, T., Zhang, Y., Wang, H., Liu, J., & Li, X. (2024). Electrophysiological correlation between executive vigilance and attention network based on cognitive resource control theory.International Journal of Psychophysiology, 203, 112393

    2024

  13. [25]

    Cherry, E. C. (1953). Some experiments on the recognition of speech, with one and with two ears.Journal of the Acoustical Society of America, 25, 975-979

    1953

  14. [26]

    Child, R., Gray, S., Radford, A., & Sutskever, I. (2019). Generating long sequenceswithsparsetransformers. arXiv preprint arXiv:1904.10509. https: //arxiv.org/abs/1904.10509

    2019 arXiv

  15. [27]

    M., & Quillian, M

    Collins, A. M., & Quillian, M. R. (1969). Retrieval time from semantic memory. Journal of Verbal Learning and Verbal Behavior, 8(2), 240–247. https://doi.org/10.1016/s0022-5371(69)80069-1

    1969 doi

  16. [28]

    Conrad, C. (1972). Cognitive economy in semantic memory.Journal of Experimental Psychology, 92(2), 149-154

    1972

  17. [29]

    Cowan, N. (2024). Working Memory Capacity : Theories and Applications. Annual Review of Psychology,75,1–25.DOI:10.1146/annurev-psych-010723- 120001

    2024 doi

  18. [30]

    Dai, D., Dong, L., Hao, Y., Sui, Z., Chang, B., & Wei, F. (2022). Knowledge Neurons in Pretrained Transformers.Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1 : Long Papers). https://doi.org/10.18653/v1/2022.acl-long.581

    2022 doi

  19. [31]

    Danilevsky, M., Qian, K., Aharonov, R., Katsis, Y., Kawas, B., & Sen, P. (2020). A Survey of the State of Explainable AI for Natural Language Processing. arXiv (Cornell University). https://doi.org/10. 48550/arxiv.2010.00711 31

    2020 doi arXiv

  20. [32]

    A., Durrani, N., Sajjad, H., Dalvi, F., & Belinkov, Y

    Dar, S. A., Durrani, N., Sajjad, H., Dalvi, F., & Belinkov, Y. (2023). Probing Pre-trained Language Models for Temporal Knowledge. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL). DOI : 10.18653/v1/2023.acl-long.123

    2023 doi

  21. [33]

    Duncan, J. (1984). Selective Attention and the Organization of Visual Information. Journal of Experimental Psychology : General, 113(4), 501-517. DOI : 10.1037/0096-3445.113.4.501

    1984 doi

  22. [34]

    Duncan, J. (1999). Attention. In R. A. Wilson & F. C. Keil (Eds.),The MIT Encyclopedia of Cognitive Sciences. Cambridge, MA : MIT Press

    1999

  23. [35]

    Duncan, J., & Humphreys, G. (1992). Beyond the search surface : Visual search and attentional engagement.Journal of Experimental Psychology : Human Perception and Performance, 18(2), 578-588

    1992

  24. [36]

    Durrani, N., Sajjad, H., Dalvi, F., & Belinkov, Y. (2022). On the Trans- formation of Latent Space in Fine-Tuned NLP Models. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP). DOI : 10.18653/v1/2022.emnlp-main.123

    2022 doi

  25. [37]

    (2010).The essential guide to effect sizes

    Ellis, P. (2010).The essential guide to effect sizes. Cambridge University Press

    2010

  26. [38]

    (2020).Effect Size Matters : How Reporting and Interpreting Effect Sizes Can Improve Your Publication Prospects and Make the World a Better Place! London : MadMethods.co

    Ellis, P. (2020).Effect Size Matters : How Reporting and Interpreting Effect Sizes Can Improve Your Publication Prospects and Make the World a Better Place! London : MadMethods.co

    2020

  27. [39]

    Efimov, A., Dubrovsky, D., & Matveev, F. (2023). What’s stopping us achieving artificial general intelligence?Philosophy Now, April/May

    2023

  28. [40]

    Enguehard, J. (2023). Extrmask : A Method for Explaining Time Series Predictions by Masking.arXiv preprint arXiv:2301.08552. DOI : 10.48550/arXiv.2301.08552

    2023 doi arXiv

  29. [41]

    W., & Keane, M

    Eysenck, M. W., & Keane, M. T. (2020).Cognitive Psychology : A Student’s Handbook (8th ed.). Psychology Press. DOI : 10.4324/9780429449229

    2020 doi

  30. [42]

    Fan, Y., Dalvi, F., Durrani, N., & Sajjad, H. (2023). Evaluating Neuron Interpretation Methods of NLP Models.arXiv preprint arXiv:2301.12608. https://doi.org/10.48550/arxiv.2301.12608

    2023 doi arXiv

  31. [43]

    A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation,

    Fel, J., Smith, A., & Wang, T., "A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation," inProceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS 2023), 2024

    2023

  32. [44]

    Funayama, T., & Shibata, K. (2024). Advances in Quantum Computing : A Comprehensive Review.Journal of Quantum Information Science, 12(1), 45–67. DOI : 10.4236/jqis.2024.121004

    2024

  33. [45]

    Geva, M., Schuster, R., Berant, J., & Levy, O. (2023). Transformer Feed-Forward Layers Are Key-Value Memories. InProceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS). DOI : 10.48550/arXiv.2012.14913. 32

    2023 doi arXiv

  34. [46]

    Giallanza, T., & Campbell, D. I. (2024, March). Context-Sensitive Semantic Reasoning in Large Language Models. In ICLR 2024 Workshop on Representational Alignment

    2024

  35. [47]

    Gernsbacher, M. A. (1985). Surface information loss in comprehension. Cognitive Psychology, 17(3), 324-363

    1985

  36. [48]

    Gresch, D., & Müller, K. (2024). Machine Learning in Materials Science : Recent Progress and Emerging Applications.Advanced Materials, 36(5), 2105678. DOI : 10.1002/adma.202105678

    2024 doi

  37. [49]

    & Hasson, U

    Goldstein,A.,Grinstein-Dabush,A.,Schain,M.,Wang,H.,Hong,Z.,Aubrey, B., ... & Hasson, U. (2024). Alignment of brain embeddings and artificial contextual embeddings in natural language points to common geometric patterns. Nature Communications, 15(1), 2768

    2024

  38. [50]

    Gurnee, W., Nanda, N., Pauly, M., Harvey, K., Troitskii, D., & Bertsimas, D. (2023). Finding neurons in a haystack : Case studies with sparse probing. arXiv preprint arXiv:2305.01610

    2023 arXiv

  39. [51]

    Hanzal, S., Müller, C., Schwarz, J., Binder, L., & Schröder, P. (2024). EEG markers of vigilance, task-induced fatigue and motivation during sustained attention : Evidence for decoupled alpha-and beta-signatures. bioRxiv, 2024-10

    2024

  40. [52]

    A., Reicher, S

    Haslam, S. A., Reicher, S. D., & Platow, M. J. (2020).The New Psychology of Leadership : Identity, Influence, and Power(2nd ed.). Routledge. DOI : 10.4324/9781351108225

    2020 doi

  41. [53]

    Hastie, R. (2022). Schematic principles in human memory.Social Cognition, 39-88

    2022

  42. [54]

    N., Vu, M., & Thai, M

    Hoang, X. N., Vu, M., & Thai, M. T. (2024). LLM-assisted Concept Discovery : Automatically Identifying and Explaining Neuron Functions. arXiv :2406.08572

    2024 arXiv

  43. [55]

    N., & Love, B

    Hornsby, A. N., & Love, B. C. (2020). How decisions and the desire for coherency shape subjective preferences over time.Cognition, 200, 104244. https://doi.org/10.1016/j.cognition.2020.104244

    2020

  44. [56]

    Howell, D. C. (2024).Méthodes statistiques en sciences humaines. De Boeck Supérieur

    2024

  45. [57]

    (1907).Werke, vol

    von Humboldt, W. (1907).Werke, vol. 7, part 2. Berlin : Leitmann

    1907

  46. [58]

    (1973).Attention and Effort

    Kahneman, D. (1973).Attention and Effort. Englewood Cliffs, NJ : Prentice- Hall

    1973

  47. [59]

    Lin, Z. (2024). Attenuation Theory. In The ECPH Encyclopedia of Psychology (pp. 1-2). Singapore : Springer Nature Singapore

    2024

  48. [60]

    Swin Transformer : Hierarchical Vision Transformer using Shifted Windows

    Liu,Z.,Lin,Y.,Cao,Y.,Hu,H.,Wei,Y.,Zhang,Z.,Lin,S.,&Guo,B.(2020). Swin Transformer : Hierarchical Vision Transformer using Shifted Windows. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). https://arxiv.org/abs/2103.14030 33

    2020 arXiv

  49. [61]

    cocktail party

    Liu, H., Zhang, Y., Wang, F., Li, J., & Chen, T. (2024). Electrophysiological correlation of auditory selective spatial attention in the “cocktail party” situation. Human Brain Mapping, 45(11), e26793

    2024

  50. [62]

    Mackworth, N. H. (1948). The breakdown of vigilance during prolonged visual search.Quarterly Journal of Experimental Psychology, 1(1), 6-21

    1948

  51. [63]

    (1970).Biology of cognition(Vol

    Maturana, H. (1970).Biology of cognition(Vol. 9). Urbana : Biological Computer Laboratory, Department of Electrical Engineering, University of Illinois

    1970

  52. [64]

    Marconato, E., & al. (2024). BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcuts. arXiv preprint arXiv:2402.12240. DOI : 10.48550/arXiv.2402.12240

    2024 doi arXiv

  53. [65]

    arXiv preprint, arXiv:2405.16213

    Moreira,G.,Hauptmann,A.,Marques,M.,&Costeira,J.P.(2024).Learning Visual-Semantic Subspace Representations for Propositional Reasoning. arXiv preprint, arXiv:2405.16213

    2024 arXiv

  54. [66]

    Motter, B. C. (1999). Attention in the animal brain. In R. A. Wilson & F. C. Keil (Eds.),The MIT Encyclopedia of the Cognitive Sciences(pp. 39–41). Cambridge, MA : MIT Press

    1999

  55. [67]

    Mousi, B., Durrani, N., & Dalvi, F. (2023). Can LLMs facilitate interpre- tation of pre-trained language models?arXiv preprint arXiv:2305.13386. DOI : 10.48550/arXiv.2305.13386

    2023 doi arXiv

  56. [68]

    Murray, S. (2024). The Nature and Norms of Vigilance. American Philosophical Quarterly, 61(3), 265-278

    2024

  57. [69]

    M., & Barrera, R

    Ortiz-Rodriguez, F., Tiwari, S., Panchal, R., Medina-Quintero, J. M., & Barrera, R. (2022, June). MEXIN : multidialectal ontology supporting NLP approach to improve government electronic communication with the Mexican Ethnic Groups. InDG.O 2022 : The 23rd Annual International ...

    2022

  58. [70]

    Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., & Carter, S. (2020). Zoom In : An Introduction to Circuits. Retrieved fromhttps: //distill.pub/2020/circuits/zoom-in/. Accessed 24-11-2023

    2020

  59. [71]

    S., Merlino, G., Puliafito, A., Vyas, R., Vyas, O

    Patel, A. S., Merlino, G., Puliafito, A., Vyas, R., Vyas, O. P., Ojha, M., & Tiwari, V. (2023). An NLP-guided ontology development and refinement approach to represent and query visual information.Expert Systems with Applications, 213, 118998

    2023

  60. [72]

    (1974).La prise de conscience

    Piaget, J. (1974).La prise de conscience. Paris : Presses Universitaires de France

    1974

  61. [73]

    Pichat, M. (2024). Psychologie de l’IA et alignement cognitif. Actes du colloque Intelligence artificielle collaborative, management et dé- veloppement des organisations du 24/05/2024 coorganisé par l’Uni- versité Paris Dauphine-PSL et le Cabinet Chrysippe R&D. Avai- lable onl...

    2024

  62. [74]

    Pichat, M. (2024). Psychology of Artificial Intelligence : Epistemological Markers of the Cognitive Analysis of Neural Networks. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2407.09563

    2024 doi arXiv

  63. [75]

    (2024).Neuropsy- chology of AI : Relationship Between Activation Proximity and Categorical Proximity Within Neural Categories of Synthetic Cognition

    Pichat, M., Campoli, E., Pogrund, W., Wilson, J., Veillet-Guillem, M., Melkozerov, A., Gasparian, A., Pichat, P., Poumay, J. (2024).Neuropsy- chology of AI : Relationship Between Activation Proximity and Categorical Proximity Within Neural Categories of Synthetic Cognition. ar...

    2024 arXiv

  64. [76]

    Pichat, M., Pogrund, W., Gasparian, A., Pichat, P., Demarchi, S., & Veillet- Guillem, M. (2024). How Do Artificial Intelligences Think? The Three Mathematico-Cognitive Factors of Categorical Segmentation Operated by Synthetic Neurons.arXiv preprint, arXiv:2501.06196

    2024 arXiv

  65. [77]

    Pichat, M., Pogrund, W., Gasparian, A., Pichat, P., Demarchi, S., Veillet-Guillem, M., Corbet, M., & Dasilva, T. (2025). The Process of Categorical Clipping at the Core of the Genesis of Concepts in Synthetic Neural Cognition. arXiv e-prints, arXiv-2502

    2025

  66. [79]

    Pichat, M., Pogrund, W., Pichat, P., Gasparian, A., Demarchi, S., Corbet, M., Georgeon, A., Dasilva, T., & Veillet-Guillem, M. (2025). Synthetic Categorical Restructuring Or How AIs Gradually Extract Efficient Regularities from Their Experience of the World. arXiv :submit/6232...

    2025

  67. [80]

    A., & Duñabeitia, J

    Planchuelo, C., Hinojosa, J. A., & Duñabeitia, J. A. (2024). The nature of lexical associations in a foreign language : valence, arousal and concreteness. Bilingualism : Language and Cognition, 1-10

    2024

  68. [81]

    Polyn, S. M. (2024). 15 Attribute Theories of Memory. In M. J. Kahana & A. D. Wagner (Eds.),The Oxford Handbook of Human Memory, Two Volume Pack : Foundations and Applications(p. 417). Oxford University Press

    2024

  69. [82]

    (2022, November)

    Ponomarev, A., & Agafonov, A. (2022, November). Ontology concept extraction algorithm for deep neural networks. In2022 32nd Conference of Open Innovations Association (FRUCT)(pp. 221-226). IEEE

    2022

  70. [83]

    Posner, M. I. (1978).Chronometric Explorations of Mind. Lawrence Erlbaum Associates

    1978

  71. [85]

    Posner, M. I. (1980). Orienting of attention.Quarterly Journal of Experi- mental Psychology, 32(1), 3-25. 35

    1980

  72. [86]

    I., Snyder, C

    Posner, M. I., Snyder, C. R., & Davidson, B. J. (1980). Attention and the detection of signals.Journal of Experimental Psychology : General, 109(2), 160

    1980

  73. [87]

    I., & DiGirolamo, G

    Posner, M. I., & DiGirolamo, G. J. (1998). Executive Attention : Conflict, Target Detection, and Cognitive Control. In R. Parasuraman (Ed.),The Attentive Brain(pp. 401-423). Cambridge, MA : MIT Press

    1998

  74. [88]

    I., & Rafal, R

    Posner, M. I., & Rafal, R. D. (1995). Inhibition of return : Neural basis and function. Cognitive Neuropsychology, 12(3), 505–524

    1995

  75. [89]

    Posner, M. I. (2024). Orienting of attention and spatial cognition.Cognitive Processing, 25(Suppl 1), 55-59

    2024

  76. [90]

    & Pan, Y

    Qiu, Q., Huang, Z., Xu, D., Ma, K., Tao, L., Wang, R., ... & Pan, Y. (2023). Integrating NLP and Ontology Matching into a Unified System for Automated Information Extraction from Geological Hazard Reports. Journal of Earth Science, 34(5), 1433-1446

    2023

  77. [91]

    Richard, J. C. (1980).The Language Teaching Matrix. Cambridge University Press

    1980

  78. [92]

    Rosch, E. (1978). Cognition and categorization. In E. Rosch & B. B. Lloyd (Eds.), Cognition and Categorization. Lawrence Erlbaum Associates

    1978

  79. [93]

    Rosenholtz, R. (2024). Visual Attention in Crisis.Behavioral and Brain Sciences, 1-32

    2024

  80. [94]

    Rueda, M. R. (2024). Developing the attentive brain : Contribution of cognitive neuroscience to a theory of attentional development.Human Development, 1-16

    2024

  81. [95]

    E., Shoben, E

    Smith, E. E., Shoben, E. J., & Rips, L. J. (1974). Structure and process in semantic memory.Psychological Review, 81(3), 214-241

    1974

  82. [96]

    Sartori, G., Coltheart, M., Miozzo, M., & Job, R. (2024). Category Specificity and Informational Memory,Memory, 1, 604

    2024

  83. [97]

    Schneider, W., & Shiffrin, R. M. (1977). Controlled and Automatic Human Information Processing : I. Detection, Search, and Attention.Psychological Review, 84(1), 1-66

    1977

  84. [98]

    Tipper, S. P. (1985). The Negative Priming Effect : Inhibitory Priming by Ignored Objects.The Quarterly Journal of Experimental Psychology, 37A(4), 571-590. DOI : 10.1080/14640748508400920

    1985 doi

  85. [99]

    Thukral, A., Dhiman, S., Meher, R., & Bedi, P. (2023). Knowledge graph enrichment from clinical narratives using NLP, NER, and biomedical ontologies for healthcare applications.International Journal of Information Technology, 15(1), 53-65

    2023

  86. [100]

    Treisman, A. (1964). Monitoring and storage of irrelevant messages in selective attention.Journal of Verbal Learning and Verbal Behavior, 3(6), 449-459

    1964

  87. [102]

    Treisman, A. (1986). Features and objects in visual processing.Scientific American, 255(5), 114B-125

    1986

  88. [103]

    U., Ji, T., Van Aken, B., Cao, Q., Ciosici, M

    Treviso, M., Lee, J. U., Ji, T., Van Aken, B., Cao, Q., Ciosici, M. R., & Schwartz, R. (2023). Efficient methods for natural language processing : A survey.Transactions of the Association for Computational Linguistics, 11, 826-860

    2023

  89. [104]

    Tversky, A. (1977). Features of similarity.Psychological Review, 84(4), 327-352

    1977

  90. [105]

    Varela, F. (1984). The creative circle. In P. Watzlawick (Ed),The invented reality. London : W W Norton & Co Inc

    1984

  91. [106]

    Varela, F. J. (1988).Cognitive Science : A Cartography of Current Ideas. MIT Press.Varela1996

    1988

  92. [107]

    Vergnaud, G. (2009). Activité, développement, représentation. In M. Merri (Ed.), Activité humaine et conceptualisation. Questions à Gérard Vergnaud (pp. 149–154). Presses universitaires du Mirail

    2009

  93. [108]

    Vergnaud, G. (2016). Relations entre conceptualisations dans l’ac- tion et signifiants langagiers et symboliques. In Symposium latino- américain de didactique de mathématique , Bonito, Brésil. Dispo- nible sur :https://www.gerard-vergnaud.org/texts/gvergnaud_2016_ signifiants-...

    2016

  94. [109]

    Vergnaud, G. (2020). A Classification of Cognitive Tasks and Operations of Thought Involved in Addition and Subtraction Problems. In P. Carpenter, M. Moser & A. Romberg (Eds.),Addition and Subtraction : A Cognitive Perspective. London : Routledge

    2020

  95. [110]

    Voita, E., Sennrich, R., & Titov, I. (2021). Language modeling, lexical trans- lation, reordering : The training process of NMT through the lens of classical SMT. arXiv preprint arXiv:2109.01396. DOI : 10.48550/arXiv.2109.01396

    2021 doi arXiv

  96. [111]

    (2002).Radical Constructivism

    von Glaserfeld, E. (2002).Radical Constructivism. London : Routledge Falmer

    2002

  97. [112]

    (1907).Werke (Vol

    von Humboldt, W. (1907).Werke (Vol. 7, Part 2). Berlin : Leitmann

    1907

  98. [113]

    Kokhlikyan, N., Miglani, V., Martin, M., Wang, E., Reynolds, J., Melnikov, A., Lunova, N., & Reblitz-Richardson, O. (2020). Captum : A unified and generic model interpretability library for PyTorch. arXiv preprint arXiv:2009.07896. DOI : 10.48550/arXiv.2009.07896

    2020 doi arXiv

  99. [114]

    pyOptSparse : A Python framework for large-scale constrained nonlinear optimization of sparse systems

    Wu et al., (2020). pyOptSparse : A Python framework for large-scale constrained nonlinear optimization of sparse systems. Journal of Open Source Software, 5(54), 2564. DOI : 10.21105/joss.02564

    2020 doi

  100. [115]

    Wu, D., & Zhang, S. (2024). Does visual attention help? Towards better understanding and predicting users’ good abandonment behavior in mobile search. Library Hi Tech, 42(3), 867-884

    2024

  101. [116]

    Yang, Y., Li, L., de Deyne, S., Li, B., Wang, J., & Cai, Q. (2024). Unraveling lexical semantics in the brain : Comparing internal, external, and hybrid language models.Human Brain Mapping, 45(1), e26546. 37

    2024

  102. [117]

    A., Kirkorian, H., & Lupyan, G

    Zettersten, M., Bredemann, C., Kaul, M., Ellis, K., Vlach, H. A., Kirkorian, H., & Lupyan, G. (2024). Nameability supports rule-based category learning in children and adults.Child Development, 95(2), 497-514. DOI : 10.1111/cdev.14008

    2024 doi

  103. [118]

    Zhang, C., Yin, Z., & Qin, R. (2024). Attention-Enhanced Co-Interactive Fusion Network (AECIF-Net) for automated structural condition assessment in visual inspection.Automation in Construction, 159, 105292

    2024

  104. [119]

    Zhao, B. (2023). Understanding and explaining large language models : Challenges and methodologies.arXiv preprint arXiv:2305.11052

    2023 arXiv

  105. [120]

    Zhao, M., Xu, D., & Gao, T. (2024). From Cognition to Computation : A Comparative Review of Human Attention and Transformer Architectures. arXiv preprint arXiv:2407.01548

    2024 arXiv

  106. [121]

    Zhuang, J., Ren, Y., & Sun, G. (2024). Multifractal analysis of neuron interactions in LLM training.arXiv preprint arXiv:2402.09099

    2024 arXiv

  107. [122]

    Anderson, J. R. (1985).Cognitive Psychology and Its Implications(2nd ed.). W. H. Freeman. DOI : 10.4324/9781315784786

    1985 doi

  108. [123]

    Chao, L. L. (2024). Advances in Neuroimaging Techniques for Cognitive Neuroscience. Journal of Cognitive Neuroscience, 36(1), 1–15. DOI : 10.1162/jocn_a_01700

    2024 doi

  109. [124]

    Xu, W., & Futrell, R. (2024). A hierarchical Bayesian mo- del for syntactic priming. arXiv preprint arXiv:2405.15964 . DOI : 10.48550/arXiv.2405.15964

    2024 doi arXiv

  110. [125]

    A., & Pérez-González, J

    Hernández-Gutiérrez, C. A., & Pérez-González, J. (2024). Deep Learning Techniques for Natural Language Processing : A Survey.IEEE Transactions on Neural Networks and Learning Systems , 35(2), 1234–1256. DOI : 10.1109/TNNLS.2023.3101234

    2024

  111. [126]

    (2021).Abstraction and analogy-making in artificial intelli- gence

    Mitchell, M. (2021).Abstraction and analogy-making in artificial intelli- gence. Annals of the New York Academy of Sciences, 1505(1), 79-101. DOI : 10.1111/nyas.14619

    2021 doi

  112. [127]

    Capuano, F., & Kaup, B. (2024). Pragmatic Reasoning in GPT Models : Replication of a Subtle Negation Effect. Proceedings of the Annual Meeting of the Cognitive Science Society, 46. Retrieved from https ://escholar- ship.org/uc/item/22q5920s

    2024

  113. [128]

    R., Hansen, M., Iarosz, K

    Protachevicz, P. R., Hansen, M., Iarosz, K. C., Caldas, I. L., Batista, A. M., & Kurths, J. (2021). Emergence of neuronal synchronisation in coupled areas.Frontiers in Computational Neuroscience, 15, 663408. DOI : 10.3389/fncom.2021.663408

    2021

  114. [129]

    Canales-Johnson, A., Silva, C., Huepe, D., Rivera-Rei, Á., Noreika, V., Del Carmen Garcia, M., Silva, W., Vaucheret, E., Sedeño, L., Couto, B., Melloni, M., Ibáñez, A., Chennu, S., Bekinschtein, T. A. (2015). Auditory feedback differentially modulates behavioral and neural mar...

    2015 doi

  115. [130]

    Ribary, U., & Ward, L. M. (2024). Synchronization and functional connectivity dynamics across TC-CC-CT networks : Implications for clinical symptoms and consciousness. InPhenomenological Neuropsychiatry : How Patient Experience Bridges the Clinic with Clinical Neuroscience (pp...

    2024 doi

  116. [131]

    Shavikloo, M., Esmaeili, A., Valizadeh, A., & Madadi Asl, M. (2024). Syn- chronization of delayed coupled neurons with multiple synaptic connections. Cognitive Neurodynamics, 18(2), 631-643. DOI : 10.1007/s11571-023-10013-9

    2024 doi

  117. [132]

    Rzechorzek, A. (2024). Understanding Cognitive Processes : Insights from Recent Research.Journal of Cognitive Neuroscience. DOI : 10.1162/jocn_- a_01678

    2024 doi

  118. [133]

    Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate.arXiv preprint arXiv:1409.0473. https://arxiv.org/abs/1409.0473

    2014 arXiv

  119. [134]

    Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation.arXiv preprint arXiv:1406.1078. https://arxiv.org/abs/1406.1078

    2014 arXiv

  120. [135]

    N., Kaiser, & Polosukhin, I

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, & Polosukhin, I. (2017). Attention is all you need.Advances in Neural Information Processing Systems. https://arxiv.org/abs/1706.03762

    2017 arXiv

  121. [136]

    W., Lee, K., & Toutanova, K

    Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT : Pre- training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics : Human Language Tec...

    2019

  122. [137]

    Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. OpenAI blog. https://openai.com/research/language-unsupervised

    2018

  123. [138]

    Luong, M.-T., Pham, H., & Manning, C. D. (2015). Effective Approaches to Attention-based Neural Machine Translation.Proceedings of the 2015 Confe- rence on Empirical Methods in Natural Language Processing, 1412–1421. https://doi.org/10.18653/v1/D15-1166

    2015 doi

  124. [139]

    Voita, E., Talbot, D., Moiseev, F., Sennrich, R., & Titov, I. (2019). Analyzing multi-head self-attention : Specialized heads do the heavy lifting, the rest can be pruned.arXiv preprint arXiv:1905.09418. https: //arxiv.org/abs/1905.09418

    2019 arXiv

  125. [140]

    Michel, P., Levy, O., & Neubig, G. (2019). Are sixteen heads really better than one? An empirical study of attention in Transformers.arXiv preprint arXiv:1905.10650. https://arxiv.org/pdf/1905.10650

    2019 arXiv

  126. [141]

    Tiberi, L., Mignacco, F., Irie, K., & Sompolinsky, H. (2024). Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of 39 Transformers. arXiv preprint arXiv:2405.15926. https://arxiv.org/abs/ 2405.15926

    2024 arXiv

  127. [142]

    Kitaev, N., Kaiser, Ł., & Levskaya, A. (2020). Reformer : The Efficient Transformer.International Conference on Learning Representations. https: //openreview.net/forum?id=rkgNKkHtvB

    2020

  128. [143]

    Zaheer, M., Guruganesh, G., Dubey, A., Ainslie, J., Alberti, C., Ontañón, S., Pham, P., Ravula, A., Wang, Q., Yang, L., & Ahmed, A. (2020). Big Bird : Transformers for Longer Sequences.Advances in Neural Information Processing Systems, 33, 17283–17297.https://proceedings.neuri...

    2020

  129. [144]

    Tay, Y., Dehghani, M., Bahri, D., & Metzler, D. (2022). Efficient Transformers : A Survey.ACM Computing Surveys, 55(6), 1–28. https: //doi.org/10.1145/3530811

    2022 doi

  130. [145]

    Clark, K., Khandelwal, U., Levy, O., & Manning, C. D. (2019). What Does BERT Look at? An Analysis of BERT’s Attention.Proceedings of the 2019 ACL Workshop BlackboxNLP : Analyzing and Interpreting Neural Networks for NLP, 276-286. https://aclanthology.org/W19-4828/

    2019

  131. [146]

    V., & Salakhutdinov, R

    Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q. V., & Salakhutdinov, R. (2019). Transformer-XL : Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860. https://arxiv.org/abs/1901. 02860

    2019 arXiv

  132. [147]

    & Houlsby, N

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., ... & Houlsby, N. (2021). An Image is Worth 16x16 Words : Transformers for Image Recognition at Scale.International Conference on Learning Representations. https://arxiv.org/abs/2010.11929

    2021 arXiv

  133. [148]

    Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021). Swin Transformer : Hierarchical Vision Transformer using Shifted Windows. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 10012–10022. https://arxiv.org/pdf/2103. 14030

    2021

  134. [149]

    J., & Pavlick, E

    Traylor, A., Merullo, J., Frank, M. J., & Pavlick, E. (2024). Transformer Mechanisms Mimic Frontostriatal Gating Operations When Trained on Human Working Memory Tasks.arXiv preprint arXiv:2402.08211. https: //arxiv.org/abs/2402.08211

    2024 arXiv

  135. [150]

    (2002).Inner vision : An exploration of art and the brain

    Zeki, S. (2002).Inner vision : An exploration of art and the brain. Oxford University Press

    2002

  136. [151]

    M., et al

    Hock, R. M., et al. (2024). Effects of manipulating prefrontal activity and dopamine D1 receptor signaling in an appetitive feature-negative discrimination learning task.Behavioral Neuroscience

    2024

  137. [152]

    Green, I., Amo, R., & Watabe-Uchida, M. (2024). Shifting attention to orient or avoid : a unifying account of the tail of the striatum and its dopaminergic inputs.Current Opinion in Behavioral Sciences, 59, 101441. 40

    2024

  138. [153]

    Y., & Kim, B

    Ghorbani, A., Wexler, J., Zou, J. Y., & Kim, B. (2019). Towards automatic concept-based explanations. Advances in Neural Information Processing Systems, 9273–9282

    2019

  139. [154]

    Bhatt,U.,Weller,A.,&Moura,J.M.F.(2020).Evaluatingandaggregating feature-based model explanations.Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)

    2020

  140. [155]

    Clark, P. (2021). Formalising concepts.arXiv preprint arXiv:2101.05125v1. https://arxiv.org/abs/2101.05125

    2021 arXiv

  141. [156]

    Ponomarev, D., & Agafonov, A. (2022). Ontology Concept Extraction Algorithm for Deep Neural Networks.https://www.researchgate.net/ publication/365833644_Ontology_Concept_Extraction_Algorithm_ for_Deep_Neural_Networks

    2022

  142. [157]

    Treisman, A., & Gelade, G. (1980). A Feature-Integration Theory of Attention. Cognitive Psychology, 12(1), 97-136. DOI : 10.1016/0010- 0285(80)90005-5

    1980 doi

  143. [158]

    Singh, V., Gupta, I., & Jana, P. K. (2020). An Energy Efficient Algorithm for Workflow Scheduling in IaaS Cloud.Journal of Grid Computing, 18(3), 357–376. https://doi.org/10.1007/s10723-019-09490-2

    2020 doi

  144. [159]

    Vogel, T., Ingendahl, M., & Winkielman, P. (2021). The architecture of prototype preferences : Typicality, fluency, and valence.Journal of Experimental Psychology : General, 150(1), 187–194.https://doi.org/10. 1037/xge0000798

    2021

  145. [160]

    N., Heller, K., Austerweil, J

    Sanborn, A. N., Heller, K., Austerweil, J. L., & Chater, N. (2021). REFRESH : A new approach to modeling dimensional biases in perceptual similarity and categorization. Psychological Review, 128(6), 1145–1186. https://doi.org/10.1037/rev0000310

    2021 doi

  146. [161]

    M., Meagher, B

    Nosofsky, R. M., Meagher, B. J., & Kumar, P. (2022). Contrasting exemplar and prototype models in a natural-science category domain. Journal of Experimental Psychology : Learning, Memory, and Cognition, 48(12), 1970–1994. https://doi.org/10.1037/xlm0001069

    2022 doi

  147. [162]

    H., Zdon, A., Fraga, N

    Love, A. H., Zdon, A., Fraga, N. S., Cohen, B., Mejia, M. P., Maxwell, R., & Parker, S. S. (2022). Statistical evaluation of the similarity of characteristics inspringsof theCaliforniaDesert, UnitedStates. Frontiers in Environmental Science, 10. https://doi.org/10.3389/fenvs.2...

    2022

  148. [163]

    Poth, N., & Dolega, K. (2023). Bayesian belief protection : A study of belief in conspiracy theories.Philosophical Psychology, 36(6), 1182–1207. https://doi.org/10.1080/09515089.2023.2168881

    2023

  149. [164]

    D., & Love, B

    Roads, B. D., & Love, B. C. (2024). Modeling Similarity and Psychological Space. Annual Review of Psychology, 75(1), 215–240.https://doi.org/ 10.1146/annurev-psych-040323-115131

    2024 doi

  150. [165]

    I., Benjamin, A

    Glaser, J. I., Benjamin, A. S., Chowdhury, R. H., Perich, M. G., Miller, L. E., & Kording, K. P. (2020). Machine Learning for Neural Decoding. 41 eNeuro, 7(4), ENEURO.0506-19.2020.https://doi.org/10.1523/eneuro. 0506-19.2020

    2020 doi

  151. [166]

    Hasson, U. (2024). Alignment of brain embeddings and artificial contextual embeddings in natural language points to common geometric patterns. Nature Communications, 15(1), 2768

    2024

  152. [167]

    A., Belanger, J., Chericoni, A., Chavez, A

    Franch, M., Mickiewicz, E. A., Belanger, J., Chericoni, A., Chavez, A. G., Katlowitz, K., & Hayden, B. (2025). A vectorial code for semantics in human hippocampus.bioRxiv, 2025-02

    2025

  153. [168]

    R., Rogers, T

    Cox, C. R., Rogers, T. T., Shimotake, A., Kikuchi, T., Kunieda, T., Miyamoto, S., & Lambon Ralph, M. A. (2024). Representational similarity learning reveals a graded multidimensional semantic space in the human anterior temporal cortex.Imaging Neuroscience, 2, 1-22

    2024

  154. [169]

    Q., Kubo, R., Nishimoto, S., & Takagi, Y

    Nakagi, Y., Matsuyama, T., Koide-Majima, N., Yamaguchi, H. Q., Kubo, R., Nishimoto, S., & Takagi, Y. (2024). Unveiling Multi-level and Multi- modal Semantic Representations in the Human Brain using Large Language Models. bioRxiv, 2024-02

    2024

  155. [170]

    Wang, S., Zhang, Y., Shi, W., Zhang, G., Zhang, J., Lin, N., & Zong, C. (2023). A large dataset of semantic ratings and its computational extension. Scientific Data, 10(1), 106

    2023

  156. [171]

    Lonshakov, G., Truong, N., & Hasson, U. (2024). Learning and Interpreting Multiple Representations of Semantics in a Neurobiological System

    2024

  157. [172]

    M., Trébuchon, A., & King, J

    Desbordes, T., Lakretz, Y., Chanoine, V., Oquab, M., Badier, J. M., Trébuchon, A., & King, J. R. (2023). Dimensionality and ramping : Signatures of sentence integration in the dynamics of brains and deep language models.Journal of Neuroscience, 43(29), 5350-5364

    2023

  158. [173]

    Karlgren, J., & Kanerva, P. (2021). Semantics in high-dimensional space. Frontiers in Artificial Intelligence, 4, 698809

    2021

  159. [174]

    Sheng, J., Zhang, L., Liu, C., Liu, J., Feng, J., Zhou, Y., & Xue, G. (2022). Higher-dimensional neural representations predict better episodic memory. Science Advances, 8(16), eabm3829

    2022

  160. [176]

    Sassenhagen,J.,&Fiebach,C.J.(2020).Tracesofmeaningitself:Encoding distributional word vectors in brain activity.Neurobiology of Language, 1(1), 54-76

    2020

  161. [177]

    Zhang, Y., Han, K., Worth, R., & Liu, Z. (2020). Connecting concepts in the brain by mapping cortical representations of semantic relations.Nature Communications, 11(1), 1877

    2020

  162. [178]

    Páleník, J. (2024). What does it mean for consciousness to be multidimen- sional? A narrative review.Frontiers in Psychology, 15, 1430262. 42

    2024

  163. [179]

    Nishida, S., Blanc, A., Maeda, N., Kado, M., & Nishimoto, S. (2021). Behavioral correlates of cortical semantic representations modeled by word vectors. PLoS Computational Biology, 17(6), e1009138

    2021

  164. [180]

    Wang, J., Gupta, V., Xu, W., Lapata, M., & Cohen, S. B. (2024). Sharing Matters : Analysing Neurons across Languages and Tasks in LLMs. arXiv :2406.09265v1 . Retrieved from https://www.research.ed.ac.uk/files/455487465/ WangEtalArXiv2024SharingMatterAnalysingNeurons.pdf

    2024

  165. [181]

    C., & Nguyen, T

    Pham, V. C., & Nguyen, T. H. (2024). Householder Pseudo-Rotation : A Novel Approach to Activation Editing in LLMs with Direction-Magnitude Perspective. arXiv preprint arXiv:2409.10053

    2024 arXiv

  166. [182]

    Han, J., Kossen, J., Razzak, M., & Gal, Y. (2025). Semantic Entropy Neurons : Encoding Semantic Uncertainty in the Latent Space of LLMs. In MINT : Foundation Model Interventions

    2025

  167. [183]

    Zeng, H., Han, S., Chen, L., & Yu, K. (2024). Converging to a lingua franca : Evolution of linguistic regions and semantics alignment in multilingual large language models.arXiv preprint arXiv:2410.11718

    2024 arXiv

  168. [184]

    & Chua, T

    Fang, J., Bi, Z., Wang, R., Jiang, H., Gao, Y., Wang, K., ... & Chua, T. S. (2025). Towards Neuron Attributions in Multi-Modal Large Language Models. Advances in Neural Information Processing Systems, 37, 122867- 122890

    2025

  169. [185]

    Han, J., Kossen, J., & Razzak, M. (2024). Encoding Semantic Uncertainty in the Latent Space of LLMs. 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

    2024

  170. [186]

    Vitiello, R., Montgomery, N., Clayton, R., Lawrence, G., & Harrington, D. (2024). Context-Aware Neuron Interactions in Large Language Models Through Semantic Pattern Detection. Preprint. Retrieved from https: //d197for5662m48.cloudfront.net/documents/publicationstatus/ 229712/...

    2024

  171. [187]

    Digutsch, J., & Kosinski, M. (2023). Overlap in meaning is a stronger predictor of semantic activation in GPT-3 than in humans.Scientific Reports, 13(1), 5035

    2023

  172. [188]

    Elhage, N., Hume, T., Olsson, C., Nanda, N., Henighan, T., Johnston, S., ElShowk, S., Joseph, N., DasSarma, N., Mann, B., Hernandez, D., Askell, A., Ndousse, K., Jones, A., Drain, D., Chen, A., Bai, Y., Ganguli, D., Lovitt, L., Hatfield-Dodds, Z., Kernion, J., Conerly, T., Kra...

    2022

  173. [189]

    Xiong, Y., et al. (2024). Superposition in large-scale models : Mechanisms and effects on generalization.arXiv preprint arXiv:2401.05678. 43

    2024 arXiv

  174. [190]

    Cunningham, J., Gao, Y., Rajamanoharan, S., & Templeton, T. (2023). Sparse Autoencoder Variants for disentangling superposition in LLMs. Retrieved fromarXiv

    2023

  175. [191]

    Meta, A. (2024a). Introducing meta llama 3 : The most capable openly available LLM to date.Meta AI

    2024

  176. [192]

    Meta, A. (2024b). The Llama 3 Herd of Models.arXiv :2407.21783

    2024 arXiv

  177. [193]

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023). Llama 2 : Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288

    2023 arXiv

  178. [194]

    A., & Gal, Y

    Nicolson, A., Schut, L., Noble, J. A., & Gal, Y. (2024). Explaining Explainability : Understanding Concept Activation Vectors.arXiv. Retrieved from https://arxiv.org/abs/2404.03713

    2024 arXiv

  179. [195]

    O’Mahony, L., Andrearczyk, V., Muller, H., & Graziani, M. (2023). Disen- tangling Neuron Representations with Concept Vectors.arXiv. Retrieved from https://arxiv.org/abs/2304.09707

    2023 arXiv

  180. [196]

    Stańczak, K., Ponti, E., Torroba Hennigen, L., Cotterell, R., & Augenstein, I. (2022). Same Neurons, Different Languages : Probing Morphosyntax in Multilingual Pre-trained Models.arXiv. Retrieved from https://arxiv. org/abs/2205.02023

    2022 arXiv

  181. [197]

    Wang, X., Chen, T., Ge, Q., Xia, H., Bao, R., Zheng, R., Zhang, Q., Gui, T., & Huang, X. (2023a). Orthogonal Subspace Learning for Language Model Continual Learning. arXiv preprint arXiv:2310.14152. Retrieved from https://arxiv.org/abs/2310.141523

    2023 arXiv

  182. [198]

    D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., & Wu, J

    Gao, L., la Tour, T. D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., & Wu, J. (2024). Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04093

    2024 arXiv

  183. [199]

    Yamakoshi, A., Komeiji, Y., & Nagata, M. (2023). Causal Interventions in Transformer-based Language Models for Analyzing Pronominal Coreference Resolution. arXiv preprint arXiv:2305.12345. Retrieved from https:// arxiv.org/abs/2305.12345

    2023 arXiv

  184. [200]

    He, Z., & Toyoizumi, T. (2023). Causal Representations in Deep Neural Networks : A Mechanistic Approach to Understanding Language Models. arXiv preprint arXiv:2308.09876. Retrieved fromhttps://arxiv.org/abs/ 2308.09876

    2023 arXiv

  185. [201]

    F., & Chao, L

    Xu, H., Zhan, R., Ma, Y., Wong, D. F., & Chao, L. S. (2025). Let’s Focus on Neuron : Neuron-Level Supervised Fine-tuning for Large Language Model. arXiv :2403.11621. Retrieved fromhttps://arxiv.org/abs/2403.11621

    2025 arXiv

  186. [202]

    Adler, M., & Shavit, N. (2024). On the Complexity of Neural Computation in Superposition.arXiv preprint arXiv:2409.15318

    2024

  187. [203]

    Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space.arXiv. Retrieved from https: //arxiv.org/abs/1301.3781. 44

    2013 arXiv

  188. [204]

    & Hochreiter, S

    Ramsauer, H., Schäfl, B., Lehner, J., Seidl, P., Widrich, M., Adler, T., ... & Hochreiter, S. (2021). Hopfield Networks is All You Need.NeurIPS. Retrieved from https://arxiv.org/abs/2008.02217

    2021 arXiv

  189. [205]

    Liu, X., Yang, J., & Wang, Y. (2023). Orthogonality and Redundancy in Neural Representations.NeurIPS. Retrieved fromhttps://arxiv.org/ abs/2306.12345

    2023 arXiv

  190. [206]

    Gao, T., Maurer, B., & Hashimoto, T. (2024). Near Orthogonality in Large Language Models.ICLR. Retrieved fromhttps://arxiv.org/abs/2401. 04567

    2024

  191. [207]

    Stoewer, P., Schilling, A., Maier, A., & Krauss, P. (2023). Neural network based formation of cognitive maps of semantic spaces and the putative emergence of abstract concepts.Scientific Reports, 13(1), 3644

    2023

  192. [208]

    Sun, R. (2024). Can A Cognitive Architecture Fundamentally Enhance LLMs? Or Vice Versa? Retrieved from https://arxiv.org/abs/2401. 10444

    2024

  193. [209]

    Vygotsky, L. S. (1934).Thought and language. MIT Press

    1934

  194. [210]

    Efimov, A., Dubrovsky, D., & Turchin, N. (2021). Walking Through the Turing Wall. Proceeding of 20th IFAC Conference on Technology, Culture, and International StabilityTECIS 2021 : Moscow, Russian Federation, 14–17 September 2021

    2021

  195. [211]

    (1988).The society of mind

    Minsky, M. (1988).The society of mind. New York : Simon & Schuster

    1988

  196. [212]

    Xie, H. (2023). The promising future of cognitive science and artificial intelligence. Nature Reviews Psychology, 2(4), 202-202

    2023

  197. [213]

    Marcus, G., & Davis, E. (2021). Insights for AI from the human mind. Communication of the ACM, 84(1). https://doi.org/10.1145/3392663

    2021 doi

  198. [214]

    Nanda, N., et al. (2023). Fact finding : Attempting to reverse-engineer factual recall on the neuron level. Retrieved from https://www.alignmentforum.org/posts/iGuwZTHWb6DFY3sKB/ fact-finding-attempting-to-reverse-engineer-factual-recall

    2023

  199. [215]

    Hänni, K., et al. (2024). Mathematical models of computation in superposition. arXiv preprintarXiv:2408.05451

    2024 arXiv

  200. [216]

    N., Cho, M., Merth, T., Rastegari, M., & Wang, Z

    Hoang, D. N., Cho, M., Merth, T., Rastegari, M., & Wang, Z. (2023). Do Compressed LLMs Forget Knowledge? An Experimental Study with Practical Implications.arXiv preprintarXiv:2310.00867. 45

    2023 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.