The role of System 1 and System 2 semantic memory structure in human and LLM biases
Pith reviewed 2026-05-10 14:45 UTC · model grok-4.3
The pith
Human semantic memory networks show unique irreducible structures that link to lower implicit bias in deliberative thinking, a pattern absent in LLMs.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
We model System 1 and System 2 thinking as semantic memory networks with distinct structures, built from comparable datasets generated by both humans and LLMs. We find that semantic memory structures are irreducible only in humans, suggesting that LLMs lack certain types of human-like conceptual knowledge. Moreover, semantic memory structure relates consistently to implicit bias only in humans, with lower levels of bias in System 2 structures.
What carries the argument
Semantic memory networks built separately for System 1 and System 2 processes from human and LLM data, then measured with network metrics to test relations to implicit gender bias.
Load-bearing premise
That the networks extracted from human and LLM datasets truly capture the cognitive split between fast associative and slow deliberative thinking, and that the chosen network measures reflect the parts of structure relevant to bias.
What would settle it
Finding that LLMs prompted with different data or methods produce semantic networks with the same irreducible properties as human ones, or that bias scores in humans show no correlation with those network properties once other variables are controlled.
Figures
read the original abstract
Implicit biases in both humans and large language models (LLMs) pose significant societal risks. Dual process theories propose that biases arise primarily from associative System 1 thinking, while deliberative System 2 thinking mitigates bias, but the cognitive mechanisms that give rise to this phenomenon remain poorly understood. To better understand what underlies this duality in humans, and possibly in LLMs, we model System 1 and System 2 thinking as semantic memory networks with distinct structures, built from comparable datasets generated by both humans and LLMs. We then investigate how these distinct semantic memory structures relate to implicit gender bias using network-based evaluation metrics. We find that semantic memory structures are irreducible only in humans, suggesting that LLMs lack certain types of human-like conceptual knowledge. Moreover, semantic memory structure relates consistently to implicit bias only in humans, with lower levels of bias in System~2 structures. These findings suggest that certain types of conceptual knowledge contribute to bias regulation in humans, but not in LLMs, highlighting fundamental differences between human and machine cognition.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper models System 1 (associative) and System 2 (deliberative) thinking as semantic memory networks constructed from comparable human and LLM-generated datasets. It applies network-based metrics to examine how these structures relate to implicit gender bias, reporting that the structures are irreducible only in humans and that the structure-bias relation holds only for humans, with lower bias associated with System 2 networks. The authors conclude that certain conceptual knowledge supports bias regulation in humans but not in LLMs.
Significance. If the network constructions and metrics are shown to validly instantiate dual-process distinctions, the results would provide evidence of a qualitative divergence in how humans and LLMs organize conceptual knowledge and regulate bias. The network-analytic approach to linking memory topology with implicit bias is a potentially useful bridge between cognitive science and AI evaluation, though its interpretive power depends on the untested assumption that prompting regimes produce cognitively analogous structures.
major comments (3)
- [§3] §3 (Network Construction and Elicitation): The claim that LLM networks under different prompting conditions instantiate System 1 vs. System 2 distinctions analogous to humans is load-bearing for both the irreducibility and bias-correlation results, yet the manuscript provides no validation against established dual-process markers (e.g., response latency, explicit/implicit dissociation, or IAT scores) in the human data, nor demonstrates expected qualitative differences such as higher local clustering in System 1 networks.
- [Results] Results, bias-correlation analysis (around Tables 2–4): The reported finding that semantic memory structure relates to implicit bias only in humans rests on network metrics whose sensitivity to dataset size, density, or generation style is not controlled or reported; without these controls it is impossible to rule out that the human-only correlation is an artifact of differing network properties rather than a genuine cognitive difference.
- [§4] §4 (Irreducibility claim): The assertion that semantic memory structures are 'irreducible only in humans' requires a precise definition of irreducibility (e.g., via specific topological invariants or embedding dimensionality) and a demonstration that the same metrics applied to LLM networks do not simply reflect surface-level differences in output coherence; the current presentation leaves this distinction underspecified.
minor comments (2)
- [Abstract, §2] The abstract and §2 use 'System~2' LaTeX spacing inconsistently; standardize notation throughout.
- [Figures] Figure captions should explicitly state the number of participants/LLM generations and the exact prompting templates used to build each network.
Simulated Author's Rebuttal
We thank the referee for their constructive and detailed comments, which have prompted us to strengthen the methodological justifications and controls in the manuscript. We address each major comment below and indicate the revisions made.
read point-by-point responses
-
Referee: [§3] The claim that LLM networks under different prompting conditions instantiate System 1 vs. System 2 distinctions analogous to humans is load-bearing for both the irreducibility and bias-correlation results, yet the manuscript provides no validation against established dual-process markers (e.g., response latency, explicit/implicit dissociation, or IAT scores) in the human data, nor demonstrates expected qualitative differences such as higher local clustering in System 1 networks.
Authors: We agree that direct validation against markers such as response latency would provide stronger support for the analogy. The human dataset, based on established semantic memory elicitation protocols, does not contain latency or per-participant IAT data, so such validation is not possible with the current resources. In the revision we have expanded the methods section with a detailed theoretical mapping of the prompting regimes to dual-process theory. We have also added reporting of clustering coefficients and other local metrics, confirming higher clustering in human System 1 networks relative to System 2 (as predicted), with smaller differences observed in the LLM networks. These additions address the qualitative-difference concern while acknowledging the data limitation. revision: partial
-
Referee: [Results] The reported finding that semantic memory structure relates to implicit bias only in humans rests on network metrics whose sensitivity to dataset size, density, or generation style is not controlled or reported; without these controls it is impossible to rule out that the human-only correlation is an artifact of differing network properties rather than a genuine cognitive difference.
Authors: We accept that explicit controls are necessary. The revised manuscript now includes subsampling procedures that equalize network size and density across all conditions, together with sensitivity analyses that vary generation style. After these controls the human-specific structure-bias correlation remains statistically significant while the LLM correlation does not, indicating that the result is not an artifact of differing network properties. revision: yes
-
Referee: [§4] The assertion that semantic memory structures are 'irreducible only in humans' requires a precise definition of irreducibility (e.g., via specific topological invariants or embedding dimensionality) and a demonstration that the same metrics applied to LLM networks do not simply reflect surface-level differences in output coherence; the current presentation leaves this distinction underspecified.
Authors: We have revised §4 to supply an explicit operational definition: irreducibility is quantified by the persistence of higher-dimensional topological features (via persistent homology) that cannot be recovered from a lower-dimensional embedding without substantial loss of information. We further demonstrate that human networks retain higher irreducibility scores than LLM networks even after matching for coherence metrics and after comparison to randomized null models, indicating that the distinction is not reducible to surface-level output differences. revision: yes
- Direct empirical validation against response latency or explicit/implicit dissociation measures from the same human participants cannot be performed because the source dataset does not contain these variables.
Circularity Check
No circularity: empirical network construction and correlation analysis remain independent of inputs.
full rationale
The paper builds semantic memory networks from separate human and LLM datasets under different prompting conditions to represent System 1 versus System 2 structures, then applies network metrics to examine relations with implicit bias. No equations, fitted parameters, or self-citations are shown that would make the reported irreducibility or bias correlations reduce to the construction method by definition. The derivation chain consists of data generation, network extraction, metric computation, and statistical comparison; these steps do not collapse into self-definition or renaming of the input data. The central claims therefore retain independent empirical content.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Dual process theories accurately describe human cognition with System 1 as associative and System 2 as deliberative.
Reference graph
Works this paper leans on
-
[1]
K. Abramski, R. Improta, G. Rossetti, and M. Stella. The ”llm world of words” english free association norms generated by large language models.Scientific data, 12(1):803, 2025
work page 2025
-
[2]
K. Abramski, G. Rossetti, and M. Stella. A word association network methodology for evaluating implicit biases in llms compared to humans.arXiv preprint arXiv:2510.24488, 2025
-
[3]
A. Acerbi and J. M. Stubbersfield. Large language models show human-like content biases in transmission chain experiments.Proceedings of the National Academy of Sciences, 120(44):e2313790120, 2023
work page 2023
-
[4]
G. Agrawal, T. Kumarage, Z. Alghamdi, and H. Liu. Can knowledge graphs reduce hallucinations in llms?: A survey. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3947–3960, 2024
work page 2024
-
[5]
H. An, X. Liu, and D. Zhang. Learning bias-reduced word embeddings using dictionary definitions. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1139–1152, 2022
work page 2022
-
[6]
J. R. Anderson. A spreading activation theory of memory.Journal of verbal learning and verbal behavior, 22(3):261–295, 1983
work page 1983
-
[7]
J. R. Anderson, D. Bothell, C. Lebiere, and M. Matessa. An integrated theory of list memory.Journal of Memory and Language, 38(4):341–380, 1998
work page 1998
-
[8]
J. R. Anderson, M. Matessa, and C. Lebiere. Act-r: A theory of higher level cognition and its relation to visual attention.Human–Computer Interaction, 12(4):439–462, 1997
work page 1997
-
[9]
X. Bai, A. Wang, I. Sucholutsky, and T. L. Griffiths. Explicitly unbiased large language models still form biased associations.Proceedings of the National Academy of Sciences, 122(8):e2416228122, 2025
work page 2025
-
[10]
J. A. Bargh and T. L. Chartrand. The unbearable automaticity of being.American psychologist, 54(7):462, 1999
work page 1999
-
[11]
A. Barr, E. A. Feigenbaum, and P. R. Cohen.The handbook of artificial intelligence, volume 1. HeurisTech Press, 1981
work page 1981
-
[12]
E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell. On the dangers of stochastic parrots: Can language models be too big? InProceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623, 2021
work page 2021
-
[13]
S. Boccaletti, G. Bianconi, R. Criado, C. I. Del Genio, J. G´ omez-Gardenes, M. Romance, I. Sendina-Nadal, Z. Wang, and M. Zanin. The structure and dynamics of multilayer networks.Physics reports, 544(1):1–122, 2014
work page 2014
-
[14]
R. J. Brachman. On the epistemological status of semantic networks. InAssociative networks, pages 3–50. Elsevier, 1979
work page 1979
- [15]
-
[16]
M. Bursell and F. Olsson. Do we need dual-process theory to understand implicit bias? a study of the nature of implicit bias against muslims.Poetics, 87:101549, 2021
work page 2021
-
[17]
N. Castro and C. S. Siew. Contributions of modern network science to the cognitive sciences: Revisiting research spirals of representation and process.Proceedings of the Royal Society A, 476(2238):20190825, 2020. 27/31
work page 2020
-
[18]
S. Citraro, M. S. Vitevitch, M. Stella, and G. Rossetti. Feature-rich multiplex lexical networks reveal mental strategies of early language learning.Scientific Reports, 13(1):1474, 2023
work page 2023
-
[19]
A. M. Collins and E. F. Loftus. A spreading-activation theory of semantic processing.Psychological review, 82(6):407, 1975
work page 1975
-
[20]
A. M. Collins and M. R. Quillian. Retrieval time from semantic memory.Journal of verbal learning and verbal behavior, 8(2):240–247, 1969
work page 1969
-
[21]
S. De Deyne, D. J. Navarro, A. Perfors, M. Brysbaert, and G. Storms. The ”small world of words” english word association norms for over 12,000 cue words.Behavior research methods, 51(3):987–1006, 2019
work page 2019
-
[22]
M. De Domenico, V. Nicosia, A. Arenas, and V. Latora. Structural reducibility of multilayer networks. Nature communications, 6(1):6864, 2015
work page 2015
-
[23]
M. De Domenico, A. Sol´ e-Ribalta, E. Cozzo, M. Kivel¨ a, Y. Moreno, M. A. Porter, S. G´ omez, and A. Arenas. Mathematical formulation of multilayer networks.Physical Review X, 3(4):041022, 2013
work page 2013
-
[24]
E. S. De Duro, E. Franchino, R. Improta, G. A. Veltri, and M. Stella. Cognitive networks identify ai biases on societal issues in large language models.EPJ Data Science, 15(1):7, 2026
work page 2026
-
[25]
J. De Houwer, P. Van Dessel, and T. Moran. Attitudes as propositional representations.Trends in Cognitive Sciences, 25(10):870–882, 2021
work page 2021
-
[26]
J. C. de Winter, D. Dodou, and Y. B. Eisma. System 2 thinking in openai’s o1-preview model: Near-perfect performance on a mathematics exam.Computers, 13(11):278, 2024
work page 2024
-
[27]
P. G. Devine. Stereotypes and prejudice: Their automatic and controlled components.Journal of personality and social psychology, 56(1):5, 1989
work page 1989
-
[28]
P. G. Devine, P. S. Forscher, A. J. Austin, and W. T. Cox. Long-term reduction in implicit race bias: A prejudice habit-breaking intervention.Journal of experimental social psychology, 48(6):1267–1278, 2012
work page 2012
-
[29]
J. S. B. Evans. Dual-processing accounts of reasoning, judgment, and social cognition.Annu. Rev. Psychol., 59(1):255–278, 2008
work page 2008
-
[30]
J. S. B. Evans. Intuition and reasoning: A dual-process perspective.Psychological Inquiry, 21(4):313–326, 2010
work page 2010
-
[31]
J. S. B. Evans. Dual-process theories of reasoning: Contemporary issues and developmental applications. Developmental review, 31(2-3):86–102, 2011
work page 2011
-
[32]
J. S. B. Evans and K. E. Stanovich. Dual-process theories of higher cognition: Advancing the debate. Perspectives on psychological science, 8(3):223–241, 2013
work page 2013
-
[33]
J. Fodor.The language of thought. Harvard University Press, 1975
work page 1975
-
[34]
J. A. Fodor and Z. W. Pylyshyn. Connectionism and cognitive architecture: A critical analysis.Cognition, 28(1-2):3–71, 1988
work page 1988
-
[35]
A. Garimella, A. Amarnath, K. Kumar, A. P. Yalla, N. Chhaya, B. V. Srinivasan, et al. He is very intelligent, she is very beautiful? on mitigating social biases in language modelling and generation. In Findings of the association for computational linguistics: ACL-IJCNLP 2021, pages 4534–4545, 2021
work page 2021
-
[36]
B. Gawronski and G. V. Bodenhausen. Associative and propositional processes in evaluation: an integrative review of implicit and explicit attitude change.Psychological bulletin, 132(5):692, 2006
work page 2006
-
[37]
B. Gawronski and G. V. Bodenhausen. The associative–propositional evaluation model: Theory, evidence, and open questions.Advances in experimental social psychology, 44:59–127, 2011. 28/31
work page 2011
-
[38]
A. G. Greenwald and M. R. Banaji. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychological review, 102(1):4, 1995
work page 1995
-
[39]
A. G. Greenwald, D. E. McGhee, and J. L. Schwartz. Measuring individual differences in implicit cognition: the implicit association test.Journal of personality and social psychology, 74(6):1464, 1998
work page 1998
-
[40]
T. Hagendorff, S. Fabi, and M. Kosinski. Human-like intuitive behavior and reasoning biases emerged in large language models but disappeared in chatgpt.Nature Computational Science, 3(10):833–838, 2023
work page 2023
-
[41]
T. T. Hills and Y. N. Kenett. Is the mind a network? maps, vehicles, and skyhooks in cognitive network science.Topics in Cognitive Science, 14(1):189–208, 2022
work page 2022
-
[42]
K. A. Hutchison, D. A. Balota, J. H. Neely, M. J. Cortese, E. R. Cohen-Shikora, C.-S. Tse, M. J. Yap, J. J. Bengson, D. Niemeyer, and E. Buchanan. The semantic priming project.Behavior research methods, 45:1099–1114, 2013
work page 2013
-
[43]
Kahneman.Thinking, fast and slow
D. Kahneman.Thinking, fast and slow. macmillan, 2011
work page 2011
-
[44]
Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li
M. Kamruzzaman and G. L. Kim. Prompting techniques for reducing social bias in llms through system 1 and system 2 cognitive processes.arXiv preprint arXiv:2404.17218, 2024
-
[45]
M. Kaneko and D. Bollegala. Dictionary-based debiasing of pre-trained word embeddings.arXiv preprint arXiv:2101.09525, 2021
-
[46]
Kintsch.Comprehension: A paradigm for cognition
W. Kintsch.Comprehension: A paradigm for cognition. Cambridge university press, 1998
work page 1998
-
[47]
M. Kivel¨ a, A. Arenas, M. Barthelemy, J. P. Gleeson, Y. Moreno, and M. A. Porter. Multilayer networks. Journal of complex networks, 2(3):203–271, 2014
work page 2014
-
[48]
H. Kozima and T. Furugori. Similarity between words computed by spreading activation on an english dictionary.arXiv preprint cmp-lg/9601004, 1996
- [49]
-
[50]
Z.-Z. Li, D. Zhang, M.-L. Zhang, J. Zhang, Z. Liu, Y. Yao, H. Xu, J. Zheng, P.-J. Wang, X. Chen, et al. From system 1 to system 2: A survey of reasoning large language models.arXiv preprint arXiv:2502.17419, 2025
work page internal anchor Pith review arXiv 2025
-
[51]
C. Ma, T. Zhao, and M. Okumura. Debiasing large language models with structured knowledge. InFindings of the Association for Computational Linguistics: ACL 2024, pages 10274–10287, 2024
work page 2024
- [52]
-
[53]
G. A. Miller. Wordnet: a lexical database for english.Communications of the ACM, 38(11):39–41, 1995
work page 1995
-
[54]
M. J. Monteith. Self-regulation of prejudiced responses: Implications for progress in prejudice-reduction efforts.Journal of personality and social psychology, 65(3):469, 1993
work page 1993
-
[55]
M. J. Monteith, C. I. Voils, and L. Ashburn-Nardo. Taking a look underground: Detecting, interpreting, and reacting to implicit racial biases.Social Cognition, 19(4):395–417, 2001
work page 2001
-
[56]
A. Moors and J. De Houwer. Problems with dividing the realm of processes.Psychological Inquiry, 17(3):199–204, 2006
work page 2006
-
[57]
V. Mruthyunjaya, P. Pezeshkpour, E. Hruschka, and N. Bhutani. Rethinking language models as symbolic knowledge graphs.arXiv preprint arXiv:2308.13676, 2023. 29/31
-
[58]
B. A. Nosek, F. L. Smyth, J. J. Hansen, T. Devos, N. M. Lindner, K. A. Ranganath, C. T. Smith, K. R. Olson, D. Chugh, A. G. Greenwald, et al. Pervasiveness and correlates of implicit attitudes and stereotypes. European review of social psychology, 18(1):36–88, 2007
work page 2007
-
[59]
E. Pavlick. Symbols and grounding in large language models.Philosophical Transactions of the Royal Society A, 381(2251):20220041, 2023
work page 2023
-
[60]
B. K. Payne. Conceptualizing control in social cognition: how executive functioning modulates the expression of automatic stereotyping.Journal of personality and social psychology, 89(4):488, 2005
work page 2005
- [61]
-
[62]
L. M. Reder and J. R. Anderson. A partial resolution of the paradox of interference: The role of integrating knowledge.Cognitive Psychology, 12(4):447–472, 1980
work page 1980
-
[63]
D. E. Rumelhart, J. L. McClelland, P. R. Group, et al.Parallel distributed processing, volume 1: Explorations in the microstructure of cognition: Foundations. The MIT press, 1986
work page 1986
-
[64]
C. S. Siew. spreadr: An r package to simulate spreading activation in a network.Behavior Research Methods, 51(2):910–929, 2019
work page 2019
-
[65]
C. S. Siew, D. U. Wulff, N. M. Beckage, and Y. N. Kenett. Cognitive network science: A review of research on cognition through the lens of network representations, processes, and dynamics.Complexity, 2019, 2019
work page 2019
-
[66]
H. A. Simon and A. Newell. Human problem solving: The state of the theory in 1970.American psychologist, 26(2):145, 1971
work page 1970
-
[67]
S. A. Sloman. The empirical case for two systems of reasoning.Psychological bulletin, 119(1):3, 1996
work page 1996
-
[68]
E. E. Smith, E. J. Shoben, and L. J. Rips. Structure and process in semantic memory: A featural model for semantic decisions.Psychological review, 81(3):214, 1974
work page 1974
-
[69]
E. R. Smith and J. DeCoster. Dual-process models in social and cognitive psychology: Conceptual integration and links to underlying memory systems.Personality and social psychology review, 4(2):108–131, 2000
work page 2000
- [70]
-
[71]
J. F. Sowa. Generating language from conceptual graphs. InComputational Linguistics, pages 29–43. Elsevier, 1983
work page 1983
-
[72]
K. E. Stanovich.Who is rational?: Studies of individual differences in reasoning. Psychology Press, 1999
work page 1999
- [73]
-
[74]
M. Steyvers and J. B. Tenenbaum. The large-scale structure of semantic networks: Statistical analyses and a model of semantic growth.Cognitive science, 29(1):41–78, 2005
work page 2005
-
[75]
F. Strack and R. Deutsch. Reflective and impulsive determinants of social behavior.Personality and social psychology review, 8(3):220–247, 2004
work page 2004
-
[76]
P. Vincent-Lamarre, A. B. Mass´ e, M. Lopes, M. Lord, O. Marcotte, and S. Harnad. The latent structure of dictionaries.Topics in cognitive science, 8(3):625–659, 2016
work page 2016
-
[77]
J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, et al. Emergent abilities of large language models.arXiv preprint arXiv:2206.07682, 2022. 30/31
work page internal anchor Pith review arXiv 2022
-
[78]
J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in Neural Information Processing Systems, 35:24824–24837, 2022
work page 2022
-
[79]
W. A. Woods. What’s in a link: Foundations for semantic networks. InRepresentation and understanding, pages 35–82. Elsevier, 1975
work page 1975
-
[80]
Towards system 2 reasoning in llms: Learning how to think with meta chain-of-thought, 2025
V. Xiang, C. Snell, K. Gandhi, A. Albalak, A. Singh, C. Blagden, D. Phung, R. Rafailov, N. Lile, D. Mahan, et al. Towards system 2 reasoning in llms: Learning how to think with meta chain-of-thought.arXiv preprint arXiv:2501.04682, 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.