Recognition: unknown
The role of System 1 and System 2 semantic memory structure in human and LLM biases
Pith reviewed 2026-05-10 14:45 UTC · model grok-4.3
The pith
Human semantic memory networks show unique irreducible structures that link to lower implicit bias in deliberative thinking, a pattern absent in LLMs.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
We model System 1 and System 2 thinking as semantic memory networks with distinct structures, built from comparable datasets generated by both humans and LLMs. We find that semantic memory structures are irreducible only in humans, suggesting that LLMs lack certain types of human-like conceptual knowledge. Moreover, semantic memory structure relates consistently to implicit bias only in humans, with lower levels of bias in System 2 structures.
What carries the argument
Semantic memory networks built separately for System 1 and System 2 processes from human and LLM data, then measured with network metrics to test relations to implicit gender bias.
Load-bearing premise
That the networks extracted from human and LLM datasets truly capture the cognitive split between fast associative and slow deliberative thinking, and that the chosen network measures reflect the parts of structure relevant to bias.
What would settle it
Finding that LLMs prompted with different data or methods produce semantic networks with the same irreducible properties as human ones, or that bias scores in humans show no correlation with those network properties once other variables are controlled.
Figures
read the original abstract
Implicit biases in both humans and large language models (LLMs) pose significant societal risks. Dual process theories propose that biases arise primarily from associative System 1 thinking, while deliberative System 2 thinking mitigates bias, but the cognitive mechanisms that give rise to this phenomenon remain poorly understood. To better understand what underlies this duality in humans, and possibly in LLMs, we model System 1 and System 2 thinking as semantic memory networks with distinct structures, built from comparable datasets generated by both humans and LLMs. We then investigate how these distinct semantic memory structures relate to implicit gender bias using network-based evaluation metrics. We find that semantic memory structures are irreducible only in humans, suggesting that LLMs lack certain types of human-like conceptual knowledge. Moreover, semantic memory structure relates consistently to implicit bias only in humans, with lower levels of bias in System~2 structures. These findings suggest that certain types of conceptual knowledge contribute to bias regulation in humans, but not in LLMs, highlighting fundamental differences between human and machine cognition.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper models System 1 (associative) and System 2 (deliberative) thinking as semantic memory networks constructed from comparable human and LLM-generated datasets. It applies network-based metrics to examine how these structures relate to implicit gender bias, reporting that the structures are irreducible only in humans and that the structure-bias relation holds only for humans, with lower bias associated with System 2 networks. The authors conclude that certain conceptual knowledge supports bias regulation in humans but not in LLMs.
Significance. If the network constructions and metrics are shown to validly instantiate dual-process distinctions, the results would provide evidence of a qualitative divergence in how humans and LLMs organize conceptual knowledge and regulate bias. The network-analytic approach to linking memory topology with implicit bias is a potentially useful bridge between cognitive science and AI evaluation, though its interpretive power depends on the untested assumption that prompting regimes produce cognitively analogous structures.
major comments (3)
- [§3] §3 (Network Construction and Elicitation): The claim that LLM networks under different prompting conditions instantiate System 1 vs. System 2 distinctions analogous to humans is load-bearing for both the irreducibility and bias-correlation results, yet the manuscript provides no validation against established dual-process markers (e.g., response latency, explicit/implicit dissociation, or IAT scores) in the human data, nor demonstrates expected qualitative differences such as higher local clustering in System 1 networks.
- [Results] Results, bias-correlation analysis (around Tables 2–4): The reported finding that semantic memory structure relates to implicit bias only in humans rests on network metrics whose sensitivity to dataset size, density, or generation style is not controlled or reported; without these controls it is impossible to rule out that the human-only correlation is an artifact of differing network properties rather than a genuine cognitive difference.
- [§4] §4 (Irreducibility claim): The assertion that semantic memory structures are 'irreducible only in humans' requires a precise definition of irreducibility (e.g., via specific topological invariants or embedding dimensionality) and a demonstration that the same metrics applied to LLM networks do not simply reflect surface-level differences in output coherence; the current presentation leaves this distinction underspecified.
minor comments (2)
- [Abstract, §2] The abstract and §2 use 'System~2' LaTeX spacing inconsistently; standardize notation throughout.
- [Figures] Figure captions should explicitly state the number of participants/LLM generations and the exact prompting templates used to build each network.
Simulated Author's Rebuttal
We thank the referee for their constructive and detailed comments, which have prompted us to strengthen the methodological justifications and controls in the manuscript. We address each major comment below and indicate the revisions made.
read point-by-point responses
-
Referee: [§3] The claim that LLM networks under different prompting conditions instantiate System 1 vs. System 2 distinctions analogous to humans is load-bearing for both the irreducibility and bias-correlation results, yet the manuscript provides no validation against established dual-process markers (e.g., response latency, explicit/implicit dissociation, or IAT scores) in the human data, nor demonstrates expected qualitative differences such as higher local clustering in System 1 networks.
Authors: We agree that direct validation against markers such as response latency would provide stronger support for the analogy. The human dataset, based on established semantic memory elicitation protocols, does not contain latency or per-participant IAT data, so such validation is not possible with the current resources. In the revision we have expanded the methods section with a detailed theoretical mapping of the prompting regimes to dual-process theory. We have also added reporting of clustering coefficients and other local metrics, confirming higher clustering in human System 1 networks relative to System 2 (as predicted), with smaller differences observed in the LLM networks. These additions address the qualitative-difference concern while acknowledging the data limitation. revision: partial
-
Referee: [Results] The reported finding that semantic memory structure relates to implicit bias only in humans rests on network metrics whose sensitivity to dataset size, density, or generation style is not controlled or reported; without these controls it is impossible to rule out that the human-only correlation is an artifact of differing network properties rather than a genuine cognitive difference.
Authors: We accept that explicit controls are necessary. The revised manuscript now includes subsampling procedures that equalize network size and density across all conditions, together with sensitivity analyses that vary generation style. After these controls the human-specific structure-bias correlation remains statistically significant while the LLM correlation does not, indicating that the result is not an artifact of differing network properties. revision: yes
-
Referee: [§4] The assertion that semantic memory structures are 'irreducible only in humans' requires a precise definition of irreducibility (e.g., via specific topological invariants or embedding dimensionality) and a demonstration that the same metrics applied to LLM networks do not simply reflect surface-level differences in output coherence; the current presentation leaves this distinction underspecified.
Authors: We have revised §4 to supply an explicit operational definition: irreducibility is quantified by the persistence of higher-dimensional topological features (via persistent homology) that cannot be recovered from a lower-dimensional embedding without substantial loss of information. We further demonstrate that human networks retain higher irreducibility scores than LLM networks even after matching for coherence metrics and after comparison to randomized null models, indicating that the distinction is not reducible to surface-level output differences. revision: yes
- Direct empirical validation against response latency or explicit/implicit dissociation measures from the same human participants cannot be performed because the source dataset does not contain these variables.
Circularity Check
No circularity: empirical network construction and correlation analysis remain independent of inputs.
full rationale
The paper builds semantic memory networks from separate human and LLM datasets under different prompting conditions to represent System 1 versus System 2 structures, then applies network metrics to examine relations with implicit bias. No equations, fitted parameters, or self-citations are shown that would make the reported irreducibility or bias correlations reduce to the construction method by definition. The derivation chain consists of data generation, network extraction, metric computation, and statistical comparison; these steps do not collapse into self-definition or renaming of the input data. The central claims therefore retain independent empirical content.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Dual process theories accurately describe human cognition with System 1 as associative and System 2 as deliberative.
Reference graph
Works this paper leans on
-
[1]
Abramski, R
K. Abramski, R. Improta, G. Rossetti, and M. Stella. The ”llm world of words” english free association norms generated by large language models.Scientific data, 12(1):803, 2025
2025
-
[2]
K. Abramski, G. Rossetti, and M. Stella. A word association network methodology for evaluating implicit biases in llms compared to humans.arXiv preprint arXiv:2510.24488, 2025
-
[3]
Acerbi and J
A. Acerbi and J. M. Stubbersfield. Large language models show human-like content biases in transmission chain experiments.Proceedings of the National Academy of Sciences, 120(44):e2313790120, 2023
2023
-
[4]
Agrawal, T
G. Agrawal, T. Kumarage, Z. Alghamdi, and H. Liu. Can knowledge graphs reduce hallucinations in llms?: A survey. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3947–3960, 2024
2024
-
[5]
H. An, X. Liu, and D. Zhang. Learning bias-reduced word embeddings using dictionary definitions. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1139–1152, 2022
2022
-
[6]
J. R. Anderson. A spreading activation theory of memory.Journal of verbal learning and verbal behavior, 22(3):261–295, 1983
1983
-
[7]
J. R. Anderson, D. Bothell, C. Lebiere, and M. Matessa. An integrated theory of list memory.Journal of Memory and Language, 38(4):341–380, 1998
1998
-
[8]
J. R. Anderson, M. Matessa, and C. Lebiere. Act-r: A theory of higher level cognition and its relation to visual attention.Human–Computer Interaction, 12(4):439–462, 1997
1997
-
[9]
X. Bai, A. Wang, I. Sucholutsky, and T. L. Griffiths. Explicitly unbiased large language models still form biased associations.Proceedings of the National Academy of Sciences, 122(8):e2416228122, 2025
2025
-
[10]
J. A. Bargh and T. L. Chartrand. The unbearable automaticity of being.American psychologist, 54(7):462, 1999
1999
-
[11]
A. Barr, E. A. Feigenbaum, and P. R. Cohen.The handbook of artificial intelligence, volume 1. HeurisTech Press, 1981
1981
-
[12]
E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell. On the dangers of stochastic parrots: Can language models be too big? InProceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623, 2021
2021
-
[13]
Boccaletti, G
S. Boccaletti, G. Bianconi, R. Criado, C. I. Del Genio, J. G´ omez-Gardenes, M. Romance, I. Sendina-Nadal, Z. Wang, and M. Zanin. The structure and dynamics of multilayer networks.Physics reports, 544(1):1–122, 2014
2014
-
[14]
R. J. Brachman. On the epistemological status of semantic networks. InAssociative networks, pages 3–50. Elsevier, 1979
1979
-
[15]
Brady, P
O. Brady, P. Nulty, L. Zhang, T. E. Ward, and D. P. McGovern. Dual-process theory and decision-making in large language models.Nature Reviews Psychology, pages 1–16, 2025
2025
-
[16]
Bursell and F
M. Bursell and F. Olsson. Do we need dual-process theory to understand implicit bias? a study of the nature of implicit bias against muslims.Poetics, 87:101549, 2021
2021
-
[17]
Castro and C
N. Castro and C. S. Siew. Contributions of modern network science to the cognitive sciences: Revisiting research spirals of representation and process.Proceedings of the Royal Society A, 476(2238):20190825, 2020. 27/31
2020
-
[18]
Citraro, M
S. Citraro, M. S. Vitevitch, M. Stella, and G. Rossetti. Feature-rich multiplex lexical networks reveal mental strategies of early language learning.Scientific Reports, 13(1):1474, 2023
2023
-
[19]
A. M. Collins and E. F. Loftus. A spreading-activation theory of semantic processing.Psychological review, 82(6):407, 1975
1975
-
[20]
A. M. Collins and M. R. Quillian. Retrieval time from semantic memory.Journal of verbal learning and verbal behavior, 8(2):240–247, 1969
1969
-
[21]
De Deyne, D
S. De Deyne, D. J. Navarro, A. Perfors, M. Brysbaert, and G. Storms. The ”small world of words” english word association norms for over 12,000 cue words.Behavior research methods, 51(3):987–1006, 2019
2019
-
[22]
De Domenico, V
M. De Domenico, V. Nicosia, A. Arenas, and V. Latora. Structural reducibility of multilayer networks. Nature communications, 6(1):6864, 2015
2015
-
[23]
De Domenico, A
M. De Domenico, A. Sol´ e-Ribalta, E. Cozzo, M. Kivel¨ a, Y. Moreno, M. A. Porter, S. G´ omez, and A. Arenas. Mathematical formulation of multilayer networks.Physical Review X, 3(4):041022, 2013
2013
-
[24]
E. S. De Duro, E. Franchino, R. Improta, G. A. Veltri, and M. Stella. Cognitive networks identify ai biases on societal issues in large language models.EPJ Data Science, 15(1):7, 2026
2026
-
[25]
De Houwer, P
J. De Houwer, P. Van Dessel, and T. Moran. Attitudes as propositional representations.Trends in Cognitive Sciences, 25(10):870–882, 2021
2021
-
[26]
J. C. de Winter, D. Dodou, and Y. B. Eisma. System 2 thinking in openai’s o1-preview model: Near-perfect performance on a mathematics exam.Computers, 13(11):278, 2024
2024
-
[27]
P. G. Devine. Stereotypes and prejudice: Their automatic and controlled components.Journal of personality and social psychology, 56(1):5, 1989
1989
-
[28]
P. G. Devine, P. S. Forscher, A. J. Austin, and W. T. Cox. Long-term reduction in implicit race bias: A prejudice habit-breaking intervention.Journal of experimental social psychology, 48(6):1267–1278, 2012
2012
-
[29]
J. S. B. Evans. Dual-processing accounts of reasoning, judgment, and social cognition.Annu. Rev. Psychol., 59(1):255–278, 2008
2008
-
[30]
J. S. B. Evans. Intuition and reasoning: A dual-process perspective.Psychological Inquiry, 21(4):313–326, 2010
2010
-
[31]
J. S. B. Evans. Dual-process theories of reasoning: Contemporary issues and developmental applications. Developmental review, 31(2-3):86–102, 2011
2011
-
[32]
J. S. B. Evans and K. E. Stanovich. Dual-process theories of higher cognition: Advancing the debate. Perspectives on psychological science, 8(3):223–241, 2013
2013
-
[33]
Fodor.The language of thought
J. Fodor.The language of thought. Harvard University Press, 1975
1975
-
[34]
J. A. Fodor and Z. W. Pylyshyn. Connectionism and cognitive architecture: A critical analysis.Cognition, 28(1-2):3–71, 1988
1988
-
[35]
Garimella, A
A. Garimella, A. Amarnath, K. Kumar, A. P. Yalla, N. Chhaya, B. V. Srinivasan, et al. He is very intelligent, she is very beautiful? on mitigating social biases in language modelling and generation. In Findings of the association for computational linguistics: ACL-IJCNLP 2021, pages 4534–4545, 2021
2021
-
[36]
Gawronski and G
B. Gawronski and G. V. Bodenhausen. Associative and propositional processes in evaluation: an integrative review of implicit and explicit attitude change.Psychological bulletin, 132(5):692, 2006
2006
-
[37]
Gawronski and G
B. Gawronski and G. V. Bodenhausen. The associative–propositional evaluation model: Theory, evidence, and open questions.Advances in experimental social psychology, 44:59–127, 2011. 28/31
2011
-
[38]
A. G. Greenwald and M. R. Banaji. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychological review, 102(1):4, 1995
1995
-
[39]
A. G. Greenwald, D. E. McGhee, and J. L. Schwartz. Measuring individual differences in implicit cognition: the implicit association test.Journal of personality and social psychology, 74(6):1464, 1998
1998
-
[40]
Hagendorff, S
T. Hagendorff, S. Fabi, and M. Kosinski. Human-like intuitive behavior and reasoning biases emerged in large language models but disappeared in chatgpt.Nature Computational Science, 3(10):833–838, 2023
2023
-
[41]
T. T. Hills and Y. N. Kenett. Is the mind a network? maps, vehicles, and skyhooks in cognitive network science.Topics in Cognitive Science, 14(1):189–208, 2022
2022
-
[42]
K. A. Hutchison, D. A. Balota, J. H. Neely, M. J. Cortese, E. R. Cohen-Shikora, C.-S. Tse, M. J. Yap, J. J. Bengson, D. Niemeyer, and E. Buchanan. The semantic priming project.Behavior research methods, 45:1099–1114, 2013
2013
-
[43]
Kahneman.Thinking, fast and slow
D. Kahneman.Thinking, fast and slow. macmillan, 2011
2011
-
[44]
Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jingtao Ding, Zhilun Zhou, Fengli Xu, and Yong Li
M. Kamruzzaman and G. L. Kim. Prompting techniques for reducing social bias in llms through system 1 and system 2 cognitive processes.arXiv preprint arXiv:2404.17218, 2024
-
[45]
M. Kaneko and D. Bollegala. Dictionary-based debiasing of pre-trained word embeddings.arXiv preprint arXiv:2101.09525, 2021
-
[46]
Kintsch.Comprehension: A paradigm for cognition
W. Kintsch.Comprehension: A paradigm for cognition. Cambridge university press, 1998
1998
-
[47]
Kivel¨ a, A
M. Kivel¨ a, A. Arenas, M. Barthelemy, J. P. Gleeson, Y. Moreno, and M. A. Porter. Multilayer networks. Journal of complex networks, 2(3):203–271, 2014
2014
-
[48]
H. Kozima and T. Furugori. Similarity between words computed by spreading activation on an english dictionary.arXiv preprint cmp-lg/9601004, 1996
-
[49]
Kumar, H
R. Kumar, H. Kumar, and K. Shalini. Detecting and mitigating bias in llms through knowledge graph- augmented training. In2025 International Conference on Artificial Intelligence and Data Engineering (AIDE), pages 608–613. IEEE, 2025
2025
-
[50]
Z.-Z. Li, D. Zhang, M.-L. Zhang, J. Zhang, Z. Liu, Y. Yao, H. Xu, J. Zheng, P.-J. Wang, X. Chen, et al. From system 1 to system 2: A survey of reasoning large language models.arXiv preprint arXiv:2502.17419, 2025
work page internal anchor Pith review arXiv 2025
-
[51]
C. Ma, T. Zhao, and M. Okumura. Debiasing large language models with structured knowledge. InFindings of the Association for Computational Linguistics: ACL 2024, pages 10274–10287, 2024
2024
-
[52]
McRae, G
K. McRae, G. S. Cree, M. S. Seidenberg, and C. McNorgan. Semantic feature production norms for a large set of living and nonliving things.Behavior research methods, 37(4):547–559, 2005
2005
-
[53]
G. A. Miller. Wordnet: a lexical database for english.Communications of the ACM, 38(11):39–41, 1995
1995
-
[54]
M. J. Monteith. Self-regulation of prejudiced responses: Implications for progress in prejudice-reduction efforts.Journal of personality and social psychology, 65(3):469, 1993
1993
-
[55]
M. J. Monteith, C. I. Voils, and L. Ashburn-Nardo. Taking a look underground: Detecting, interpreting, and reacting to implicit racial biases.Social Cognition, 19(4):395–417, 2001
2001
-
[56]
Moors and J
A. Moors and J. De Houwer. Problems with dividing the realm of processes.Psychological Inquiry, 17(3):199–204, 2006
2006
-
[57]
V. Mruthyunjaya, P. Pezeshkpour, E. Hruschka, and N. Bhutani. Rethinking language models as symbolic knowledge graphs.arXiv preprint arXiv:2308.13676, 2023. 29/31
-
[58]
B. A. Nosek, F. L. Smyth, J. J. Hansen, T. Devos, N. M. Lindner, K. A. Ranganath, C. T. Smith, K. R. Olson, D. Chugh, A. G. Greenwald, et al. Pervasiveness and correlates of implicit attitudes and stereotypes. European review of social psychology, 18(1):36–88, 2007
2007
-
[59]
E. Pavlick. Symbols and grounding in large language models.Philosophical Transactions of the Royal Society A, 381(2251):20220041, 2023
2023
-
[60]
B. K. Payne. Conceptualizing control in social cognition: how executive functioning modulates the expression of automatic stereotyping.Journal of personality and social psychology, 89(4):488, 2005
2005
-
[61]
Perugini
M. Perugini. Predictive models of implicit and explicit attitudes.British Journal of Social Psychology, 44(1):29–45, 2005
2005
-
[62]
L. M. Reder and J. R. Anderson. A partial resolution of the paradox of interference: The role of integrating knowledge.Cognitive Psychology, 12(4):447–472, 1980
1980
-
[63]
D. E. Rumelhart, J. L. McClelland, P. R. Group, et al.Parallel distributed processing, volume 1: Explorations in the microstructure of cognition: Foundations. The MIT press, 1986
1986
-
[64]
C. S. Siew. spreadr: An r package to simulate spreading activation in a network.Behavior Research Methods, 51(2):910–929, 2019
2019
-
[65]
C. S. Siew, D. U. Wulff, N. M. Beckage, and Y. N. Kenett. Cognitive network science: A review of research on cognition through the lens of network representations, processes, and dynamics.Complexity, 2019, 2019
2019
-
[66]
H. A. Simon and A. Newell. Human problem solving: The state of the theory in 1970.American psychologist, 26(2):145, 1971
1970
-
[67]
S. A. Sloman. The empirical case for two systems of reasoning.Psychological bulletin, 119(1):3, 1996
1996
-
[68]
E. E. Smith, E. J. Shoben, and L. J. Rips. Structure and process in semantic memory: A featural model for semantic decisions.Psychological review, 81(3):214, 1974
1974
-
[69]
E. R. Smith and J. DeCoster. Dual-process models in social and cognitive psychology: Conceptual integration and links to underlying memory systems.Personality and social psychology review, 4(2):108–131, 2000
2000
-
[70]
Smolensky
P. Smolensky. On the proper treatment of connectionism.Behavioral and brain sciences, 11(1):1–23, 1988
1988
-
[71]
J. F. Sowa. Generating language from conceptual graphs. InComputational Linguistics, pages 29–43. Elsevier, 1983
1983
-
[72]
K. E. Stanovich.Who is rational?: Studies of individual differences in reasoning. Psychology Press, 1999
1999
-
[73]
Stella, S
M. Stella, S. Citraro, G. Rossetti, D. Marinazzo, Y. N. Kenett, and M. S. Vitevitch. Cognitive modelling of concepts in the mental lexicon with multilayer networks: Insights, advancements, and future challenges. Psychonomic Bulletin & Review, 31(5):1981–2004, 2024
1981
-
[74]
Steyvers and J
M. Steyvers and J. B. Tenenbaum. The large-scale structure of semantic networks: Statistical analyses and a model of semantic growth.Cognitive science, 29(1):41–78, 2005
2005
-
[75]
Strack and R
F. Strack and R. Deutsch. Reflective and impulsive determinants of social behavior.Personality and social psychology review, 8(3):220–247, 2004
2004
-
[76]
Vincent-Lamarre, A
P. Vincent-Lamarre, A. B. Mass´ e, M. Lopes, M. Lord, O. Marcotte, and S. Harnad. The latent structure of dictionaries.Topics in cognitive science, 8(3):625–659, 2016
2016
-
[77]
J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, et al. Emergent abilities of large language models.arXiv preprint arXiv:2206.07682, 2022. 30/31
work page internal anchor Pith review arXiv 2022
-
[78]
J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in Neural Information Processing Systems, 35:24824–24837, 2022
2022
-
[79]
W. A. Woods. What’s in a link: Foundations for semantic networks. InRepresentation and understanding, pages 35–82. Elsevier, 1975
1975
-
[80]
Towards system 2 reasoning in llms: Learning how to think with meta chain-of-though,
V. Xiang, C. Snell, K. Gandhi, A. Albalak, A. Singh, C. Blagden, D. Phung, R. Rafailov, N. Lile, D. Mahan, et al. Towards system 2 reasoning in llms: Learning how to think with meta chain-of-thought.arXiv preprint arXiv:2501.04682, 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.