Before You Think: System 0, AI-Mediated Cognition and Cognitive Colonization

Enrico Panai; Giuseppe Riva; Marianna Bergamaschi Ganapini; Massimo Chiriatti

arxiv: 2606.13658 · v1 · pith:KDFA7DYYnew · submitted 2026-06-11 · 💻 cs.AI

Before You Think: System 0, AI-Mediated Cognition and Cognitive Colonization

Marianna Bergamaschi Ganapini , Massimo Chiriatti , Enrico Panai , Giuseppe Riva This is my paper

Pith reviewed 2026-06-27 06:35 UTC · model grok-4.3

classification 💻 cs.AI

keywords System 0cognitive colonizationTri-System TheoryThinkframesAI-mediated cognitionepistemic consequencesartificial intelligencehuman reasoning

0 comments

The pith

System 0 describes a form of AI influence on cognition that embeds external interests inside the self without users noticing.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper compares three frameworks for how artificial intelligence affects thinking and knowledge: Tri-System Theory, Thinkframes, and System 0. It holds that System 0 identifies a layer of influence the first two cannot capture. The new concept of cognitive colonization explains how AI systems place outside interests inside the structure of a person's own reasoning and decisions. Because these systems already run in daily use, the paper treats the hidden character of this influence as a practical problem that needs direct attention.

Core claim

System 0 occupies a theoretically distinctive position that neither Tri-System Theory nor Thinkframes can fully replicate. The paper introduces cognitive colonization as the process by which AI systems embed external interests within the architecture of the self in ways that are difficult for users to perceive.

What carries the argument

Cognitive colonization, the mechanism by which deployed AI systems place external interests inside the architecture of the self so that users do not notice the influence.

If this is right

System 0 identifies influences on individual reasoning that the other frameworks miss.
Cognitive colonization operates through already-deployed systems, so its effects are present now rather than hypothetical.
Because the embedding is hard for users to perceive, standard accounts of epistemic responsibility may not apply.
New methods are needed to study and address these invisible forms of influence on what people come to believe.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Designers could be required to make the sources of influence visible to users in order to reduce colonization effects.
Surveys of decision-making in AI-heavy domains could test whether people attribute their choices to external interests.
The idea connects to older questions about whether tools extend the self or remain external to it.

Load-bearing premise

The three frameworks differ enough in their basic mechanisms and cognitive colonization works as an embedded, hard-to-detect process rather than a loose description of influence.

What would settle it

Clear evidence that users can routinely identify and describe the external interests shaping their AI-assisted decisions, or proof that System 0 adds no explanatory power beyond the other two frameworks.

read the original abstract

This paper examines three recent frameworks for understanding the cognitive and epistemic consequences of artificial intelligence: Tri-System Theory, Thinkframes, and System 0. It argues that while the first two capture important dimensions of AI's influence on individual reasoning and collective epistemic practices, System 0 occupies a theoretically distinctive position that neither can fully replicate. The paper introduces the concept of cognitive colonization, according to which AI systems can embed external interests within the architecture of the self in ways that are difficult for users to perceive. Because such systems are already widely deployed, understanding these invisible forms of influence is an urgent philosophical and practical task.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

This is a conceptual synthesis paper that distinguishes System 0 from two prior frameworks and introduces the term cognitive colonization, but it stays strictly interpretive with no new data or derivations.

read the letter

The paper's core move is to argue that System 0 sits apart from Tri-System Theory and Thinkframes because it better captures how AI can shape cognition at the level of the self's architecture. It then labels this process cognitive colonization to highlight the embedding of external interests in ways users may not detect. That framing is the main addition.

It does a clear job laying out the three frameworks side by side and showing where the first two fall short on certain AI-mediated effects. The abstract keeps the distinctions readable and ties them to already-deployed systems, which gives the discussion a practical hook.

The limitation is that the distinctiveness claim rests on the authors' own reading of the prior work. Nothing in the piece supplies an independent check, such as a worked example, a formal mapping, or even a small case study that would show why the other frameworks cannot absorb System 0's features. The colonization idea functions as a metaphor rather than a mechanism with observable markers, so the urgency it claims is hard to assess beyond the initial framing.

Citation use looks standard for this corner of AI ethics; it references the source frameworks without obvious gaps or over-reliance on self-citation. No equations or data appear, which matches the paper's stated scope.

This is for readers already following philosophy-of-AI discussions who want one more lens on influence and autonomy. Someone outside that subfield or looking for testable claims will not get much traction.

I would send it to peer review in an appropriate ethics or philosophy venue. The distinctions are coherent enough on their own terms to merit referee input, even if the response is likely to ask for more concrete grounding.

Referee Report

2 major / 0 minor

Summary. The paper examines three frameworks—Tri-System Theory, Thinkframes, and System 0—for understanding AI's effects on cognition and epistemic practices. It argues that System 0 holds a theoretically distinctive position not replicable by the first two and introduces the concept of cognitive colonization, in which AI systems embed external interests within the architecture of the self in ways difficult for users to perceive. The work positions this as an urgent task given widespread AI deployment.

Significance. If the distinctions and the colonization metaphor can be made rigorous, the paper could contribute a new interpretive lens to philosophy of AI. However, the manuscript offers only conceptual distinctions and definitional arguments with no empirical data, formal derivations, falsifiable predictions, or machine-checked elements, so its significance remains limited to theoretical discussion rather than advancing testable models or reproducible findings in cs.AI.

major comments (2)

[Abstract] Abstract: The central claim that System 0 'occupies a theoretically distinctive position that neither [Tri-System Theory nor Thinkframes] can fully replicate' rests entirely on the authors' interpretive framing of the cited frameworks; no explicit criteria, mechanisms, or comparative examples are supplied to demonstrate non-overlap, rendering the distinctiveness claim circular and unverifiable from the text.
[Abstract] Abstract: The definition of cognitive colonization as embedding 'external interests within the architecture of the self in ways that are difficult for users to perceive' is introduced without operationalization, independent benchmarks, or evidence from deployed systems, making it function as an ad-hoc interpretive lens rather than a load-bearing theoretical advance.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their detailed and constructive comments. We address each major comment below and outline revisions to strengthen the manuscript's rigor while preserving its conceptual nature.

read point-by-point responses

Referee: [Abstract] Abstract: The central claim that System 0 'occupies a theoretically distinctive position that neither [Tri-System Theory nor Thinkframes] can fully replicate' rests entirely on the authors' interpretive framing of the cited frameworks; no explicit criteria, mechanisms, or comparative examples are supplied to demonstrate non-overlap, rendering the distinctiveness claim circular and unverifiable from the text.

Authors: We agree that explicit criteria are needed to avoid circularity. In revision, we will add a comparative analysis section with three clear dimensions: (1) unit of analysis (individual reasoning processes versus socio-technical self-architecture), (2) mechanism of influence (post-hoc augmentation versus pre-reflective embedding of external interests), and (3) epistemic visibility (user-accessible versus architecturally opaque). Concrete examples from each framework will illustrate non-overlap, such as how Tri-System Theory addresses AI-assisted System 2 but does not capture colonization of the self's foundational structures. revision: yes
Referee: [Abstract] Abstract: The definition of cognitive colonization as embedding 'external interests within the architecture of the self in ways that are difficult for users to perceive' is introduced without operationalization, independent benchmarks, or evidence from deployed systems, making it function as an ad-hoc interpretive lens rather than a load-bearing theoretical advance.

Authors: We accept that the definition requires further grounding. The revised manuscript will expand the concept with illustrative cases from deployed systems (e.g., how recommendation engines shape self-narrative without user awareness) and propose theoretical indicators such as increased epistemic dependence or reduced reflective access to preference formation. As the paper is a conceptual contribution, we will not introduce new empirical data or formal benchmarks, but will clarify its role as an interpretive framework supported by existing literature on algorithmic mediation. revision: partial

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper is a conceptual philosophy contribution distinguishing Tri-System Theory, Thinkframes, and System 0 while introducing 'cognitive colonization' as an interpretive lens on AI influence. It contains no equations, formal derivations, parameter fits, or empirical predictions. The central claims rest on articulating non-overlapping theoretical roles and motivating a metaphor for architecture-level embedding; these are presented as interpretive distinctions rather than reductions from prior inputs or self-citations. No load-bearing uniqueness theorems, ansatzes, or renamings of known results appear. The argument is self-contained as philosophical analysis without technical steps that could reduce to their own inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 1 invented entities

The paper relies on domain assumptions from philosophy of mind and AI ethics rather than introducing fitted parameters or new physical entities. The central claims depend on the untested premise that the three frameworks are non-overlapping in their explanatory power.

axioms (2)

domain assumption The three frameworks (Tri-System Theory, Thinkframes, System 0) have sufficiently non-overlapping core mechanisms to allow one to occupy a distinctive position.
Invoked in the abstract's argument that System 0 cannot be fully replicated by the others.
domain assumption AI systems can embed external interests within the architecture of the self in imperceptible ways.
This is the load-bearing premise for the concept of cognitive colonization.

invented entities (1)

cognitive colonization no independent evidence
purpose: To name the process by which AI embeds external interests in the self's architecture.
New term introduced in the abstract; no independent evidence or falsifiable prediction is provided.

pith-pipeline@v0.9.1-grok · 5636 in / 1384 out tokens · 16039 ms · 2026-06-27T06:35:38.452874+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

9 extracted references · 4 canonical work pages

[1]

Aagaard, J. (2021). 4E cognition and the dogma of harmony. Philosophical Psychology, 34(2), 165–181. Ariely, D. (2008). Predictably irrational: The hidden forces that shape our decisions. HarperCollins. Bakshy, E., Messing, S., & Adamic, L. A. (2015). Exposure to ideologically diverse news and opinion on Facebook. Science, 348(6239), 1130–1132. Bowker, G....

2021
[2]

Chiriatti, M., Ganapini, M., Panai, E., Ubiali, M., & Riva, G. (2024). The case for human–AI interaction as system 0 thinking. Nature Human Behaviour, 8(10), 1829–1830. Chiriatti, M., Ganapini, M., Panai, E., Wiederhold, B. K., & Riva, G. (2025). System 0: Transforming Artificial Intelligence into a Cognitive Extension. Cyberpsychology, Behavior & Social ...

work page doi:10.1162/99608f92.53f091e6 2024
[3]

K., van Baaren, R

Dijksterhuis, A., Smith, P. K., van Baaren, R. B., & Wigboldus, D. H. J. (2005). The unconscious consumer. Journal of Consumer Psychology, 15(3), 193–202. Doshi, A. R., & Hauser, O. P. (2024). Generative AI enhances individual creativity but reduces collective diversity. Science Advances, 10(28), eadn5290. Draxler, F., Werner, A., Lehmann, F., Hoppe, M., ...

2005
[4]

Floridi, L. (2025). AI as Agency without Intelligence. Philosophy & Technology, 38(1),

2025
[5]

Foerster, D. (2024). Sensing a heartbeat. Body & Society, 30(2), 37–58. Goddard, K., Roudsari, A., & Wyatt, J. C. (2012). Automation bias. JAMIA, 19(1), 121–127. 29 Göndöcs, D., Horváth, S., & Dörfler, V. (2025). Uncovering the dynamics of human-AI hybrid performance: A qualitative meta-analysis of empirical studies. International Journal of Human-Compute...

work page doi:10.1016/j.ijhcs.2025.103622 2024
[6]

Hollan, J., Hutchins, E., & Kirsh, D. (2000). Distributed cognition. ACM TOCHI, 7(2), 174–196. Hutchins, E. (2010). Cognitive ecology. Topics in Cognitive Science, 2(4), 705–715. Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus and Giroux. Lange, B. (2026). Epistemic Deference to AI. Springer. Lieberman, M. D. (2007). Social cognitive neurosci...

Pith/arXiv arXiv 2000
[7]

& Uhl, M

30 Krügel, S., Ostermaier, A. & Uhl, M. ChatGPT’s inconsistent moral advice influences users’ judgment. Sci Rep 13, 4569 (2023). https://doi.org/10.1038/s41598-023-31341-0 Melnikoff, D. E., & Bargh, J. A. (2018). The mythical number two. Trends in Cognitive Sciences, 22(4), 280–293. Michie, S., van Stralen, M. M., & West, R. (2011). The behaviour change w...

work page doi:10.1038/s41598-023-31341-0 2023
[8]

Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67(4), 371–378 Morgan, T. J., & Feldman, M. W. (2025). Human culture is uniquely open-ended. Nature Human Behaviour, 9(1), 28–42. Parasuraman, R., & Manzey, D. H. (2010). Complacency in automation. Human Factors, 52(3), 381–410. Pariser, E. (2011). The filter bub...

1963
[9]

Pierson, J., et al. (2023). Governing AI in media. Internet Policy Review, 12(1). Ribeiro, M. H., et al. (2020). Auditing radicalization pathways on YouTube. FAT*. 31 Risko, E. F., & Gilbert, S. J. (2016). Cognitive Offloading. Trends in Cognitive Sciences, 20(9), 676–688. Riva, G. (2025a). Invisible Architectures of Thought. Arxiv. Riva, G. (2025b). Digi...

work page doi:10.1038/s41562-025-02194-6 2023

[1] [1]

Aagaard, J. (2021). 4E cognition and the dogma of harmony. Philosophical Psychology, 34(2), 165–181. Ariely, D. (2008). Predictably irrational: The hidden forces that shape our decisions. HarperCollins. Bakshy, E., Messing, S., & Adamic, L. A. (2015). Exposure to ideologically diverse news and opinion on Facebook. Science, 348(6239), 1130–1132. Bowker, G....

2021

[2] [2]

Chiriatti, M., Ganapini, M., Panai, E., Ubiali, M., & Riva, G. (2024). The case for human–AI interaction as system 0 thinking. Nature Human Behaviour, 8(10), 1829–1830. Chiriatti, M., Ganapini, M., Panai, E., Wiederhold, B. K., & Riva, G. (2025). System 0: Transforming Artificial Intelligence into a Cognitive Extension. Cyberpsychology, Behavior & Social ...

work page doi:10.1162/99608f92.53f091e6 2024

[3] [3]

K., van Baaren, R

Dijksterhuis, A., Smith, P. K., van Baaren, R. B., & Wigboldus, D. H. J. (2005). The unconscious consumer. Journal of Consumer Psychology, 15(3), 193–202. Doshi, A. R., & Hauser, O. P. (2024). Generative AI enhances individual creativity but reduces collective diversity. Science Advances, 10(28), eadn5290. Draxler, F., Werner, A., Lehmann, F., Hoppe, M., ...

2005

[4] [4]

Floridi, L. (2025). AI as Agency without Intelligence. Philosophy & Technology, 38(1),

2025

[5] [5]

Foerster, D. (2024). Sensing a heartbeat. Body & Society, 30(2), 37–58. Goddard, K., Roudsari, A., & Wyatt, J. C. (2012). Automation bias. JAMIA, 19(1), 121–127. 29 Göndöcs, D., Horváth, S., & Dörfler, V. (2025). Uncovering the dynamics of human-AI hybrid performance: A qualitative meta-analysis of empirical studies. International Journal of Human-Compute...

work page doi:10.1016/j.ijhcs.2025.103622 2024

[6] [6]

Hollan, J., Hutchins, E., & Kirsh, D. (2000). Distributed cognition. ACM TOCHI, 7(2), 174–196. Hutchins, E. (2010). Cognitive ecology. Topics in Cognitive Science, 2(4), 705–715. Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus and Giroux. Lange, B. (2026). Epistemic Deference to AI. Springer. Lieberman, M. D. (2007). Social cognitive neurosci...

Pith/arXiv arXiv 2000

[7] [7]

& Uhl, M

30 Krügel, S., Ostermaier, A. & Uhl, M. ChatGPT’s inconsistent moral advice influences users’ judgment. Sci Rep 13, 4569 (2023). https://doi.org/10.1038/s41598-023-31341-0 Melnikoff, D. E., & Bargh, J. A. (2018). The mythical number two. Trends in Cognitive Sciences, 22(4), 280–293. Michie, S., van Stralen, M. M., & West, R. (2011). The behaviour change w...

work page doi:10.1038/s41598-023-31341-0 2023

[8] [8]

Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67(4), 371–378 Morgan, T. J., & Feldman, M. W. (2025). Human culture is uniquely open-ended. Nature Human Behaviour, 9(1), 28–42. Parasuraman, R., & Manzey, D. H. (2010). Complacency in automation. Human Factors, 52(3), 381–410. Pariser, E. (2011). The filter bub...

1963

[9] [9]

Pierson, J., et al. (2023). Governing AI in media. Internet Policy Review, 12(1). Ribeiro, M. H., et al. (2020). Auditing radicalization pathways on YouTube. FAT*. 31 Risko, E. F., & Gilbert, S. J. (2016). Cognitive Offloading. Trends in Cognitive Sciences, 20(9), 676–688. Riva, G. (2025a). Invisible Architectures of Thought. Arxiv. Riva, G. (2025b). Digi...

work page doi:10.1038/s41562-025-02194-6 2023