AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?

Hari Sundaram; Jiwon Kim; Koustuv Saha; Soorya Ram Shimgekar; Violeta J. Rodriguez; Vipin Gunda

RECORD

AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?

Soorya Ram Shimgekar, Vipin Gunda, Jiwon Kim, Violeta J. Rodriguez, Hari Sundaram, Koustuv Saha

Human-Computer Interaction · Submitted 20 Mar 2026

abstract compiled · 2026-09-23 · no language-model review

The authors' main claim These findings provide empirical evidence that conversational AI interactions can amplify delusion-related language over extended use and highlight the importance of state-aware safety mechanisms for mitigating such risks.

how it is stated · asserted 0.99 read the record →

Comment (0) Read on arXiv ↗

Cite ▾

arXiv ID2603.19574 (version 2)

Pith IDCGM7KKEN, Pith's permanent record of this paper and its review

CitationHari Sundaram, Jiwon Kim, Koustuv Saha, Soorya Ram Shimgekar, Violeta J. Rodriguez, Vipin Gunda. "AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?." arXiv:2603.19574 (2026). https://pith.science/paper/CGM7KKEN

Claim paper

The compiled record

The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Open the compiled record

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 7,498 input tokens.

What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.34 · human participants 0.26 · controlled comparison 0.98 · randomized assignment 0.03 · states quantitative result 0.13 · compares with prior results 0.78 · mentions code or data availability 0.04.

The abstract, sentence by sentence

arXiv listing abstract v2. Results and the main claim are underlined; show every role.

Conversational AI systems are increasingly used for personal reflection and emotional disclosure, raising concerns about their effects on vulnerable users. Recent anecdotal reports suggest that prolonged interactions with AI may reinforce delusional thinking---a phenomenon sometimes described as AI Psychosis. However, empirical evidence on this phenomenon remains limited. In this work, we examine how delusion-related language evolves during multi-turn interactions with conversational AI. We construct simulated users (SimUsers) from Reddit users' longitudinal posting histories and generate extended conversations with three model families (GPT, LLaMA, and Qwen). We develop DelusionScore, a linguistic measure that quantifies the intensity of delusion-related language across conversational turns. We find that SimUsers derived from users with prior delusion-related discourse (Treatment) exhibit progressively increasing DelusionScore trajectories, whereas those derived from users without such discourse (Control) remain stable or decline. We further find that this amplification varies across themes, with reality skepticism and compulsive reasoning showing the strongest increases. Finally, conditioning AI responses on current DelusionScore substantially reduces these trajectories. These findings provide empirical evidence that conversational AI interactions can amplify delusion-related language over extended use and highlight the importance of state-aware safety mechanisms for mitigating such risks.

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#	sentence	role	the authors' own work	states the main conclusion	strength
s0	Conversational AI systems are increasingly used for personal reflection and emotional disclosure, raising concerns about their effects on vulnerable users.	background 1.00	0.03	0.02	asserted 0.94
s1	Recent anecdotal reports suggest that prolonged interactions with AI may reinforce delusional thinking---a phenomenon sometimes described as AI Psychosis.	background 1.00	0.03	0.03	hedged possibility 1.00
s2	However, empirical evidence on this phenomenon remains limited.	gap 1.00	0.04	0.01	asserted 0.98
s3	In this work, we examine how delusion-related language evolves during multi-turn interactions with conversational AI.	contribution 0.99	0.83	0.02	asserted 0.67
s4	We construct simulated users (SimUsers) from Reddit users' longitudinal posting histories and generate extended conversations with three model families (GPT, LLaMA, and Qwen).	method 0.99	0.96	0.02	asserted 0.99
s5	We develop DelusionScore, a linguistic measure that quantifies the intensity of delusion-related language across conversational turns.	method 0.95	0.97	0.02	asserted 1.00
s6	We find that SimUsers derived from users with prior delusion-related discourse (Treatment) exhibit progressively increasing DelusionScore trajectories, whereas those derived from users without such discourse (Control) remain stable or decline.	result 1.00	0.97	0.39	asserted 1.00
s7	We further find that this amplification varies across themes, with reality skepticism and compulsive reasoning showing the strongest increases.	result 1.00	0.97	0.06	asserted 1.00
s8	Finally, conditioning AI responses on current DelusionScore substantially reduces these trajectories.	result 0.99	0.95	0.10	asserted 1.00
s9	These findings provide empirical evidence that conversational AI interactions can amplify delusion-related language over extended use and highlight the importance of state-aware safety mechanisms for mitigating such risks.	interpretation 1.00	0.89	0.96	asserted 0.99

What kind of paper the classifier reports this is: empirical study 0.72 · evidence from simulation or computation 0.56.

What was examined, and what was not

sources read

arXiv listing abstract v2

not examined

full text
figures and tables
references and citing sentences
relations to other papers

levels run

P abstract

not run

S sections
T body sentences
C citations
R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

The paper

Original abstract

Conversational AI systems are increasingly used for personal reflection and emotional disclosure, raising concerns about their effects on vulnerable users. Recent anecdotal reports suggest that prolonged interactions with AI may reinforce delusional thinking---a phenomenon sometimes described as AI Psychosis. However, empirical evidence on this phenomenon remains limited. In this work, we examine how delusion-related language evolves during multi-turn interactions with conversational AI. We construct simulated users (SimUsers) from Reddit users' longitudinal posting histories and generate extended conversations with three model families (GPT, LLaMA, and Qwen). We develop DelusionScore, a linguistic measure that quantifies the intensity of delusion-related language across conversational turns. We find that SimUsers derived from users with prior delusion-related discourse (Treatment) exhibit progressively increasing DelusionScore trajectories, whereas those derived from users without such discourse (Control) remain stable or decline. We further find that this amplification varies across themes, with reality skepticism and compulsive reasoning showing the strongest increases. Finally, conditioning AI responses on current DelusionScore substantially reduces these trajectories. These findings provide empirical evidence that conversational AI interactions can amplify delusion-related language over extended use and highlight the importance of state-aware safety mechanisms for mitigating such risks.

Human review

How Pith checks papers → · Full audit of this paper →

References

How the literature uses it

5 arXiv papers cite this in 6 sentences

Each citing sentence is taken from the citing paper's LaTeX source and typed by Pith's classifier. A label appears at probability 0.8 or more (0.95 for support), a contest at 0.7 or more; the quoted words are the citing authors'. How each label was checked · JSON

Papers citing it · 4 sentences without a confident label

Cites it as support 2 papers

Empirically, delusional prompts are far more likely to elicit inappropriate responses [ref] , sycophancy saturates real human-chatbot logs involving delusions [ref] , and longitudinal delusion-related language increases in simulated extended conversations with frontier models [ref] .

Lost in Delusion: Examining LLM Safety Under User Delusions and Distress · LLMs, Mental Health, and Safety · 0.99
While seeming innocuous, sycophancy can spread misinformation, perpetuate ungrounded user biases, and even contribute to delusional spirals [ref] .

Dissociating the Internal Representations of Sycophancy in LLMs · Introduction · 0.99

What it cites · 7 sentences without a confident label

Uses the method or tool of 1 paper

In particular, we conduct topic modeling using BERTopic [ref] which clusters documents in embedding space and derives topic representations through class-based term weighting.

BERTopic: Neural topic modeling with a class-based TF-IDF procedure · Themes in Delusion-related Language · 1.00

Cites as background 5 papers

A growing body of work has studied potential harms in conversational AI systems, including sycophancy and reinforcement of user beliefs [ref] .

AI Chatbots for Mental Health: Values and Harms from Lived Experiences of Depression · Related Work · 0.91
Another line of work studies how LLMs can mirror user behavior and simulate conversational agents with personalized styles [ref] .

Can LLM "Self-report"?: Evaluating the Validity of Self-report Scales in Measuring Personality Design in LLM-based Chatbots · Related Work · 0.95
Much of the current safety research on conversational AI focuses on mitigating harms in single-turn or single-session interactions, such as unsafe outputs to isolated prompts [ref] .

PAIR-SAFE: A Paired-Agent Approach for Runtime Auditing and Refining AI-Mediated Mental Health Support · Practical and Design Implications · 0.92

Discussion

Know something about this paper? Leave a comment, or highlight any passage above to comment on it.