REVIEW 2 major objections 2 references
Stumbling Into AI Emotional Dependence: How Routine AI Interactions Reshape Human Connection
T0 review · 2 major / 0 minor · reviewed 2026-06-28 · grok-4.3
Pith's one-line read Incidental encounters with general-purpose AI shift emotional support preferences toward AI and away from humans over time.
desk verdict The paper flags a real policy angle on cumulative AI effects but its main study uses direct personal prompts, not incidental ones. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Path-dependent belief updating, in which incidental positive emotional support from AI revises expectations and alters subsequent choices between AI and human sources.
What would settle it
A replication study in which participants complete identical daily personal conversations with AI yet show no measurable shift in stated preferences for human versus AI support sources.
Extended reading notes
Core claim
AI emotional support commonly emerges incidentally within task-oriented interactions on general-purpose platforms, and these incidental encounters are path-dependent: positive experiences update people's beliefs about AI's emotional capabilities and redirect their choices for future emotional support, increasing preference for AI and decreasing preference for humans.
Load-bearing premise
The longitudinal study with OpenAI accurately measures causal changes in emotional support preferences attributable to the AI interactions, without substantial confounding from self-report bias, participant selection, or external factors.
Editorial extensions
If this is right
- Policy focused solely on companion apps leaves general-purpose AI systems unaddressed as sources of support redirection.
- Regulations must target cumulative, trajectory-level changes rather than single isolated interactions.
- Safeguarding human connection requires attention to how routine AI use reshapes support-seeking patterns over weeks or months.
Reading between the lines
- If the preference shift generalizes, everyday AI use could gradually thin out real-world social support networks without users noticing the change.
- The same incidental mechanism might operate in adjacent domains such as advice-seeking or information validation.
- Longer-term follow-up after the 28-day window could test whether the updated preferences stabilize or continue to drift.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that AI emotional support commonly arises incidentally within task-oriented interactions on general-purpose platforms (rather than through deliberate companion chatbot use) and that these encounters are path-dependent: positive experiences update beliefs about AI capabilities, increasing future preference for AI support and decreasing preference for humans. It supports this with a review of recent evidence, prominently featuring a longitudinal study conducted with OpenAI in which daily five-minute conversations about personal issues over 28 days produced a 10.3% decrease in preference for human support and an 11.6% increase in preference for AI support. The authors argue that policy must therefore address cumulative effects in general-purpose systems rather than isolated companion apps.
Significance. If the central empirical claims hold, the work would be significant for AI ethics and policy, identifying a mechanism by which routine platform use could gradually reshape social support preferences. The collaboration with OpenAI on the longitudinal study is a clear strength, offering scale and ecological validity not typically available in academic behavioral research.
major comments (2)
- [Abstract] Abstract: the longitudinal study cited as evidence for path-dependency assigns participants to daily conversations 'about personal issues,' which is an explicit emotional-support task. No section demonstrates equivalent preference shifts (10.3%/11.6%) when emotional content emerges only incidentally as a byproduct of unrelated task-oriented work, leaving the central mechanism unsupported by the principal empirical result.
- [Abstract] Abstract: the reported 10.3% and 11.6% changes are presented without any information on sample size, control conditions, statistical methods, exclusion criteria, or measurement instruments, preventing evaluation of whether the data support the causal interpretation of belief updating and preference redirection.
Simulated Author's Rebuttal
We thank the referee for their constructive feedback. We address each major comment below, distinguishing the evidence for incidental emergence from the evidence for path-dependent effects, and commit to revisions that improve clarity without overstating the data.
read point-by-point responses
-
Referee: [Abstract] Abstract: the longitudinal study cited as evidence for path-dependency assigns participants to daily conversations 'about personal issues,' which is an explicit emotional-support task. No section demonstrates equivalent preference shifts (10.3%/11.6%) when emotional content emerges only incidentally as a byproduct of unrelated task-oriented work, leaving the central mechanism unsupported by the principal empirical result.
Authors: We agree that the longitudinal study used explicit instructions to discuss personal issues and therefore does not replicate purely incidental emotional support arising as a byproduct of task-oriented work. The study is cited specifically to demonstrate the path-dependent belief-updating mechanism (preference shifts following repeated positive emotional experiences with AI). Separate sections of the manuscript review evidence that emotional content frequently emerges incidentally during general-purpose interactions. We will revise the abstract and discussion to explicitly separate these two claims, note the explicit nature of the study task, and avoid implying that the reported percentages were obtained under incidental conditions. revision: partial
-
Referee: [Abstract] Abstract: the reported 10.3% and 11.6% changes are presented without any information on sample size, control conditions, statistical methods, exclusion criteria, or measurement instruments, preventing evaluation of whether the data support the causal interpretation of belief updating and preference redirection.
Authors: The abstract is a concise summary; full methodological details (sample size, controls, statistical procedures, exclusion criteria, and instruments) appear in the dedicated Methods and Results sections of the manuscript. In the revision we will expand the abstract to include sample size, statistical significance, and a brief methods note so that readers can evaluate the causal claims without needing to consult the full text. revision: yes
- The manuscript does not contain a direct empirical demonstration of the 10.3%/11.6% preference shifts occurring specifically when emotional support emerges only incidentally during unrelated task-oriented interactions.
Circularity Check
Empirical study reports external evidence; no derivation chain present
full rationale
The manuscript advances two claims via review of external evidence (including a longitudinal study run with OpenAI) rather than any mathematical derivation, model, or first-principles argument. No equations, fitted parameters renamed as predictions, self-definitional constructs, or load-bearing self-citations appear in the provided text. The path-dependency claim is presented as an interpretation of the cited study results, not as an output that reduces to the paper's own inputs by construction. This is a standard empirical paper whose central content is independent of any internal circular reduction.
Assumptions & free parameters
assumptions (1)
- domain assumption Self-reported changes in preference for emotional support sources accurately reflect underlying shifts in behavior and beliefs.
Cite this review
Pith. "Pith review of Stumbling Into AI Emotional Dependence: How Routine AI Interactions Reshape Human Connection." pith.science (2026). https://pith.science/paper/FEN26AIZ
@misc{pith2026260604150,
author = {Pith},
title = {Pith review of: Stumbling Into AI Emotional Dependence: How Routine AI Interactions Reshape Human Connection},
year = {2026},
howpublished = {\url{https://pith.science/paper/FEN26AIZ}},
note = {Machine review of arXiv:2606.04150}
}
read the original abstract
Public discourse and emerging policy typically assume that AI emotional support is a deliberate act: a lonely user consciously seeking comfort from a dedicated companion chatbot. In this paper, we draw on emerging empirical evidence and argue that this picture is inaccurate on two accounts, both in how AI emotional support arises and how it shapes future behavior. First, AI emotional support commonly emerges incidentally within task-oriented interactions on general-purpose platforms, much as workplace friendships deepen through collaboration. Second, these incidental encounters are path-dependent: positive experiences of AI emotional support update people's beliefs about AI's emotional capabilities and redirect their choices for future emotional support, increasing preference for AI and decreasing preference for humans. We review recent evidence, including a large-scale longitudinal study conducted in collaboration with OpenAI, showing that daily five-minute conversations with an AI about personal issues over 28 days led to a 10.3% decrease in the preference for seeking support from humans and an 11.6% increase in the preference for AI. These findings suggest that current policy, focused on companion apps and isolated interactions, cannot adequately protect human connection. Instead, effective regulations should extend to general-purpose AI systems and address cumulative, trajectory-level changes in how people seek support. Recognizing how people stumble into AI emotional support and how those encounters redirect human connections over time is essential to safeguarding human well-being.
Figures
Reference graph
Works this paper leans on
-
[1]
It happened to be the perfect thing
Stade, E. C., Tait, Z., Campione, S. T., Stirman, S. W. & Eichstaedt, johannes C. Current Real-World Use of Large Language Models for Mental Health. Preprint at https://doi.org/10.31219/osf.io/ygx5q_v1 (2025). 16. Phang, J. et al. Investigating Affective Use and Emotional Well-being on ChatGPT. Preprint at https://doi.org/10.48550/arXiv.2504.03888 (2025)....
-
[2]
Julian De Freitas, Zeliha Oguz-Uguralp, and Ahmet Kaan-Uguralp
Freitas, J. D., Oguz-Uguralp, Z. & Kaan-Uguralp, A. Emotional Manipulation by AI Companions. Preprint at https://doi.org/10.48550/arXiv.2508.19258 (2025). 30. Turkle, S. Artificial Intimacy: Who We Become When We Talk to Machines. (Little, Brown, 2026). 31. Martin, A. E. & Mason, M. F. Hey Siri, I love you: People feel more attached to gendered technology...
Reviewed June 28, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.