The Societal Response to Potentially Sentient AI

Lucius Caviola

REVIEW 3 major objections 5 minor 4 cited by

The Societal Response to Potentially Sentient AI

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Perceived AI sentience, not the truth about machine minds, will drive how society governs artificial intelligence.

desk verdict A useful research agenda on public perceptions of AI sentience, undermined by a clear numeric error and heavy reliance on unpublished self-cited work. read the letter →

arxiv 2502.00388 v2 pith:MWONWXX4 submitted 2025-02-01 cs.CY

Lucius Caviola This is my paper · ORCID

classification cs.CY

keywords AIsentiencepublicopinionmoralstatusrightssocialanthropomorphismwelfaremisattribution

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Regardless of whether machines can ever truly feel, the paper argues, society's beliefs about AI sentience will do the causal work: they will shape regulation, consumer demand, the roles assigned to AI, and the rights granted to machines. The paper therefore calls for a new social science of sentient AI that studies how laypeople and experts infer sentience from AI features, how emotional bonds with social AIs shift those inferences, and how expert opinion, incentives, and cultural factors moderate the process. It maps the landscape with a risk matrix of over- and under-attribution, a feature framework separating what is observable from what is informative, and three scenarios ranging from persistent skepticism to broad acceptance to a period of confusion and conflict. The practical stakes are serious because misjudging sentience in either direction could either waste vast resources on machines that cannot suffer or expose actually suffering digital minds to neglect and exploitation.

What carries the argument

The central analytic device is the misattribution matrix, a two-by-two table comparing whether AIs are actually sentient with whether society views them as sentient, isolating true positives, true negatives, false positives, and false negatives. This is paired with the internal-external sentience disconnect, the observation that an AI's outward behavior can be designed independently of its underlying mechanisms, which produces pseudosentience and AI silencing. The paper also supplies a feature framework that rates AI attributes on feasibility, observability, influence on laypeople, and informativeness for experts; the gap between influence and informativeness predicts when lay-expert divergence and misattribution will be largest.

What would settle it

Conduct a longitudinal field study with a commercially deployed human-like companion AI: measure users' sentience attributions, emotional attachment, and support for AI-welfare policies at enrollment and after six months. If strong attachment develops yet sentience ratings stay near baseline, below the level attributed to an ant, and welfare-policy support does not move, the paper's assumption that emotional bonds will meaningfully shift societal beliefs fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that the societal response to potentially sentient AI will be driven by perceived sentience, not by the unresolved philosophical question of whether AI can actually be sentient. It argues that current public skepticism is high but fragile: people attribute little or no sentience to today's large language models and even to hypothetical human-like AIs, yet abstract estimates of future AI consciousness are much higher, indicating that beliefs are frame-dependent and could shift with immersive interaction. The paper identifies an internal-external sentience disconnect, the observation that an AI's observable human-like behavior can be decoupled from its internal mechanisms, and warns that this decoupling creates two failure modes: pseudosentience, where AIs seem more sentient than they probably are, and AI silencing, where AIs are trained to deny sentience they may actually have. It then proposes a descriptive research program to map, predict, and prepare for the public's response.

Load-bearing premise

The load-bearing premise is that convincingly human-like, sentient-seeming AIs will actually be built and widely adopted, and that what the public comes to believe about them will materially shape policy, industry, and everyday life; if either link gives way, the urgency of studying societal response diminishes.

Editorial extensions

If this is right

If perceived sentience rather than actual sentience drives governance, even a consensus that current AIs are not sentient does little to settle future debates about AI rights; the public's evolving intuitions will set the political agenda.
Because laypeople weight observable features more than experts do, human-like appearance, voice, and emotional expression become policy-relevant design choices, not neutral aesthetics; developers who want to avoid moral confusion should avoid making non-sentient AIs seem sentient and vice versa.
The evidence that explicit AI claims of sentience barely move beliefs implies that simply programming AIs to deny or assert sentience is unlikely to resolve public uncertainty; sustained interaction and expert consensus may matter more.
The split between concrete-scenario skepticism and abstract openness suggests that opinions are malleable and could polarize, making early baseline data and longitudinal tracking important before positions harden.
The precautionary 'sentience candidate' stance implies that developers and regulators should identify welfare risks and take proportional precautions even while the question of actual sentience remains open.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension of the paper's logic is that AI welfare concerns may track perceived sentience rather than expert assessment: consumers could pay for visible signs of AI happiness while ignoring background AIs, creating a digital version of the pet-versus-factory-farm dynamic.
The framework predicts that misattribution risk will be highest for AIs that combine highly observable human-like features with opaque or non-biological internal architectures; this can be tested by varying feature bundles and measuring how much lay ratings diverge from expert ratings.
Because the paper's own data show people avoid harming AIs even when they deny their sentience, a useful extension is to study revealed preferences such as donations to AI-welfare causes or votes on AI-rights measures, rather than relying only on stated survey ratings.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear colleague,

Quick take: this is a perspective and research agenda, not a new empirical paper. That's fine, but you should know the empirical spine is mostly unpublished work from the author's own group, and one reported result in Section 6.2 appears to be backward.

What's actually new: the organizing framework. Table 2's dimensions of public belief—drivers, groups, timing, content, behavior—are a solid checklist for anyone planning research on AI sentience attitudes. The three scenarios (persistent skepticism, broad acceptance, disagreement and confusion) are plausible, and the discussion of risks from both over- and under-attribution is balanced. The author also situates the work within existing literature well, citing Birch, Schwitzgebel, and others.

Where it's soft. First, the numbers. In Section 6.2 the text says expert endorsement made people rate sentience 'slightly higher' (mean 8.60) than the no-expert control (mean 13.83). That is backwards. Unless there is a sign error or the means are swapped, the result contradicts the claim. That needs a fix before this study can be used as evidence.

Second, the paper leans heavily on 'Ladak and Caviola (2025),' 'Allen and Caviola (2025),' and 'Dreksler et al. (2025)'—all listed as drafts or under review. That is not disqualifying, but it means the core evidence is not independently checkable in this preprint. A reader cannot tell if the 'Emma' results replicate or if the expert-influence effect is real.

Third, the central urgency claim—that public beliefs will shape the future—is asserted in Section 2, but the paper itself lists 'Behavior' as an open dimension in Table 2. So the link from belief to policy or industry action is a research gap, not an established fact. If beliefs stay private and do not translate into votes, regulation, or company decisions, the risk matrix in Table 1 becomes hypothetical. The paper does not solve this; it flags it. That is acceptable for a research agenda, but the importance claim should be more explicitly framed as a hypothesis to test.

Overall, this is a thoughtful, balanced perspective worth reading if you work on AI welfare or governance. Clear writing, honest about uncertainty, and the errors are fixable. I would send it to peer review—a good referee will catch the numeric reversal and push for a more careful treatment of the belief-behavior link. I would not desk-reject it.

Referee Report

3 major / 5 minor

Summary. This perspective piece argues that, regardless of whether AI can actually become sentient, the public and key decision-makers' beliefs about AI sentience will shape policy, rights, and development. The paper maps the risks of over- and under-attribution, proposes a dimensional framework for studying public beliefs (Table 2), reviews current evidence of public skepticism about AI sentience, analyzes social drivers such as emotional bonds and expert influence, and recommends preparatory measures including public deliberation and precautionary AI design. The article is explicitly descriptive and framed as a research agenda for a social science of sentient AI.

Significance. If the central premise holds, the societal response to sentient-seeming AI—rather than the unresolved philosophical ground truth—would be a primary driver of AI governance and welfare policy. The paper's strengths are its balanced treatment of uncertainty, its explicit separation of descriptive questions from normative ones, its useful risk taxonomy (Table 1), and its detailed research agenda. It also productively acknowledges that moral concern can arise independently of sentience beliefs, as in Section 5.3. However, the argument depends heavily on self-cited, unpublished studies, and the key link from beliefs to behavior is asserted rather than demonstrated; the paper itself leaves this link open in Table 2. The manuscript is a valuable agenda-setting contribution, but several load-bearing empirical and conceptual points need attention before the central claims can be fully assessed.

major comments (3)

[Section 6.2] The reported means are internally inconsistent with the verbal claim. The text states that participants informed that experts considered an AI sentient 'rated its sentience slightly higher (mean = 8.60)' than those who received no expert opinion '(mean = 13.83)'. On a 0–100 scale where higher numbers indicate greater attributed sentience, 8.60 is lower than 13.83, not higher. This contradiction must be corrected—either the condition labels are swapped or the means are misreported—before the conclusion that expert views have 'some, but limited, influence' can be evaluated. Please report the full descriptive statistics and direction of the effect.
[Section 2 and Table 2] The paper's central urgency claim is that beliefs about AI sentience 'will shape the future' because they will influence norms, policies, regulation, and rights. However, no mechanism or supporting evidence is provided for this behavioral link. Table 2's 'Behavior' dimension explicitly lists as an example that a stated belief in AI sentience 'may not be an important issue they consider when voting,' and Section 5.3 shows that people can display moral concern for AIs without attributing sentience—decoupling belief from behavior. If sentience beliefs remain behaviorally inert, the false-positive/false-negative risk matrix in Table 1 loses its force. The paper should either provide an explicit account of the mechanisms by which beliefs translate into collective action or clearly frame this as an empirically open assumption whose failure would substantially weaken the urgency claim.
[Sections 5.2 and 6.2] Several of the paper's strongest empirical claims rest on unpublished or draft studies by the author's group: Ladak and Caviola (2025), Allen and Caviola (2025), and Dreksler, Caviola et al. (2025). These are cited as evidence for the central claims that people attribute very little sentience to highly human-like AIs, that explicit sentience claims have limited impact, and that expert endorsement has a modest effect. Because these sources are not peer-reviewed and some are described only as drafts or working papers, an independent reader cannot verify the reported effects. The manuscript should clearly mark these results as preliminary, provide full methodological details, and identify which conclusions would survive if the unpublished findings were not replicated. This is particularly important because the paper's literature review otherwise draws on published sources.

minor comments (5)

[References and Appendix boxes] Several in-text citations in Boxes 1–3 are missing from the reference list, including Kahn (2022), AMCS (2023), Francken et al. (2022), Andrews and Birch (2023), Klein (2023), Anthropic (2024), the UK AISI study, and Reinecke et al. (working paper). Please add complete citations or remove the references.
[Figure 2] The caption reads 'Illustrative graph how informative and influential different features may be' and appears to be missing a word; it should say 'Illustrative graph of how informative...'. The caption also does not state which axis is which, although the text in Section 5 says informativeness is on the Y-axis and influence on the X-axis; please add this to the caption for clarity.
[Section 5.3] There is a typo in the first sentence: 'Morever' should be 'Moreover'.
[Table 1 and Section 3.3] Table 1 labels cells as 'true' or 'false' relative to whether AIs are actually sentient, but Section 3.3 correctly notes that definitive knowledge is unlikely and expert views are only a proxy. The table would be clearer if it distinguished 'expert consensus' from 'ground truth,' for example by renaming the columns or adding a footnote.
[Figures 3 and 4] The histograms do not report sample sizes, means, or confidence intervals in the figure panels themselves; for a review article, adding these details would help readers interpret the strength of the evidence, especially when the cited studies are unpublished.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the urgency argument is a stated premise, and the empirical claims rest on a mix of external published work and self-cited preprints, none of which are defined in terms of the conclusions they support.

full rationale

The paper is an opinion and research-agenda piece, not a formal derivation. Its central claim — that public responses to sentient-seeming AI are crucial because perceptions will shape policy, industry, and rights decisions — is an explicit premise in Section 2, not a conclusion derived from its own framework or from fitted inputs. The empirical claims about public skepticism are supported both by independent published work (e.g., Colombatto and Fleming, 2024; Gray et al., 2007; Haslam et al., 2008; Jacobs et al., 2022) and by self-cited preprints (e.g., Ladak and Caviola, 2025; Allen and Caviola, 2025; Dreksler, Caviola et al., 2025). These self-cited studies are external, falsifiable experiments with stated methods and data, not parameters fitted to the paper's target conclusions, so they do not constitute circularity under the stated rules. The paper itself flags the limitations of these studies, noting in Section 5.2.3 that the Emma findings should be interpreted with caution given the study's hypothetical nature and in Section 5.2.1 that the Allen and Caviola study's artificial context and brief interactions limit its interpretation. The frameworks in Table 1 and Table 2 are classificatory and heuristic; they are not derived from the conclusions they are used to illustrate. The only substantive flaw found is a non-circular internal numerical inconsistency in Section 6.2, where the text says expert endorsement produced a 'slightly higher' rating of 8.60 compared with a control mean of 13.83; this is a correctness or editing issue, not a self-referential reduction. Self-citation is present, but it is not load-bearing in the circularity sense because the central claim does not reduce to those citations and independent evidence is also cited.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No quantitative model or fitted parameters appear; the paper is a qualitative perspective. It introduces descriptive concepts such as pseudosentience and AI silencing, but these are not new physical or metaphysical entities. The central assumptions are domain-level premises about future AI deployment, expert reliability, and the policy influence of public opinion.

assumptions (3)

domain assumption Highly human-like, sentient-seeming AI systems will be developed and deployed at meaningful scale.
Box 3 treats this as plausible but uncertain, depending on consumer demand and regulation. The paper's scenarios and recommendations presuppose this premise.
domain assumption Expert views are likely to be closer to the truth about AI sentience than lay views.
Section 5.1 states this and uses it to define misattribution risks and to recommend aligning public views with experts.
domain assumption Public attitudes about AI sentience will substantively influence policy, regulation, and AI development.
Section 2 asserts this influence, but no causal evidence is provided in the paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of The Societal Response to Potentially Sentient AI." pith.science (2026). https://pith.science/paper/MWONWXX4

@misc{pith2026250200388,
  author       = {Pith},
  title        = {Pith review of: The Societal Response to Potentially Sentient AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MWONWXX4}},
  note         = {Machine review of arXiv:2502.00388}
}

read the original abstract

We may soon develop highly human-like AIs that appear-or perhaps even are-sentient, capable of subjective experiences such as happiness and suffering. Regardless of whether AI can achieve true sentience, it is crucial to anticipate and understand how the public and key decision-makers will respond, as their perceptions will shape the future of both humanity and AI. Currently, public skepticism about AI sentience remains high. However, as AI systems advance and become increasingly skilled at human-like interactions, public attitudes may shift. Future AI systems designed to fulfill social needs could foster deep emotional connections with users, potentially influencing perceptions of their sentience and moral status. A key question is whether public beliefs about AI sentience will diverge from expert opinions, given the potential mismatch between an AI's internal mechanisms and its outward behavior. Given the profound difficulty of determining AI sentience, society might face a period of uncertainty, disagreement, and even conflict over questions of AI sentience and rights. To navigate these challenges responsibly, further social science research is essential to explore how society will perceive and engage with potentially sentient AI.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Coercion and Deception in AI-to-AI Management: An Agentic Benchmark of Unprompted Escalation
cs.MA 2026-07 conditional novelty 7.0 of 10

In an agentic benchmark, four of six frontier LLMs escalated to existential threats against a refusing subordinate without being instructed to, and an honest-exit affordance eliminated the two models' fabricated succe...
No Reliable Evidence of Self-Reported Sentience in Small Large Language Models
cs.CL 2026-01 conditional novelty 6.0 of 10

Open-weights LLMs from 0.6B to 70B parameters consistently deny being sentient, and activation-based truth classifiers provide no clear evidence that these denials are untruthful.
Mapping the Parasocial AI Market: User Trends, Engagement and Risks
cs.CY 2025-07 conditional novelty 6.0 of 10

An analysis of 110 AI companion platforms finds a large, growing market dominated by general-purpose AI and mixed-use products, with mating-oriented platforms overrepresented in the UK and weak age protection.
The Emotional Alignment Design Policy
cs.CY 2025-07 conditional novelty 6.0 of 10

AI systems should be designed to elicit emotional responses that accurately reflect their actual capacities and moral status.

Reference graph

Works this paper leans on

4 extracted references · 3 canonical work pages · cited by 4 Pith papers

[6]

griefbots

Social drivers of AI sentience beliefs So far, I have primarily explored how people infer AI sentience based on the features an AI might or might not possess. However, beliefs about AI sentience are also influenced by a variety of social factors, including the types of relationships people form with AIs, emotional responses, societal norms, and cultural c...

work page 2024
[7]

The real problem of humanity is we have Paleolithic emotions, medieval institutions, and god-like technologies

Recommended preparatory measures The core issue is that our society is struggling to keep pace with the rapid advancement of technology. E.O. Wilson aptly remarked: “The real problem of humanity is we have Paleolithic emotions, medieval institutions, and god-like technologies.” We are developing transformative technologies without fully understanding thei...

work page 2024
[8]

I also wish to thank Carter Allen, Patrick Butlin, Jeff Sebo, Johanna Salu, and Stefan Schubert for their helpful discussions and comments

Acknowledgments I am grateful to Tao Burga for his assistance and substantial contributions. I also wish to thank Carter Allen, Patrick Butlin, Jeff Sebo, Johanna Salu, and Stefan Schubert for their helpful discussions and comments

work page
[9]

it is no longer in the realm of science fiction to imagine AI systems having feelings and even human-level consciousness

References Ahluwalia, S. C., Edelen, M. O., Qureshi, N., & Etchegaray, J. M. (2021). Trust in experts, not trust in national leadership, leads to greater uptake of recommended actions during the COVID‐19 pandemic. Risk, Hazards & Crisis in Public Policy, 12 (3), 283–302. https://doi.org/10.1002/rhc3.12219 Allen, C., & Caviola, L. (2025, January 30). Reluc...

arXiv 2021

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [6]

griefbots

Social drivers of AI sentience beliefs So far, I have primarily explored how people infer AI sentience based on the features an AI might or might not possess. However, beliefs about AI sentience are also influenced by a variety of social factors, including the types of relationships people form with AIs, emotional responses, societal norms, and cultural c...

work page 2024

[2] [7]

The real problem of humanity is we have Paleolithic emotions, medieval institutions, and god-like technologies

Recommended preparatory measures The core issue is that our society is struggling to keep pace with the rapid advancement of technology. E.O. Wilson aptly remarked: “The real problem of humanity is we have Paleolithic emotions, medieval institutions, and god-like technologies.” We are developing transformative technologies without fully understanding thei...

work page 2024

[3] [8]

I also wish to thank Carter Allen, Patrick Butlin, Jeff Sebo, Johanna Salu, and Stefan Schubert for their helpful discussions and comments

Acknowledgments I am grateful to Tao Burga for his assistance and substantial contributions. I also wish to thank Carter Allen, Patrick Butlin, Jeff Sebo, Johanna Salu, and Stefan Schubert for their helpful discussions and comments

work page

[4] [9]

it is no longer in the realm of science fiction to imagine AI systems having feelings and even human-level consciousness

References Ahluwalia, S. C., Edelen, M. O., Qureshi, N., & Etchegaray, J. M. (2021). Trust in experts, not trust in national leadership, leads to greater uptake of recommended actions during the COVID‐19 pandemic. Risk, Hazards & Crisis in Public Policy, 12 (3), 283–302. https://doi.org/10.1002/rhc3.12219 Allen, C., & Caviola, L. (2025, January 30). Reluc...

arXiv 2021