pith. sign in

arxiv: 2603.24853 · v2 · submitted 2026-03-25 · 💻 cs.AI

Resisting Humanization: Ethical Front-End Design Choices in AI for Sensitive Contexts

Silvia Rossi , Diletta Huyskes , Mackenzie Jorgensen This is my paper

Pith reviewed 2026-05-14 23:54 UTC · model grok-4.3

classification 💻 cs.AI
keywords AI ethicsfront-end designhumanizationconversational user interfacesvalue-sensitive designtrauma-informed designprocedural ethics
0
0 comments X

The pith

Humanizing AI front-end design is an ethical choice that shapes mental models and risks misplaced trust in vulnerable contexts.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that front-end design elements in conversational AI, such as dialogue styles and anthropomorphic features, function as value-laden decisions rather than neutral technical choices. These elements influence how users form expectations, calibrate trust, and exercise autonomy, with particular risks in sensitive settings like support for survivors of gender-based violence. Drawing on human-computer interaction research and value-sensitive design, the analysis shows how deliberate restraint in humanization can align interfaces with user needs. A case study of nonprofit AI tools demonstrates that ethical considerations can override standard engagement-driven norms. The work positions front-end choices as procedural ethics enacted through interaction rather than backend logic alone.

Core claim

Humanization in AI front-end design is a value-driven choice that profoundly shapes users' mental models, trust calibration, and behavioral responses, and in vulnerable contexts this can misalign expectations, foster misplaced trust, and undermine autonomy, as shown through trauma-informed design principles that favor principled restraint over human-like interaction.

What carries the argument

Humanization via dialogue-based interaction, emotive language, personality modes, and anthropomorphic metaphors in conversational user interfaces, which serve as the mechanism for influencing user perceptions and responses.

If this is right

  • Ethical AI development requires treating interface choices as procedural ethics equivalent in importance to algorithmic decisions.
  • In contexts involving vulnerable users, non-humanizing designs can better preserve autonomy by reducing false expectations of human-like understanding.
  • Organizations can operationalize trauma-informed principles by defaulting to restrained interaction styles in AI tools.
  • Contemporary AI product norms favoring engagement metrics may conflict with ethical requirements in sensitive domains.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same restraint logic could apply to AI chatbots in mental health or legal advice where over-humanization might encourage inappropriate reliance.
  • Empirical tests could isolate which specific humanizing features most strongly affect behavioral responses in vulnerable groups.
  • This framing connects front-end ethics to existing work on anthropomorphism limits in other interactive systems.

Load-bearing premise

That humanizing front-end elements are the primary driver of misaligned expectations and misplaced trust, rather than other factors such as content accuracy or individual user background.

What would settle it

A comparative user study measuring trust calibration and perceived autonomy in the same AI system with versus without humanizing interface elements, conducted with participants in simulated sensitive support scenarios.

read the original abstract

Ethical debates in AI have primarily focused on back-end issues such as data governance, model training, and algorithmic decision-making. Less attention has been paid to the ethical significance of front-end design choices, such as the interaction and representation-based elements through which users interact with AI systems. This gap is particularly significant for Conversational User Interfaces (CUI) based on Natural Language Processing (NLP) systems, where humanizing design elements such as dialogue-based interaction, emotive language, personality modes, and anthropomorphic metaphors are increasingly prevalent. This work argues that humanization in AI front-end design is a value-driven choice that profoundly shapes users' mental models, trust calibration, and behavioral responses. Drawing on research in human-computer interaction (HCI), conversational AI, and value-sensitive design, we examine how interfaces can play a central role in misaligning user expectations, fostering misplaced trust, and subtly undermining user autonomy, especially in vulnerable contexts. To ground this analysis, we discuss two AI systems developed by Chayn, a nonprofit organization supporting survivors of gender-based violence. Chayn is extremely cautious when building AI that interacts with or impacts survivors by operationalizing their trauma-informed design principles. This Chayn case study illustrates how ethical considerations can motivate principled restraint in interface design, challenging engagement-based norms in contemporary AI products. We argue that ethical front-end AI design is a form of procedural ethics, enacted through interaction choices rather than embedded solely in system logic.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript argues that front-end design choices in AI systems—particularly humanizing elements such as dialogue-based interaction, emotive language, personality modes, and anthropomorphic metaphors in conversational interfaces—are value-driven decisions with significant ethical implications. These choices, the paper claims, shape users' mental models, calibrate (or miscalibrate) trust, and influence behavioral responses, with heightened risks of misaligned expectations and undermined autonomy in vulnerable contexts such as support for survivors of gender-based violence. Drawing on HCI, conversational AI, and value-sensitive design literature, the work uses the Chayn nonprofit's trauma-informed AI systems as a case study to illustrate principled restraint in humanization, positioning ethical front-end design as a form of procedural ethics enacted through interaction choices rather than solely through back-end logic.

Significance. If the interpretive synthesis holds, the paper would usefully redirect attention in AI ethics from predominantly back-end concerns (data, training, algorithms) to front-end interaction design, especially for sensitive applications. The Chayn case supplies a practical example of operationalizing trauma-informed principles through design restraint, which could inform guidelines for developers working with vulnerable users. The contribution rests on literature integration and case illustration rather than new data or formal models; its value would increase if the causal links between specific humanizing features and outcomes like trust or autonomy were more directly evidenced.

major comments (2)
  1. [Abstract] Abstract and opening sections: The central claim that humanization 'profoundly shapes users' mental models, trust calibration, and behavioral responses' is asserted without direct empirical measurement or controlled comparison in the manuscript. The argument relies on external HCI and ethics citations plus the Chayn illustration, but provides no pre/post metrics, user studies, or isolation of humanization effects from confounders such as content accuracy or user background, which is load-bearing for the 'misaligned expectations' and 'undermined autonomy' assertions.
  2. [Chayn Case Study] Chayn case study section: The discussion of Chayn's two AI systems illustrates restraint in humanizing elements but does not report any behavioral data, trust scales, autonomy measures, or comparative analysis against more humanized baselines. This leaves the demonstration of ethical impact interpretive rather than demonstrated, weakening the claim that such design choices are primary drivers in vulnerable contexts.
minor comments (2)
  1. The manuscript would benefit from an explicit early definition or operationalization of 'humanization' (e.g., listing specific front-end features with examples) to distinguish it from related concepts like anthropomorphism.
  2. A brief table or structured comparison of humanizing vs. restrained design choices across the Chayn systems and a typical commercial CUI would improve clarity and allow readers to assess the specific differences being advocated.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for these constructive comments, which help clarify the scope of our contribution. Our manuscript is a conceptual and ethical analysis that synthesizes existing literature and uses the Chayn case as an illustration of design principles, rather than an empirical study. We address each point below and outline targeted revisions to better articulate this framing.

read point-by-point responses

  1. Referee: [Abstract] Abstract and opening sections: The central claim that humanization 'profoundly shapes users' mental models, trust calibration, and behavioral responses' is asserted without direct empirical measurement or controlled comparison in the manuscript. The argument relies on external HCI and ethics citations plus the Chayn illustration, but provides no pre/post metrics, user studies, or isolation of humanization effects from confounders such as content accuracy or user background, which is load-bearing for the 'misaligned expectations' and 'undermined autonomy' assertions.

    Authors: We agree that the manuscript contains no new empirical measurements, user studies, or controlled comparisons. The central claim is derived from cited prior work in HCI, conversational AI, and value-sensitive design, with the Chayn example serving to illustrate application in a sensitive context. We will revise the abstract and opening sections to explicitly state that the paper offers an interpretive ethical analysis grounded in existing literature, rather than new causal evidence. This clarification will reduce any implication of direct measurement while preserving the argument's focus on front-end design as procedural ethics. revision: partial

  2. Referee: [Chayn Case Study] Chayn case study section: The discussion of Chayn's two AI systems illustrates restraint in humanizing elements but does not report any behavioral data, trust scales, autonomy measures, or comparative analysis against more humanized baselines. This leaves the demonstration of ethical impact interpretive rather than demonstrated, weakening the claim that such design choices are primary drivers in vulnerable contexts.

    Authors: The Chayn case is presented as an illustration of how trauma-informed principles translate into specific interface choices, drawn from Chayn's publicly documented approach. No behavioral data, trust scales, or comparative metrics are reported because such internal evaluation data are not available to the authors. We will revise the case study section to more explicitly label it as illustrative, add a brief discussion of this as a limitation, and note that future empirical work could test the links between design choices and outcomes such as trust calibration. revision: partial

standing simulated objections not resolved
  • We cannot add original empirical data, user studies, or quantitative metrics from Chayn, as the manuscript is conceptual rather than empirical and we lack access to any internal Chayn evaluation data.

Circularity Check

0 steps flagged

No significant circularity; argument grounded in external HCI literature and case illustration without self-referential reduction.

full rationale

The paper advances an ethical argument that humanization in AI front-end design shapes mental models and trust, supported by synthesis of external references in HCI, conversational AI, and value-sensitive design plus the Chayn case study of trauma-informed restraint. No equations, fitted parameters, predictions, or self-citations appear as load-bearing steps. The central claim does not reduce by construction to its own inputs or prior author work; it remains an interpretive synthesis reliant on independent external sources.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on domain assumptions drawn from HCI and value-sensitive design literature plus the specific trauma-informed principles of the Chayn organization; no free parameters or new entities are introduced.

axioms (2)
  • domain assumption Front-end design choices in conversational AI influence users' mental models, trust calibration, and behavioral responses
    Invoked as the mechanism by which humanization produces ethical effects.
  • domain assumption Trauma-informed design principles provide an ethically appropriate basis for restraining humanization in sensitive AI contexts
    Used to justify the Chayn case study and the recommendation for principled restraint.

pith-pipeline@v0.9.0 · 5561 in / 1401 out tokens · 60837 ms · 2026-05-14T23:54:59.812348+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references · 21 canonical work pages

  1. [1]

    Martin Brenncke. 2024. Regulating dark patterns.Notre Dame Journal of Interna- tional & Comparative Law14 (2024), 39–66

    work page 2024

  2. [2]

    Federica Cena and Francesca Grasso. 2025. Exploring users’ mental models of conversational agents.Behaviour & Information Technology(2025), 1–24. doi:10.1080/0144929X.2025.2573436

    work page doi:10.1080/0144929x.2025.2573436 2025

  3. [3]

    Alessandra Cenci and Dylan Cawthorne. 2020. Refining value sensitive design. Science and Engineering Ethics26, 5 (2020), 2629–2662. doi:10.1007/s11948-020- 00223-3

    work page doi:10.1007/s11948-020- 2020

  4. [4]

    Chayn. 2025. A Chayn LLM Case Study (1/2): Little Window.Medium (2025). https://blog.chayn.co/a-chayn-llm-case-study-%C2%BD-little-window- f637b79b3fa0

    work page 2025

  5. [5]

    Chayn. 2025. A Chayn LLM case study (2/2): Survivor AI.Medium(2025). https://blog.chayn.co/a-chayn-llm-case-study-2-2-survivor-ai85b8673c27db

    work page 2025

  6. [6]

    Clara Colombatto, Jonathan Birch, and Stephen M. Fleming. 2025. The influence of mental state attributions on trust in large language models.Communications Psychology3 (2025), 84. doi:10.1038/s44271-025-00262-1

    work page doi:10.1038/s44271-025-00262-1 2025

  7. [7]

    Hera Hussain. 2023. Why Chayn took down its chatbot in 2020 and what we’ve learned about building culturally-aware chatbots since.Medium (2023). https://blog.chayn.co/why-chayn-took-down-its-chatbot-in-2020-and- what-weve-learned-about-culturally-aware-chatbots-a9587cf80df8

    work page 2023

  8. [8]

    2023.Chayn’s trauma-informed design principles: An ex- ploration of how our principles have evolved and how you can put them into practice

    Hera Hussein. 2023.Chayn’s trauma-informed design principles: An ex- ploration of how our principles have evolved and how you can put them into practice. Technical Report. Chayn. https://cdn.prod.website- files.com/60fdc9111506063bb9fe8e49/64b081438e3221d7ffc92b12_Trauma- informed%20design_%20the%20whitepaper%20by%20Chayn.pdf

    work page 2023

  9. [9]

    Lujain Ibrahim, Franziska Sofia Hafner, and Luc Rocher. 2025. Training language models to be warm and empathetic makes them less reliable.arXiv preprint (2025). https://arxiv.org/abs/2507.21919

    work page arXiv 2025

  10. [10]

    Marcello Ienca. 2023. On artificial intelligence and manipulation.Topoi42, 3 (2023), 833–842. doi:10.1007/s11245-023-09940-3

    work page doi:10.1007/s11245-023-09940-3 2023

  11. [11]

    Collins, Lujain Ibrahim, Arina Shah, Petra Ivanovic, Noah Broestl, Gabriel Piles, Paul Dongha, Hatim Abdul- hussein, Adrian Weller, Jillian Powers, and Umang Bhatt

    Mackenzie Jorgensen, Kendall Brogle, Katherine M. Collins, Lujain Ibrahim, Arina Shah, Petra Ivanovic, Noah Broestl, Gabriel Piles, Paul Dongha, Hatim Abdul- hussein, Adrian Weller, Jillian Powers, and Umang Bhatt. 2025. Documenting Deployment with Fabric: A Repository of Real-World AI Governance.Proceedings of the AAAI/ACM Conference on AI, Ethics, and S...

    work page doi:10.1609/aies.v8i2.36636 2025

  12. [12]

    Mackenzie Jorgensen, Hannah Richert, Elizabeth Black, Natalia Criado, and Jose Such. 2023. Not So Fair: The Impact of Presumably Fair Machine Learning Models. InProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (Montréal, QC, Canada)(AIES ’23). Association for Computing Machinery, New York, NY, USA, 297–311. doi:10.1145/3600211.3604699

    work page doi:10.1145/3600211.3604699 2023

  13. [13]

    Scott, Eva Blum-Dumontet, Kayla Evans, Sakina Hansen, Ashley Khor, Nadine Krishnamurthy-Spencer, Mayra Russo, and Sophia Worth

    Mackenzie Jorgensen, Kristen M. Scott, Eva Blum-Dumontet, Kayla Evans, Sakina Hansen, Ashley Khor, Nadine Krishnamurthy-Spencer, Mayra Russo, and Sophia Worth. 2025.From principles to practice, CHAYN’s guide to building a feminist AI: A framework for building trauma- informed, survivor-centered LLM products. Technical Report. Chayn. https://chayn.notion.s...

    work page 2025

  14. [14]

    Horstmann

    Nicole Krämer, Tristan Kühn, and Aike C. Horstmann. 2025. My AI is not my (virtual) friend: When Anthropomorphic Design Cues in Human-Chatbot Interactions do not Foster Trust. InProceedings of the 25th ACM International Conference on Intelligent Virtual Agents. 1–8. doi:10.1145/3717511.3747077

    work page doi:10.1145/3717511.3747077 2025

  15. [15]

    Marie Muehlhaus and Jürgen Steimle. 2024. Interaction design with generative AI.arXiv preprint(2024). https://arxiv.org/abs/2411.02662

    work page arXiv 2024

  16. [16]

    Jakob Nielsen. 1994. Enhancing the explanatory power of usability heuristics. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 152–158. doi:10.1145/191666.191729

    work page doi:10.1145/191666.191729 1994

  17. [17]

    Jakob Nielsen. 2023. AI: First new UI paradigm in 60 years. https://www.nngroup. com/articles/ai-first-new-ui-paradigm/

    work page 2023

  18. [18]

    Geovana Ramos Silva and Edna Dias Canedo. 2024. Human factors in the design of chatbot interactions: Conversational design practices. InProceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems. 1–12. doi:10. 1145/3702038.3702083

    work page arXiv 2024

  19. [19]

    Kyoko Sugisaki and Andreas Bleiker. 2020. Usability guidelines and evaluation criteria for conversational user interfaces. InProceedings of Mensch und Computer

    work page 2020

  20. [20]

    doi:10.1145/3404983.3405505

    309–319. doi:10.1145/3404983.3405505

    work page doi:10.1145/3404983.3405505

  21. [21]

    Sophia Worth, Georgia Panagiotidou, and Elena Simperl. 2025. Brokering Ethics: A Retrospective Study on the Use of a Data Ethics Framework.Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society8, 3 (Oct. 2025), 2732–2743. doi:10.1609/aies.v8i3.36753

    work page doi:10.1609/aies.v8i3.36753 2025