Mapping Perceptions of Humanness in Speech-Based Intelligent Personal Assistant Interaction
Pith reviewed 2026-05-24 15:18 UTC · model grok-4.3
The pith
Users perceive humanness in speech assistants through eight themes and view them as more formal, fact-based, impersonal and less authentic than humans.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Through analysis of constructs from the repertory grid comparisons, perceptions of humanness are multidimensional, focusing on eight key themes: partner knowledge set, interpersonal connection, linguistic content, partner performance and capabilities, conversational interaction, partner identity and role, vocal qualities and behavioral affordances. Through these themes, it is clear that users define the capabilities of speech interfaces differently to humans, seeing them as more formal, fact based, impersonal and less authentic.
What carries the argument
The repertory grid technique applied to participants' comparisons of dialogues with a human versus two speech-based intelligent personal assistants, which elicits personal constructs later grouped into eight perceptual themes.
If this is right
- The eight themes provide a scaffold for categorizing and targeting future research and design work on speech interfaces.
- Design choices can address specific themes such as vocal qualities or conversational interaction to adjust perceived humanness.
- Emulating humanness should be evaluated for appropriateness by considering whether the interface is positioned as formal and fact-based or interpersonal.
- Interfaces can lean into strengths like factual accuracy rather than attempting to replicate interpersonal connection or authenticity.
Where Pith is reading between the lines
- The themes could guide context-specific rules, such as increasing formality in task-oriented assistants while avoiding it in social ones.
- Testing whether the themes predict measurable outcomes like user trust or continued use would extend the work beyond perception mapping.
- Similar distinctions might appear in non-speech AI interactions, suggesting the framework could apply to text or visual interfaces.
Load-bearing premise
The repertory grid technique applied to 21 participants yields a stable, generalizable set of themes that accurately reflect how users conceptualize humanness in speech interfaces.
What would settle it
A follow-up study with a larger and demographically varied sample that elicits a substantially different set of themes or fails to replicate the reported differences in formality, fact-focus, impersonality, and authenticity between humans and speech interfaces.
read the original abstract
Humanness is core to speech interface design. Yet little is known about how users conceptualise perceptions of humanness and how people define their interaction with speech interfaces through this. To map these perceptions n=21 participants held dialogues with a human and two speech interface based intelligent personal assistants, and then reflected and compared their experiences using the repertory grid technique. Analysis of the constructs show that perceptions of humanness are multidimensional, focusing on eight key themes: partner knowledge set, interpersonal connection, linguistic content, partner performance and capabilities, conversational interaction, partner identity and role, vocal qualities and behavioral affordances. Through these themes, it is clear that users define the capabilities of speech interfaces differently to humans, seeing them as more formal, fact based, impersonal and less authentic. Based on the findings, we discuss how the themes help to scaffold, categorise and target research and design efforts, considering the appropriateness of emulating humanness.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that perceptions of humanness in speech-based IPAs are multidimensional, based on a repertory grid study with n=21 participants who compared interactions with a human and two IPAs. Thematic analysis yields eight constructs (partner knowledge set, interpersonal connection, linguistic content, partner performance and capabilities, conversational interaction, partner identity and role, vocal qualities and behavioral affordances), and users view IPAs as more formal, fact-based, impersonal, and less authentic than humans. The work positions these themes as a scaffold for design and research on emulating humanness.
Significance. If the themes prove stable, the study offers a structured way to categorize user perceptions that could inform when humanness emulation is appropriate in IPA design. The repertory grid approach is a methodological strength for eliciting participant-generated constructs rather than imposing predefined categories.
major comments (2)
- [Methods] Methods section: the thematic grouping of repertory grid constructs into the eight themes reports no inter-rater reliability statistics, saturation criteria, member checking, or hold-out validation. This directly undermines the central claim that the themes constitute a reliable mapping of user conceptualizations.
- [Results] Results/Discussion: the assertion that the eight themes are generalizable enough to 'scaffold, categorise and target research and design efforts' rests on a single sample of 21 without cross-validation or replication evidence; the observed differences (e.g., IPAs seen as more formal/fact-based) could reflect sample idiosyncrasies or method artifacts.
minor comments (1)
- [Abstract] Abstract: 'Analysis of the constructs show' should specify the exact qualitative procedure used to derive the eight themes.
Simulated Author's Rebuttal
We are grateful to the referee for their detailed and constructive feedback on our manuscript. We respond to each of the major comments below, indicating where revisions will be made to address the concerns raised.
read point-by-point responses
-
Referee: [Methods] Methods section: the thematic grouping of repertory grid constructs into the eight themes reports no inter-rater reliability statistics, saturation criteria, member checking, or hold-out validation. This directly undermines the central claim that the themes constitute a reliable mapping of user conceptualizations.
Authors: We agree that the Methods section would benefit from greater transparency regarding the thematic analysis process. The grouping of constructs into themes was performed iteratively by the authors based on semantic similarity and patterns across the repertory grid data. No formal inter-rater reliability was calculated as the analysis was primarily interpretive and led by one researcher with review by co-authors. We will revise the Methods section to describe the analysis steps in more detail, including how saturation was approached through the number of participants and construct elicitation. We will also adjust the abstract and discussion to avoid overstating the reliability of the themes as a mapping. revision: yes
-
Referee: [Results] Results/Discussion: the assertion that the eight themes are generalizable enough to 'scaffold, categorise and target research and design efforts' rests on a single sample of 21 without cross-validation or replication evidence; the observed differences (e.g., IPAs seen as more formal/fact-based) could reflect sample idiosyncrasies or method artifacts.
Authors: The referee is correct that the study is based on a single sample of 21 participants and does not include cross-validation. We position the eight themes as an initial framework derived from this exploratory study to help scaffold future research, not as a fully generalizable model. The differences noted are supported by the data from this sample and consistent with related work. In the revised manuscript, we will emphasize the exploratory character of the study, note the limitations of the sample size, and suggest that future work should validate the themes through replication. This will temper the claims while preserving the contribution of the identified themes. revision: partial
- The study does not include replication or cross-validation data, which cannot be addressed without new empirical work.
Circularity Check
No circularity: qualitative thematic analysis grounded in participant constructs with no derivations or self-referential reductions
full rationale
The paper applies the repertory grid technique to elicit constructs from 21 participants comparing human and IPA interactions, then performs thematic analysis to identify eight themes. No equations, fitted parameters, predictions, or mathematical derivations exist. The central claim (multidimensional humanness perceptions captured by the eight themes) is presented as emerging directly from the elicited constructs rather than reducing to prior self-citations, internal definitions, or renamed inputs. No load-bearing self-citation chains or ansatzes are invoked. The method is self-contained against external benchmarks in the sense that the output themes are explicitly tied to the participant data without circular redefinition.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption The repertory grid technique accurately elicits and structures user perceptions of humanness in speech interfaces.
Reference graph
Works this paper leans on
-
[1]
Kei Akuzawa, Yusuke Iwasawa, and Yutaka Matsuo. 2018. Ex- pressive Speech Synthesis via Modeling Expressions with Variational Autoencoder. In Interspeech 2018 . ISCA, 3067–3071. https://doi.org/10.21437/Interspeech.2018-1113
-
[2]
René Amalberti, Noëlle Carbonell, and Pierre Falzon. 1993. Use r rep- resentations of computer systems in human-computer speech interac- tion. International Journal of Man-Machine Studies 38, 4 (April 1993), 547–566. https://doi.org/10.1006/imms.1993.1026
-
[3]
Matthew P Aylett, Benjamin R Cowan, and Leigh Clark. 2019. Siri, Echo and Performance: You have to Suffer Darling. In Extended Ab- stracts of the 2019 CHI Conference on Human Factors in Comput ing Systems. ACM, alt08
work page 2019
-
[4]
Holly P. Branigan, Martin J. Pickering, Jamie Pearson, Janet F. McLean, and Ash Brown. 2011. The role of beliefs in lexical alignment: Ev- idence from dialogs with humans and computers. Cognition 121, 1 (Oct. 2011), 41–57. https://doi.org/10.1016/j.cognition.201 1.05.011
-
[5]
Cynthia Breazeal. 2003. Emotion and sociable humanoid robots. In- ternational journal of human-computer studies 59, 1-2 (2003), 119–155
work page 2003
-
[6]
Susan E Brennan and Herbert H Clark. 1996. Conceptual pacts and lexical choice in conversation. Journal of Experimental Psychology: Learning, Memory, and Cognition 22, 6 (1996), 1482
work page 1996
-
[7]
Allison Bruce, Illah Nourbakhsh, and Reid Simmons. 2002. The r ole of expressiveness and attention in human-robot interaction. In Proceed- ings 2002 IEEE International Conference on Robotics and Aut omation (Cat. No. 02CH37292), Vol. 4. IEEE, 4138–4142
work page 2002
-
[8]
Justine Cassell. 2001. Embodied conversational agents: repres entation and intelligence in user interfaces. AI magazine 22, 4 (2001), 67
work page 2001
-
[9]
Vincent Cho and Robert Wright. 2010. Exploring the evaluation framework of strategic information systems using repertory grid tech- nique: a cognitive perspective from chief information officers. Be- haviour & Information Technology 29, 5 (2010), 447–457
work page 2010
-
[10]
Leigh Clark. 2018. Social Boundaries of Appropriate Speech in HCI: A Politeness Perspective. In Proceedings of the 32nd British Human Computer Interaction Conference. British Computer Societ y. https://doi. org/10.14236/ewic/HCI2018, Vol. 76
- [11]
-
[12]
Leigh Clark, Abdulmalik Ofemile, Svenja Adolphs, and Tom Rod den
-
[13]
ACM Transactions on Interactive Intelligent Systems (TiiS) 6, 4 (2016), 29
A multimodal approach to assessing user experiences with agent helpers. ACM Transactions on Interactive Intelligent Systems (TiiS) 6, 4 (2016), 29
work page 2016
-
[14]
Leigh Clark, Nadia Pantidi, Orla Cooney, Philip Doyle, Diego Gara - ialde, Justin Edwards, Brendan Spillane, Christine Murad, Cos- min Munteanu, Vincent Wade, and Benjamin R. Cowan. 2019. What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents. arXiv:1901.06525 [cs] (Jan. 2019). https://doi.org/10.1145/3290605.3300705 arXiv: 19...
work page internal anchor Pith review Pith/arXiv arXiv doi:10.1145/3290605.3300705 2019
-
[15]
Benjamin R Cowan and Holly P Branigan. 2015. Does voice anthropo- morphism affect lexical alignment in speech-based human-computer dialogue?. In Sixteenth Annual Conference of the International Speech Communication Association
work page 2015
-
[16]
Benjamin R Cowan and Holly P Branigan. 2017. They Know as Much as We Do: Knowledge Estimation and Partner Modelling of Artificial Partners
work page 2017
-
[17]
Benjamin R. Cowan, Holly P. Branigan, Mateo Obregón, Enas Bugis, and Russell Beale. 2015. Voice anthropomorphism, interlocutor mo d- elling and alignment effects on syntactic choices in human−computer dialogue. International Journal of Human-Computer Studies 83 (Nov. 2015), 27–42. https://doi.org/10.1016/j.ijhcs.2015.05.0 08
-
[18]
Benjamin R Cowan, Nadia Pantidi, David Coyle, Kellie Morrissey, Pe - ter Clarke, Sara Al-Shehri, David Earley, and Natasha Bandeira. 2 017. What can i help you with?: infrequent users’ experiences of intelli- gent personal assistants. In Proceedings of the 19th International Con- ference on Human-Computer Interaction with Mobile Devices and Ser- vices. ACM, 43
-
[19]
Laurence Devillers, Sophie Rosset, Guillaume Dubuisson Dup lessis, Lucile Bechade, Yucel Yemez, Bekir B Turker, Metin Sezgin, Engin Erzin, Kevin El Haddad, Stephane Dupont, et al. 2018. Multifaceted Engagement in Social Interaction with a Machine: The JOKER Project. In 2018 13th IEEE International Conference on Automatic Face &Gesture Recognition (FG 2018...
work page 2018
-
[20]
Penny Dick and Devi Jankowicz. 2001. A social construction- ist account of police culture and its influence on the represen- tation and progression of female officers: A repertory grid anal- ysis in a UK police force. Policing: An International Journal of Police Strategies & Management 24, 2 (June 2001), 181–199. https://doi.org/10.1108/13639510110390936
-
[21]
Mateusz Dubiel, Martin Halvey, and Leif Azzopardi. 2018. A Su rvey Investigating Usage of Virtual Personal Assistants. arXiv:1807.04606 [cs] (July 2018). http://arxiv.org/abs/1807.04606 arXiv: 1807. 04606
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[22]
Jens Edlund, Julia Bell Hirschberg, and Mattias Heldner. 2009. Pause and gap length in face-to-face interaction. Columbia University (2009). https://doi.org/10.7916/d82f7wt9
-
[23]
Daniel Fallman and John Waterworth. 2010. Capturing user exp eri- ences of mobile information technology with the repertory grid tech- nique. Human Technology: An Interdisciplinary Journal on Humans i n ICT Environments (2010)
work page 2010
-
[24]
Hao Fang, Hao Cheng, Elizabeth Clark, Ariel Holtzman, Maarte n Sap, Mari Ostendorf, Yejin Choi, and Noah A Smith. 2017. Sounding board– university of washington’s alexa prize submission. (2017)
work page 2017
-
[25]
Fay Fransella, Richard Bell, and D Bannister. 2004. A manual for reper- tory grid technique . Wiley, Hoboken, NJ. OCLC: 53978922
work page 2004
-
[26]
Moojan Ghafurian, Neil Budnarain, and Jesse Hoey. 2019. Role of Emo- tions in Perception of Humanness of Virtual Agents. In Proceedings of the 18th International Conference on Autonomous Agents and MultiA- gent Systems. International Foundation for Autonomous Agents and Multiagent Systems, 1979–1981
work page 2019
-
[27]
Emer Gilmartin, Marine Collery, Ketong Su, Yuyun Huang, Christy Elias, Benjamin R. Cowan, and Nick Campbell. 2017. Social talk: mak- ing conversation with people and machine. In Proceedings of the 1st ACM SIGCHI International Workshop on Investigating Social Interac- tions with Artificial Agents - ISIAA 2017. ACM Press, Glasgow, UK, 31–
work page 2017
-
[28]
https://doi.org/10.1145/3139491.3139494
-
[29]
Nick Haslam. 2006. Dehumanization: An integrative review. Person- ality and social psychology review 10, 3 (2006), 252–264
work page 2006
-
[30]
Nick Haslam, Paul Bain, Lauren Douge, Max Lee, and Brock Bast ian
-
[31]
Journal of personality and social psychology 89, 6 (2005), 937
More human than you: Attributing humanness to self and oth- ers. Journal of personality and social psychology 89, 6 (2005), 937
work page 2005
-
[32]
Chin-Chang Ho and Karl F MacDorman. 2017. Measuring the un- canny valley effect. International Journal of Social Robotics 9, 1 (2017), 129–139
work page 2017
-
[33]
Trevor Hogan and Eva Hornecker. 2013. Blending the repertory grid technique with focus groups to reveal rich design relevant insight. In Proceedings of the 6th International Conference on Designi ng Pleasur- able Products and Interfaces - DPPI ’13 . ACM Press, Newcastle upon Tyne, United Kingdom, 116. https://doi.org/10.1145/2513506.25 13519
-
[34]
Devi Jankowicz. 2004. The easy guide to repertory grids . Wiley, Chich- ester, West Sussex, England ; Hoboken, N.J
work page 2004
-
[35]
George Kelly. 1991. The psychology of personal constructs. Vol. 2, Vol. 2,. Routledge in association with the Centre for Personal Construct Psychology, London; New York. OCLC: 973052807
work page 1991
-
[36]
Alan Kennedy, A Wilkes, L Elder, and Wayne Murray
-
[37]
Dialogue with machines. Cognition 30 (1988), 37–72. https://doi.org/10.1016/0010-0277(88)90003-0
-
[38]
Yoshinori Kuno, Kazuhisa Sadazuka, Michie Kawashima, Keiichi Ya - mazaki, Akiko Yamazaki, and Hideaki Kuzuoka. 2007. Museum guide robot based on sociological interaction analysis. In Proceedings of the SIGCHI conference on Human factors in computing systems . ACM, 1191–1194
work page 2007
-
[39]
Ludovic Le Bigot, Jean-François Rouet, and Eric Jamet. 2007. Ef - fects of Speech- and Text-Based Interaction Modes in Natural Lan- guage Human-Computer Dialogue. Human Factors: The Journal of the Human Factors and Ergonomics Society 49, 6 (Dec. 2007), 1045–1053. https://doi.org/10.1518/001872007X249901
-
[40]
Lucian Leahu, Marisa Cohn, and Wendy March. 2013. How categor ies come to matter. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’13. ACM Press, Paris, France, 3331. https://doi.org/10.1145/2470654.2466455
-
[41]
Kwan-Min Lee and Clifford Nass. 2005. Social-Psychological Or i- gins of Feelings of Presence: Creating Social Presence With Machine- Generated Voices. Media Psychology 7, 1 (Feb. 2005), 31–45. https://doi.org/10.1207/S1532785XMEP0701_2
-
[42]
Ewa Luger and Abigail Sellen. 2016. Like having a really bad PA: the gulf between user expectation and experience of conversational agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 5286–5297
work page 2016
-
[43]
Meddeb and Patricia Frenz-Belkin
Elizabeth J. Meddeb and Patricia Frenz-Belkin. 2010. What? I Didn’t Say THAT!: Linguistic strategies when speaking to write. Journal of Pragmatics 42, 9 (Sept. 2010), 2415–2429. https://doi.org/10.1016/j.pragma.2009.12.022
-
[44]
Moore, Hui Li, and Shih-Hao Liao
Roger K. Moore, Hui Li, and Shih-Hao Liao. 2016. Progress and Prospects for Spoken Language Technology: What Ordinary People Think. 3007–3011. https://doi.org/10.21437/Interspeech.20 16-874
-
[45]
Masahiro Mori, Karl F MacDorman, and Norri Kageki. 1970. 2012 . The Uncanny Valley [From the Field]. Robotics & Automation Magazine, IEEE 19, 2 (1970), 2012
work page 1970
-
[46]
Sharon Oviatt, Jon Bernard, and Gina-Anne Levow. 1998. Lin- guistic Adaptations During Spoken and Multimodal Error Res- olution. Language and Speech 41, 3-4 (July 1998), 419–442. https://doi.org/10.1177/002383099804100409
-
[47]
Ioannis Papaioannou, Amanda Cercas Curry, Jose L Part, Igor Sha - lyminov, Xinnuo Xu, Yanchao Yu, Ondrej Dušek, Verena Rieser, and Oliver Lemon. 2017. Alana: Social dialogue using an ensemble model and a ranker trained on user feedback. Alexa Prize Proceedings (2017)
work page 2017
-
[48]
Martin Porcheron, Joel E Fischer, Stuart Reeves, and Sarah S harples
-
[49]
In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
Voice Interfaces in Everyday Life. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems . ACM, 640
work page 2018
-
[50]
Amanda Purington, Jessie G. Taft, Shruti Sannon, Natalya N. Bazarova, and Samuel Hardman Taylor. 2017. "Alexa is my new BFF": Social Roles, User Satisfaction, and Personification of the Amazon Echo. In Proceedings of the 2017 CHI Confer- ence Extended Abstracts on Human Factors in Computing Syste ms - CHI EA ’17 . ACM Press, Denver, Colorado, USA, 2853–285...
-
[51]
Valentin Schwind, Katrin Wolf, and Niels Henze. 2018. Avoiding the uncanny valley in virtual character design. interactions 25, 5 (2018), 45–49
work page 2018
-
[52]
Mildred L.G. Shaw and Laurie F. Thomas. 1978. FOCUS on education—an interactive computer system for the develop- ment and analysis of repertory grids. International Jour- nal of Man-Machine Studies 10, 2 (March 1978), 139–173. https://doi.org/10.1016/S0020-7373(78)80009-1
-
[53]
Brendan Spillane, Emer Gilmartin, Christian Saam, Ketong Su, Ben- jamin R Cowan, Séamus Lawless, and Vincent Wade. 2017. Introduc- ing ADELE: a personalized intelligent companion. In Proceedings of the 1st ACM SIGCHI International Workshop on Investigating Social In- teractions with Artificial Agents . ACM, 43–44
work page 2017
-
[54]
James C Thompson, J Gregory Trafton, and Patrick McKnight. 201 1. The perception of humanness from the movements of synthetic agents. Perception 40, 6 (2011), 695–704
work page 2011
-
[55]
Lai Lai Tung, Yun Xu, and Felix B. Tan. 2009. Attributes of Web S ite Usability: A Study of Web Users with the Repertory Grid Technique. International Journal of Electronic Commerce 13, 4 (July 2009), 97–126. https://doi.org/10.2753/JEC1086-4415130405
-
[56]
Mirjam Wester, Matthew Aylett, Marcus Tomalin, and Rasmus D all
-
[57]
Artificial Personality and Disfluency. Interspeech 2015 (2015), 5
work page 2015
-
[58]
Mapping Perceptions of Humanness in Speech-Based Intelligent Personal Assistant Interaction
Jakub Aleksander Zlotowski, Hidenobu Sumioka, Shuichi Nishio, Dy- lan F Glas, Christoph Bartneck, and Hiroshi Ishiguro. 2015. Persis - tence of the uncanny valley: the influence of repeated interactions and a robot’s attitude on its perception. Frontiers in psychology 6 (2015), 883. This figure "sample-franklin.png" is available in "png" format from: http:...
work page internal anchor Pith review Pith/arXiv arXiv 2015
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.