pith. sign in

arxiv: 1907.11585 · v2 · pith:KJEEOXE3new · submitted 2019-07-26 · 💻 cs.HC

Mapping Perceptions of Humanness in Speech-Based Intelligent Personal Assistant Interaction

Pith reviewed 2026-05-24 15:18 UTC · model grok-4.3

classification 💻 cs.HC
keywords humannessspeech interfacesintelligent personal assistantsrepertory grid techniqueperceptionsconversational interactionuser experiencehuman-computer interaction
0
0 comments X

The pith

Users perceive humanness in speech assistants through eight themes and view them as more formal, fact-based, impersonal and less authentic than humans.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper investigates how users conceptualize humanness during interactions with speech-based intelligent personal assistants by comparing them directly to human conversations. Twenty-one participants engaged in dialogues with a human and two assistants, then applied the repertory grid technique to reflect on and differentiate their experiences. This process revealed that humanness perceptions break down into eight distinct themes covering knowledge, connection, language, performance, interaction style, identity, voice, and behavioral options. The findings highlight systematic differences in how people define assistant capabilities versus human ones. A reader would care because these themes offer a practical structure for deciding when and how much to make interfaces feel human-like in design work.

Core claim

Through analysis of constructs from the repertory grid comparisons, perceptions of humanness are multidimensional, focusing on eight key themes: partner knowledge set, interpersonal connection, linguistic content, partner performance and capabilities, conversational interaction, partner identity and role, vocal qualities and behavioral affordances. Through these themes, it is clear that users define the capabilities of speech interfaces differently to humans, seeing them as more formal, fact based, impersonal and less authentic.

What carries the argument

The repertory grid technique applied to participants' comparisons of dialogues with a human versus two speech-based intelligent personal assistants, which elicits personal constructs later grouped into eight perceptual themes.

If this is right

  • The eight themes provide a scaffold for categorizing and targeting future research and design work on speech interfaces.
  • Design choices can address specific themes such as vocal qualities or conversational interaction to adjust perceived humanness.
  • Emulating humanness should be evaluated for appropriateness by considering whether the interface is positioned as formal and fact-based or interpersonal.
  • Interfaces can lean into strengths like factual accuracy rather than attempting to replicate interpersonal connection or authenticity.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The themes could guide context-specific rules, such as increasing formality in task-oriented assistants while avoiding it in social ones.
  • Testing whether the themes predict measurable outcomes like user trust or continued use would extend the work beyond perception mapping.
  • Similar distinctions might appear in non-speech AI interactions, suggesting the framework could apply to text or visual interfaces.

Load-bearing premise

The repertory grid technique applied to 21 participants yields a stable, generalizable set of themes that accurately reflect how users conceptualize humanness in speech interfaces.

What would settle it

A follow-up study with a larger and demographically varied sample that elicits a substantially different set of themes or fails to replicate the reported differences in formality, fact-focus, impersonality, and authenticity between humans and speech interfaces.

read the original abstract

Humanness is core to speech interface design. Yet little is known about how users conceptualise perceptions of humanness and how people define their interaction with speech interfaces through this. To map these perceptions n=21 participants held dialogues with a human and two speech interface based intelligent personal assistants, and then reflected and compared their experiences using the repertory grid technique. Analysis of the constructs show that perceptions of humanness are multidimensional, focusing on eight key themes: partner knowledge set, interpersonal connection, linguistic content, partner performance and capabilities, conversational interaction, partner identity and role, vocal qualities and behavioral affordances. Through these themes, it is clear that users define the capabilities of speech interfaces differently to humans, seeing them as more formal, fact based, impersonal and less authentic. Based on the findings, we discuss how the themes help to scaffold, categorise and target research and design efforts, considering the appropriateness of emulating humanness.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper claims that perceptions of humanness in speech-based IPAs are multidimensional, based on a repertory grid study with n=21 participants who compared interactions with a human and two IPAs. Thematic analysis yields eight constructs (partner knowledge set, interpersonal connection, linguistic content, partner performance and capabilities, conversational interaction, partner identity and role, vocal qualities and behavioral affordances), and users view IPAs as more formal, fact-based, impersonal, and less authentic than humans. The work positions these themes as a scaffold for design and research on emulating humanness.

Significance. If the themes prove stable, the study offers a structured way to categorize user perceptions that could inform when humanness emulation is appropriate in IPA design. The repertory grid approach is a methodological strength for eliciting participant-generated constructs rather than imposing predefined categories.

major comments (2)
  1. [Methods] Methods section: the thematic grouping of repertory grid constructs into the eight themes reports no inter-rater reliability statistics, saturation criteria, member checking, or hold-out validation. This directly undermines the central claim that the themes constitute a reliable mapping of user conceptualizations.
  2. [Results] Results/Discussion: the assertion that the eight themes are generalizable enough to 'scaffold, categorise and target research and design efforts' rests on a single sample of 21 without cross-validation or replication evidence; the observed differences (e.g., IPAs seen as more formal/fact-based) could reflect sample idiosyncrasies or method artifacts.
minor comments (1)
  1. [Abstract] Abstract: 'Analysis of the constructs show' should specify the exact qualitative procedure used to derive the eight themes.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We are grateful to the referee for their detailed and constructive feedback on our manuscript. We respond to each of the major comments below, indicating where revisions will be made to address the concerns raised.

read point-by-point responses
  1. Referee: [Methods] Methods section: the thematic grouping of repertory grid constructs into the eight themes reports no inter-rater reliability statistics, saturation criteria, member checking, or hold-out validation. This directly undermines the central claim that the themes constitute a reliable mapping of user conceptualizations.

    Authors: We agree that the Methods section would benefit from greater transparency regarding the thematic analysis process. The grouping of constructs into themes was performed iteratively by the authors based on semantic similarity and patterns across the repertory grid data. No formal inter-rater reliability was calculated as the analysis was primarily interpretive and led by one researcher with review by co-authors. We will revise the Methods section to describe the analysis steps in more detail, including how saturation was approached through the number of participants and construct elicitation. We will also adjust the abstract and discussion to avoid overstating the reliability of the themes as a mapping. revision: yes

  2. Referee: [Results] Results/Discussion: the assertion that the eight themes are generalizable enough to 'scaffold, categorise and target research and design efforts' rests on a single sample of 21 without cross-validation or replication evidence; the observed differences (e.g., IPAs seen as more formal/fact-based) could reflect sample idiosyncrasies or method artifacts.

    Authors: The referee is correct that the study is based on a single sample of 21 participants and does not include cross-validation. We position the eight themes as an initial framework derived from this exploratory study to help scaffold future research, not as a fully generalizable model. The differences noted are supported by the data from this sample and consistent with related work. In the revised manuscript, we will emphasize the exploratory character of the study, note the limitations of the sample size, and suggest that future work should validate the themes through replication. This will temper the claims while preserving the contribution of the identified themes. revision: partial

standing simulated objections not resolved
  • The study does not include replication or cross-validation data, which cannot be addressed without new empirical work.

Circularity Check

0 steps flagged

No circularity: qualitative thematic analysis grounded in participant constructs with no derivations or self-referential reductions

full rationale

The paper applies the repertory grid technique to elicit constructs from 21 participants comparing human and IPA interactions, then performs thematic analysis to identify eight themes. No equations, fitted parameters, predictions, or mathematical derivations exist. The central claim (multidimensional humanness perceptions captured by the eight themes) is presented as emerging directly from the elicited constructs rather than reducing to prior self-citations, internal definitions, or renamed inputs. No load-bearing self-citation chains or ansatzes are invoked. The method is self-contained against external benchmarks in the sense that the output themes are explicitly tied to the participant data without circular redefinition.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the assumption that the repertory grid method validly captures and structures subjective perceptions and that the 21-participant sample is sufficient to identify general themes.

axioms (1)
  • domain assumption The repertory grid technique accurately elicits and structures user perceptions of humanness in speech interfaces.
    The entire analysis depends on this qualitative method producing reliable constructs.

pith-pipeline@v0.9.0 · 5698 in / 1314 out tokens · 24263 ms · 2026-05-24T15:18:50.002414+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 58 canonical work pages · 3 internal anchors

  1. [1]

    Kei Akuzawa, Yusuke Iwasawa, and Yutaka Matsuo. 2018. Ex- pressive Speech Synthesis via Modeling Expressions with Variational Autoencoder. In Interspeech 2018 . ISCA, 3067–3071. https://doi.org/10.21437/Interspeech.2018-1113

  2. [2]

    René Amalberti, Noëlle Carbonell, and Pierre Falzon. 1993. Use r rep- resentations of computer systems in human-computer speech interac- tion. International Journal of Man-Machine Studies 38, 4 (April 1993), 547–566. https://doi.org/10.1006/imms.1993.1026

  3. [3]

    Matthew P Aylett, Benjamin R Cowan, and Leigh Clark. 2019. Siri, Echo and Performance: You have to Suffer Darling. In Extended Ab- stracts of the 2019 CHI Conference on Human Factors in Comput ing Systems. ACM, alt08

  4. [4]

    Branigan, Martin J

    Holly P. Branigan, Martin J. Pickering, Jamie Pearson, Janet F. McLean, and Ash Brown. 2011. The role of beliefs in lexical alignment: Ev- idence from dialogs with humans and computers. Cognition 121, 1 (Oct. 2011), 41–57. https://doi.org/10.1016/j.cognition.201 1.05.011

  5. [5]

    Cynthia Breazeal. 2003. Emotion and sociable humanoid robots. In- ternational journal of human-computer studies 59, 1-2 (2003), 119–155

  6. [6]

    Susan E Brennan and Herbert H Clark. 1996. Conceptual pacts and lexical choice in conversation. Journal of Experimental Psychology: Learning, Memory, and Cognition 22, 6 (1996), 1482

  7. [7]

    Allison Bruce, Illah Nourbakhsh, and Reid Simmons. 2002. The r ole of expressiveness and attention in human-robot interaction. In Proceed- ings 2002 IEEE International Conference on Robotics and Aut omation (Cat. No. 02CH37292), Vol. 4. IEEE, 4138–4142

  8. [8]

    Justine Cassell. 2001. Embodied conversational agents: repres entation and intelligence in user interfaces. AI magazine 22, 4 (2001), 67

  9. [9]

    Vincent Cho and Robert Wright. 2010. Exploring the evaluation framework of strategic information systems using repertory grid tech- nique: a cognitive perspective from chief information officers. Be- haviour & Information Technology 29, 5 (2010), 447–457

  10. [10]

    Leigh Clark. 2018. Social Boundaries of Appropriate Speech in HCI: A Politeness Perspective. In Proceedings of the 32nd British Human Computer Interaction Conference. British Computer Societ y. https://doi. org/10.14236/ewic/HCI2018, Vol. 76

  11. [11]

    Leigh Clark, Phillip Doyle, Diego Garaialde, Emer Gilmartin, St ephan Schlögl, Jens Edlund, Matthew Aylett, João Cabral, Cosmin Munteanu, and Benjamin Cowan. 2018. The State of Speech in HCI: Trends, Themes and Challenges. arXiv preprint arXiv:1810.06828 (2018)

  12. [12]

    Leigh Clark, Abdulmalik Ofemile, Svenja Adolphs, and Tom Rod den

  13. [13]

    ACM Transactions on Interactive Intelligent Systems (TiiS) 6, 4 (2016), 29

    A multimodal approach to assessing user experiences with agent helpers. ACM Transactions on Interactive Intelligent Systems (TiiS) 6, 4 (2016), 29

  14. [14]

    Leigh Clark, Nadia Pantidi, Orla Cooney, Philip Doyle, Diego Gara - ialde, Justin Edwards, Brendan Spillane, Christine Murad, Cos- min Munteanu, Vincent Wade, and Benjamin R. Cowan. 2019. What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents. arXiv:1901.06525 [cs] (Jan. 2019). https://doi.org/10.1145/3290605.3300705 arXiv: 19...

  15. [15]

    Benjamin R Cowan and Holly P Branigan. 2015. Does voice anthropo- morphism affect lexical alignment in speech-based human-computer dialogue?. In Sixteenth Annual Conference of the International Speech Communication Association

  16. [16]

    Benjamin R Cowan and Holly P Branigan. 2017. They Know as Much as We Do: Knowledge Estimation and Partner Modelling of Artificial Partners

  17. [17]

    Cowan, Holly P

    Benjamin R. Cowan, Holly P. Branigan, Mateo Obregón, Enas Bugis, and Russell Beale. 2015. Voice anthropomorphism, interlocutor mo d- elling and alignment effects on syntactic choices in human−computer dialogue. International Journal of Human-Computer Studies 83 (Nov. 2015), 27–42. https://doi.org/10.1016/j.ijhcs.2015.05.0 08

  18. [18]

    Benjamin R Cowan, Nadia Pantidi, David Coyle, Kellie Morrissey, Pe - ter Clarke, Sara Al-Shehri, David Earley, and Natasha Bandeira. 2 017. What can i help you with?: infrequent users’ experiences of intelli- gent personal assistants. In Proceedings of the 19th International Con- ference on Human-Computer Interaction with Mobile Devices and Ser- vices. ACM, 43

  19. [19]

    Laurence Devillers, Sophie Rosset, Guillaume Dubuisson Dup lessis, Lucile Bechade, Yucel Yemez, Bekir B Turker, Metin Sezgin, Engin Erzin, Kevin El Haddad, Stephane Dupont, et al. 2018. Multifaceted Engagement in Social Interaction with a Machine: The JOKER Project. In 2018 13th IEEE International Conference on Automatic Face &Gesture Recognition (FG 2018...

  20. [20]

    Penny Dick and Devi Jankowicz. 2001. A social construction- ist account of police culture and its influence on the represen- tation and progression of female officers: A repertory grid anal- ysis in a UK police force. Policing: An International Journal of Police Strategies & Management 24, 2 (June 2001), 181–199. https://doi.org/10.1108/13639510110390936

  21. [21]

    Mateusz Dubiel, Martin Halvey, and Leif Azzopardi. 2018. A Su rvey Investigating Usage of Virtual Personal Assistants. arXiv:1807.04606 [cs] (July 2018). http://arxiv.org/abs/1807.04606 arXiv: 1807. 04606

  22. [22]

    Jens Edlund, Julia Bell Hirschberg, and Mattias Heldner. 2009. Pause and gap length in face-to-face interaction. Columbia University (2009). https://doi.org/10.7916/d82f7wt9

  23. [23]

    Daniel Fallman and John Waterworth. 2010. Capturing user exp eri- ences of mobile information technology with the repertory grid tech- nique. Human Technology: An Interdisciplinary Journal on Humans i n ICT Environments (2010)

  24. [24]

    Hao Fang, Hao Cheng, Elizabeth Clark, Ariel Holtzman, Maarte n Sap, Mari Ostendorf, Yejin Choi, and Noah A Smith. 2017. Sounding board– university of washington’s alexa prize submission. (2017)

  25. [25]

    Fay Fransella, Richard Bell, and D Bannister. 2004. A manual for reper- tory grid technique . Wiley, Hoboken, NJ. OCLC: 53978922

  26. [26]

    Moojan Ghafurian, Neil Budnarain, and Jesse Hoey. 2019. Role of Emo- tions in Perception of Humanness of Virtual Agents. In Proceedings of the 18th International Conference on Autonomous Agents and MultiA- gent Systems. International Foundation for Autonomous Agents and Multiagent Systems, 1979–1981

  27. [27]

    Cowan, and Nick Campbell

    Emer Gilmartin, Marine Collery, Ketong Su, Yuyun Huang, Christy Elias, Benjamin R. Cowan, and Nick Campbell. 2017. Social talk: mak- ing conversation with people and machine. In Proceedings of the 1st ACM SIGCHI International Workshop on Investigating Social Interac- tions with Artificial Agents - ISIAA 2017. ACM Press, Glasgow, UK, 31–

  28. [28]

    https://doi.org/10.1145/3139491.3139494

  29. [29]

    Nick Haslam. 2006. Dehumanization: An integrative review. Person- ality and social psychology review 10, 3 (2006), 252–264

  30. [30]

    Nick Haslam, Paul Bain, Lauren Douge, Max Lee, and Brock Bast ian

  31. [31]

    Journal of personality and social psychology 89, 6 (2005), 937

    More human than you: Attributing humanness to self and oth- ers. Journal of personality and social psychology 89, 6 (2005), 937

  32. [32]

    Chin-Chang Ho and Karl F MacDorman. 2017. Measuring the un- canny valley effect. International Journal of Social Robotics 9, 1 (2017), 129–139

  33. [33]

    Trevor Hogan and Eva Hornecker. 2013. Blending the repertory grid technique with focus groups to reveal rich design relevant insight. In Proceedings of the 6th International Conference on Designi ng Pleasur- able Products and Interfaces - DPPI ’13 . ACM Press, Newcastle upon Tyne, United Kingdom, 116. https://doi.org/10.1145/2513506.25 13519

  34. [34]

    Devi Jankowicz. 2004. The easy guide to repertory grids . Wiley, Chich- ester, West Sussex, England ; Hoboken, N.J

  35. [35]

    George Kelly. 1991. The psychology of personal constructs. Vol. 2, Vol. 2,. Routledge in association with the Centre for Personal Construct Psychology, London; New York. OCLC: 973052807

  36. [36]

    Alan Kennedy, A Wilkes, L Elder, and Wayne Murray

  37. [37]

    Cognition 30 (1988), 37–72

    Dialogue with machines. Cognition 30 (1988), 37–72. https://doi.org/10.1016/0010-0277(88)90003-0

  38. [38]

    Yoshinori Kuno, Kazuhisa Sadazuka, Michie Kawashima, Keiichi Ya - mazaki, Akiko Yamazaki, and Hideaki Kuzuoka. 2007. Museum guide robot based on sociological interaction analysis. In Proceedings of the SIGCHI conference on Human factors in computing systems . ACM, 1191–1194

  39. [39]

    Ludovic Le Bigot, Jean-François Rouet, and Eric Jamet. 2007. Ef - fects of Speech- and Text-Based Interaction Modes in Natural Lan- guage Human-Computer Dialogue. Human Factors: The Journal of the Human Factors and Ergonomics Society 49, 6 (Dec. 2007), 1045–1053. https://doi.org/10.1518/001872007X249901

  40. [40]

    Lucian Leahu, Marisa Cohn, and Wendy March. 2013. How categor ies come to matter. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’13. ACM Press, Paris, France, 3331. https://doi.org/10.1145/2470654.2466455

  41. [41]

    Kwan-Min Lee and Clifford Nass. 2005. Social-Psychological Or i- gins of Feelings of Presence: Creating Social Presence With Machine- Generated Voices. Media Psychology 7, 1 (Feb. 2005), 31–45. https://doi.org/10.1207/S1532785XMEP0701_2

  42. [42]

    Ewa Luger and Abigail Sellen. 2016. Like having a really bad PA: the gulf between user expectation and experience of conversational agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 5286–5297

  43. [43]

    Meddeb and Patricia Frenz-Belkin

    Elizabeth J. Meddeb and Patricia Frenz-Belkin. 2010. What? I Didn’t Say THAT!: Linguistic strategies when speaking to write. Journal of Pragmatics 42, 9 (Sept. 2010), 2415–2429. https://doi.org/10.1016/j.pragma.2009.12.022

  44. [44]

    Moore, Hui Li, and Shih-Hao Liao

    Roger K. Moore, Hui Li, and Shih-Hao Liao. 2016. Progress and Prospects for Spoken Language Technology: What Ordinary People Think. 3007–3011. https://doi.org/10.21437/Interspeech.20 16-874

  45. [45]

    Masahiro Mori, Karl F MacDorman, and Norri Kageki. 1970. 2012 . The Uncanny Valley [From the Field]. Robotics & Automation Magazine, IEEE 19, 2 (1970), 2012

  46. [46]

    Sharon Oviatt, Jon Bernard, and Gina-Anne Levow. 1998. Lin- guistic Adaptations During Spoken and Multimodal Error Res- olution. Language and Speech 41, 3-4 (July 1998), 419–442. https://doi.org/10.1177/002383099804100409

  47. [47]

    Ioannis Papaioannou, Amanda Cercas Curry, Jose L Part, Igor Sha - lyminov, Xinnuo Xu, Yanchao Yu, Ondrej Dušek, Verena Rieser, and Oliver Lemon. 2017. Alana: Social dialogue using an ensemble model and a ranker trained on user feedback. Alexa Prize Proceedings (2017)

  48. [48]

    Martin Porcheron, Joel E Fischer, Stuart Reeves, and Sarah S harples

  49. [49]

    In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

    Voice Interfaces in Everyday Life. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems . ACM, 640

  50. [50]

    Alexa is my new BFF

    Amanda Purington, Jessie G. Taft, Shruti Sannon, Natalya N. Bazarova, and Samuel Hardman Taylor. 2017. "Alexa is my new BFF": Social Roles, User Satisfaction, and Personification of the Amazon Echo. In Proceedings of the 2017 CHI Confer- ence Extended Abstracts on Human Factors in Computing Syste ms - CHI EA ’17 . ACM Press, Denver, Colorado, USA, 2853–285...

  51. [51]

    Valentin Schwind, Katrin Wolf, and Niels Henze. 2018. Avoiding the uncanny valley in virtual character design. interactions 25, 5 (2018), 45–49

  52. [52]

    Shaw and Laurie F

    Mildred L.G. Shaw and Laurie F. Thomas. 1978. FOCUS on education—an interactive computer system for the develop- ment and analysis of repertory grids. International Jour- nal of Man-Machine Studies 10, 2 (March 1978), 139–173. https://doi.org/10.1016/S0020-7373(78)80009-1

  53. [53]

    Brendan Spillane, Emer Gilmartin, Christian Saam, Ketong Su, Ben- jamin R Cowan, Séamus Lawless, and Vincent Wade. 2017. Introduc- ing ADELE: a personalized intelligent companion. In Proceedings of the 1st ACM SIGCHI International Workshop on Investigating Social In- teractions with Artificial Agents . ACM, 43–44

  54. [54]

    James C Thompson, J Gregory Trafton, and Patrick McKnight. 201 1. The perception of humanness from the movements of synthetic agents. Perception 40, 6 (2011), 695–704

  55. [55]

    Lai Lai Tung, Yun Xu, and Felix B. Tan. 2009. Attributes of Web S ite Usability: A Study of Web Users with the Repertory Grid Technique. International Journal of Electronic Commerce 13, 4 (July 2009), 97–126. https://doi.org/10.2753/JEC1086-4415130405

  56. [56]

    Mirjam Wester, Matthew Aylett, Marcus Tomalin, and Rasmus D all

  57. [57]

    Interspeech 2015 (2015), 5

    Artificial Personality and Disfluency. Interspeech 2015 (2015), 5

  58. [58]

    Mapping Perceptions of Humanness in Speech-Based Intelligent Personal Assistant Interaction

    Jakub Aleksander Zlotowski, Hidenobu Sumioka, Shuichi Nishio, Dy- lan F Glas, Christoph Bartneck, and Hiroshi Ishiguro. 2015. Persis - tence of the uncanny valley: the influence of repeated interactions and a robot’s attitude on its perception. Frontiers in psychology 6 (2015), 883. This figure "sample-franklin.png" is available in "png" format from: http:...