Pith. sign in

REVIEW 4 major objections 5 minor 67 references

Talking Surveys: How Photorealistic Embodied Conversational Agents Shape Response Quality, Engagement, and Satisfaction

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Talking to a photorealistic avatar yields longer, more informative survey answers than typing to a chatbot, in less speaking time.

desk verdict A solid, transparent proof-of-concept showing people talk more and faster to a speech avatar than a text chatbot, but the informativeness claim is confounded with speaking-versus-typing and the abstract oversells it. read the letter →

arxiv 2508.02376 v2 pith:VZMPDWXR submitted 2025-08-04 cs.HC

Matus Krajcovic , Peter Demcak , Eduard Kuric This is my paper · ORCID
classification cs.HC
keywords conversationalagentAI-mediatedcommunicationphotorealisticavataruserengagementandsatisfactionvirtualhumanchatbotsurveyresponsequalityopen-endedquestions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that putting a photorealistic, voice-enabled avatar into an online survey changes what people say: participants who talked to the avatar gave roughly twice as many words and twice as much information per open-ended answer as participants who typed to a text chatbot, while spending significantly less time speaking their answers. The authors built a working instrument, the Virtual Agent Interviewer, that combines AI-driven video generation, speech recognition, and a large language model, and tested it on 80 UK panel participants completing two psychometric surveys. They frame the finding as a step toward closing the gap between unmoderated online surveys and human-moderated interviews, where personal engagement and perceived accountability are higher. They also report that satisfaction ratings did not differ between the two agents, with qualitative feedback pointing to turn-taking delays and Uncanny Valley reactions as the main sources of friction.

What carries the argument

The central object is the Virtual Agent Interviewer (VAI), a prototype survey tool that embeds a conversational agent into the survey flow. In the embodied condition the agent is a photorealistic animated video avatar that speaks; in the text condition the same conversation logic runs as a typed chatbot, so the design isolates the addition of embodiment plus speech against the chatbot baseline. The response-quality machinery is a set of measures adapted from conversational-survey research: informativeness as summed word surprisal, that is, the inverse frequency of the response's words, from the wordfreq database, plus human- and LLM-coded specificity, relevance, clarity, self-disclosure, and sentiment. The argument turns on comparing these measures across the two conditions together with temporal engagement metrics such as responding time and transition time.

What would settle it

Have the same participants answer the same open-ended questions under three conditions—typing to a chatbot, speaking to a voice-only agent, and speaking to the animated avatar—and compare word-surprisal informativeness and word counts; if voice-only speech matches the avatar, the central effect is driven by speech rather than embodiment, and the paper's claim would need to be narrowed.

Watch

Extended reading notes

Core claim

The paper's central claim is that embodied conversational agents can improve the quality and engagement of survey responses without lowering satisfaction. In a between-subjects experiment that held conversation logic constant, participants who spoke to an animated video avatar produced more informative responses ($M=285.41$ vs $M=142.23$, $U(80)=1164$, $z=3.50$, $p<.001$, $r=.39$), used more words ($M=29.32$ vs $M=14.3$, $U(80)=1171$, $z=3.57$, $p<.001$, $r=.40$), and spent less time speaking (responding time $U(80)=531$, $z=2.59$, $p=.010$, $r=.29$) than participants who typed to a text-based chatbot. The authors interpret the steeper time-informativeness slope for the avatar as evidence that speaking to a humanlike agent is a more efficient communication channel, not merely a longer one. They note that spoken answers scored lower on clarity ($r=.63$) and relevance ($r=.26$), which they attribute to the spontaneous, unedited character of speech rather than to worse content. Satisfaction differences were not statistically significant, and the paper presents this as a favorable baseline for embodied agents given the prototype's technical limitations.

Load-bearing premise

The load-bearing premise is that word-surprisal—a count of how rare a word is in English—measures informativeness equally for speech and for typing; if people naturally use rarer words when speaking, the avatar's informativeness advantage could be an artifact of comparing two different communication channels.

Editorial extensions

If this is right

  • Unmoderated survey tools could collect more detailed qualitative data from open-ended questions without asking participants to spend more time producing each answer.
  • Because the time-informativeness slope was steeper for the avatar, speaking to an embodied agent appears to transmit more information per second than typing to a chatbot.
  • Satisfaction being statistically unchanged suggests that adding a photorealistic face to a survey agent does not carry an inherent satisfaction penalty, though preference data show about half of embodied-condition participants would still switch to text.
  • The larger and significant effects in the personality survey compared with the marginal effects in the cognitive-failures survey imply that the benefit of embodied agents may depend on question breadth and sensitivity.
  • Lower clarity and relevance in spoken responses mean embodied-agent transcripts may require more preprocessing or a visible question prompt to keep answers on target.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A sharper test of embodiment itself would compare the avatar against a voice-only agent with no face; the reported design cannot separate the effect of speech modality from the effect of a humanlike body and face.
  • The word-surprisal measure may overstate the informativeness advantage if natural speech systematically draws on rarer vocabulary for the same underlying content; re-analyzing with frequency-matched or lemmatized transcripts would check this.
  • If the pattern replicates, a practical design rule follows: reserve embodied agents for broad, identity-relevant open-ended questions and expect smaller gains for narrowly factual or more sensitive items.
  • A direct extension would test whether telling participants that a human moderator is watching the avatar changes response length, separating perceived accountability from the visual design itself.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents the Virtual Agent Interviewer (VAI), a survey instrument that embeds a photorealistic embodied conversational agent (ECA) with voice interaction, and compares it against a text-based chatbot in a between-subjects experiment with 80 UK participants across two psychometric surveys (BFI-2-S and CFQ). The authors report that the embodied agent produces significantly higher word counts and informativeness, shorter participant responding time, and no difference in satisfaction, and they interpret these results as evidence that ECAs can improve response quality and engagement. The paper includes a mixed-methods analysis with qualitative feedback, and it makes the instrument, code, and data publicly available. The central claim, however, is that embodiment itself drives the improvement, whereas the experimental design bundles voice input, a video avatar, and turn-taking into one condition that is compared to a typing-only chatbot.

Significance. If the causal interpretation were supported, this would be a useful proof-of-concept for bringing ECA-mediated moderation to online surveys, with practical implications for reducing satisficing and improving open-ended response quality. The paper is transparent about many null results and limitations, and the open repository and the detailed description of the instrument are commendable. However, the study's headline inference is not currently supported because the 'informativeness' measure is confounded with response length and with the speech-versus-text channel, and the effect is not robust across the two surveys. These issues reduce the significance of the current findings, although the methodological contribution may still be useful to the community.

major comments (4)
  1. [§4.6, §5.1, §5.2] The 'informativeness' measure is defined as the sum of per-word surprisal from wordfreq (§4.6). This sum scales with word count, and §5.2 reports that embodied-agent responses contain roughly twice as many words (M=29.32 vs 14.3, r=.40). Consequently, the reported informativeness advantage (r=.39) could simply reflect that participants produce more words when speaking than when typing; it does not establish that the same underlying content is more informative. The paper's own results show that the two channels differ: embodied responses were less clear (r=.63) and slightly less relevant (r=.26), consistent with a speaking-versus-typing channel effect. The authors should re-analyze the data using an information-density measure (e.g., per-word surprisal, or surprisal per 100 characters), or control for word count as a covariate, before claiming an informativeness advantage. A concrete test would be to compare the embodied condition with a voice-only condition that does not have a visible avatar.
  2. [§5.4] The headline effects on informativeness and word count are not robust across the two surveys: within CFQ, the comparisons are non-significant (informativeness p=.081, word count p=.067), while only BFI-2-S shows significant effects (r=.47). This pattern suggests that the overall significant result may be driven by only one of the two questionnaires, and it limits the abstract's generalization that embodied agents 'contribute significantly to more informative, detailed responses' across surveys. The authors should report a formal survey-by-condition interaction test, and should temper the abstract and conclusions accordingly.
  3. [§3.1, §4.4] The embodiment manipulation is compounded with response modality: the embodied condition uses voice input plus a video avatar, while the baseline uses typed text input. Thus even the robust word-count difference can be explained by 'people speak more than they type' rather than by the embodied conversational agent per se. The paper partly acknowledges this in the clarity discussion, but it does not treat the modality confound as a threat to the central 'embodiment improves quality' claim. A comparison with an additional condition (e.g., a voice-only non-visual agent, or a text-input condition with an avatar) would be necessary to isolate the contribution of embodiment.
  4. [§5.1, §5.2, §6.2] The authors report a large number of statistical tests without any correction for multiple comparisons, and the stated RQ framework includes four research questions with several measures each. While the strongest effects (p<.001) would likely survive conservative correction, the overall familywise error rate is not addressed. Reporting corrected p-values or a pre-specified analysis plan would strengthen the reliability of the secondary findings such as self-disclosure and the survey-specific results.
minor comments (5)
  1. [Abstract] The sentence 'using on AI-driven video generation' contains a grammatical error; it should be 'using AI-driven video generation.'
  2. [§5.2] The sentence 'Text-based variants' answers tend to be longer' under Figure 9 contradicts the surrounding text, which states that the embodied agent produced significantly more words; this should be corrected to 'Embodied variants' answers tend to be longer.'
  3. [§2.2] The word 'Colloquilally' is a typo for 'Colloquially.'
  4. [§4.6] The data-labeling procedure uses GPT-4.1 with majority voting, and the authors report high correlations with manual labels; however, they do not state whether the manual reviewers were blind to the experimental condition, which would be relevant given the large clarity differences between conditions.
  5. [§5.3] In Table 1 and the surrounding text, there are occasional spacing anomalies (e.g., 'T able' at the top of Table 1); these should be cleaned up for publication.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the response-quality measures come from external sources and the central comparison is an empirical contrast, not a fitted or self-referential prediction.

full rationale

The paper's central claim is an empirical between-subjects contrast (embodied agent vs. text chatbot); no parameter is fitted to the outcome and then renamed a prediction. Informativeness is imported from Xiao et al. (2020) and operationalized with the external wordfreq surprisal values (Section 4.6), not derived from the experimental result. The LLM-based coding of specificity, relevance, clarity, self-disclosure and sentiment uses an external model with reported human-correlation checks; it is not trained on the experimental contrast. The few self-citations (Kuric et al., 2024, 2025a, 2025b; Krajcovic et al., 2025) appear in background, motivation and future-work contexts, and none supplies the load-bearing statistical claim. The concern that summed surprisal partly tracks word count across speaking versus typing is a construct-validity or channel-confound issue, not circularity: the measure is not defined in terms of the experimental outcome, and the paper does not fit the measure to the data.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central result depends on a handful of hand-set design decisions (which items get follow-ups, how long conversations run) and on domain assumptions about the validity of the response-quality measures. No parameters were fitted to data, so the free-parameter count is modest.

free parameters (3)
  • Follow-up question selection = 10 of 30 BFI-2-S items; 9 of 25 CFQ items
    The authors hand-selected which items received a conversational follow-up to balance scales, meaning the measured effects apply only to these items and may not generalize to all survey questions.
  • Conversation length caps = 3 back-and-forths (Inquiry), 10 (Discussion)
    The Wrapping module threshold determines how much conversational data is collected per item; a different cap would change response volumes and the engagement metrics.
  • LLM labeling temperature = 0.2
    The GPT-4.1 labeling prompt was run at temperature 0.2 to reduce variability; this is a hand-set analysis parameter that affects the coded measures.
assumptions (4)
  • domain assumption Gricean maxims-based measures (informativeness, specificity, relevance, clarity) capture response quality
    Section 4.6 adopts measures from Xiao et al. (2020) without revalidation; they are assumed to be valid for both spoken and typed responses.
  • domain assumption wordfreq word-frequency data provides a modality-invariant surprisal baseline
    Section 4.6 uses wordfreq frequencies to compute informativeness; the assumption that speech and text can be compared on this scale is load-bearing for the main positive result.
  • domain assumption GPT-4.1 majority-vote labels with 3.6% manual correction are accurate
    Section 4.6 reports high correlations with reviewed labels but no inter-rater reliability for the human review process; the coded measures (specificity, relevance, clarity, self-disclosure, sentiment) depend on this.
  • domain assumption The two questionnaires differ mainly in question breadth and sensitivity, explaining the RQ4 differences
    Section 5.4 and 6.1 interpret the BFI vs CFQ differences as due to question characteristics; low power per subgroup (n=20) is an alternative explanation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Talking Surveys: How Photorealistic Embodied Conversational Agents Shape Response Quality, Engagement, and Satisfaction." pith.science (2026). https://pith.science/paper/VZMPDWXR

@misc{pith2026250802376,
  author       = {Pith},
  title        = {Pith review of: Talking Surveys: How Photorealistic Embodied Conversational Agents Shape Response Quality, Engagement, and Satisfaction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VZMPDWXR}},
  note         = {Machine review of arXiv:2508.02376}
}
read the original abstract

Embodied conversational agents (ECAs) are increasingly more realistic and capable of dynamic conversations. In online surveys, anthropomorphic agents could help address issues like careless responding and satisficing, which originate from the lack of personal engagement and perceived accountability. However, there is a lack of understanding of how ECAs in user experience research may affect participant engagement, satisfaction, and the quality of responses. As a proof of concept, we propose an instrument that enables the incorporation of conversations with a virtual avatar into surveys, using on AI-driven video generation, speech recognition, and Large Language Models. In our between-subjects study, 80 participants (UK, stratified random sample of general population) either talked to a voice-based agent with an animated video avatar, or interacted with a chatbot. Across surveys based on two self-reported psychometric tests, 2,265 conversation responses were obtained. Statistical comparison of results indicates that embodied agents can contribute significantly to more informative, detailed responses, as well as higher yet more time-efficient engagement. Furthermore, qualitative analysis provides valuable insights for causes of no significant change to satisfaction, linked to personal preferences, turn-taking delays and Uncanny Valley reactions. These findings support the pursuit and development of new methods toward human-like agents for the transformation of online surveys into more natural interactions resembling in-person interviews.

Figures

Figures reproduced from arXiv: 2508.02376 by the authors.

Figure 1
Figure 1. Overview of the VAI survey flow template. Before any further steps, participants are first asked for their informed consent to collect data for research purposes. This is followed by a static introduction message where participants can be informed about what they can expect and what will be required of them. Because interaction with the ECA relies on the participant 4 Heygen Interactive Avatar: https://www.heygen.co… view at source ↗
Figure 2
Figure 2. Avatar appearances that participants can choose between at the beginning of VAI survey ECA variant. Model names as adopted from Heygen are (a) Ann, (b) Shawn, (c) June, (d) Silas, (e) Judy, and (f) Wayne. Before the main body of the survey that entails interaction with the conversational agent, participants can be presented with a demographic questionnaire. This questionnaire is intended primarily to obtain descript… view at source ↗
Figure 3
Figure 3. Survey instructions with the embodied conversation agent displayed in the bottom right corner after it has been minimized (it starts maximized to the full browser window). In text-based variant, the agent is displayed analogously to [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Survey questions with the text-based conversational agent follow-up popped out on the right side after an answer was submitted. In the embodied variant, the agent is displayed analogously to [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Participant demographics and computer usage—covering gender, age, education level, and frequency of computer use—were balanced across all study variants. 4.3 Materials Even small differences between surveys—such as the subject of the questions, the complexity of unders…
Figure 6
Figure 6. Figure 6: Box plots of informativeness, indicating that participants provided more information to the embodied agent and with a higher degree of variability. Specificity. There was no significant difference between the embodied agent (M = 1.32, SD = 0.36) and text-based agent (M…
Figure 7
Figure 7. Figure 7: Box plot illustrating the distribution of thinking, answering, and total time for text-based and embodied assistants. Durations tend to be longer for the text-based variants. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Scatter plot with linear regression illustrating the relationship between Responding time and Informativeness, grouped by conversational agent. Due to noise outliers, IQR is plotted for better visibility of the relationship. Response length. On average, participants us…
Figure 9
Figure 9. Figure 9: Box plot illustrating the distribution of word counts (a) and character counts (b) for text-based and embodied assistants. Text-based variants’ answers tend to be longer [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Median word count per follow-up question response, grouped by conversation (follow-up to individual questions). Diagrams correspond to surveys (a) BFI-2-S and (b) CFQ. their analysis. 5.3 Satisfaction (RQ3) RQ3: Does facilitation of surveys with an ECA affect user sat…
Figure 11
Figure 11. Figure 11: LLM module architecture diagram of VAI conversational agents (text-based and embodied). Displays module interaction and flow of data during user interaction, from user input to system output. C Data labeling prompt Act in the role of a user researcher analyzing conver…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 48 canonical work pages

  1. [1]

    Applying stepped task in remote unmoderated test: A case report

    Shushu He. Applying stepped task in remote unmoderated test: A case report. In Constantine Stephanidis, Margherita Antona, and Stavroula Ntoa, editors, HCI International 2021 - Posters, pages 33--40, Cham, 2021. Springer International Publishing. ISBN 978-3-030-78635-9

    work page 2021

  2. [2]

    Prototypes, platforms and protocols: Identifying common issues with remote, unmoderated studies and their impact on research participants

    Steven Schirra, Sasha G Volkov, Shraddhaa Narasimha, and Frank Bentley. Prototypes, platforms and protocols: Identifying common issues with remote, unmoderated studies and their impact on research participants. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, CHI EA '23, pages 1--6, New York, NY, USA, 2023. Associati...

    arXiv 2023

  3. [3]

    Ward and Adam W

    M.K. Ward and Adam W. Meade. Applying social psychology to prevent careless responding during online surveys. Applied Psychology, 67 0 (2): 0 231--263, 2018. doi:https://doi.org/10.1111/apps.12118. URL https://iaap-journals.onlinelibrary.wiley.com/doi/abs/10.1111/apps.12118

    work page doi:10.1111/apps.12118 2018

  4. [4]

    Comparing data from chatbot and web surveys: Effects of platform and conversational style on survey response quality

    Soomin Kim, Joonhwan Lee, and Gahgene Gweon. Comparing data from chatbot and web surveys: Effects of platform and conversational style on survey response quality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI '19, page 1–12, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450359702. doi:10.1145...

    arXiv 2019

  5. [5]

    Comparison of moderated and unmoderated remote usability sessions for web-based simulation software: A randomized controlled trial

    Pedram Khayyatkhoshnevis, Savanah Tillberg, Eric Latimer, Tim Aubry, Andrew Fisher, and Vijay Mago. Comparison of moderated and unmoderated remote usability sessions for web-based simulation software: A randomized controlled trial. In Masaaki Kurosu, editor, Human-Computer Interaction. Theoretical Approaches and Design Methods, pages 232--251, Cham, 2022....

    work page 2022

  6. [6]

    Embodied conversational agents in clinical psychology: a scoping review

    Simon Provoost, Ho Ming Lau, Jeroen Ruwaard, and Heleen Riper. Embodied conversational agents in clinical psychology: a scoping review. Journal of medical Internet research, 19 0 (5): 0 e151, 2017. URL https://doi.org/10.2196/jmir.6553

    doi:10.2196/jmir.6553 2017

  7. [7]

    Ai-mediated communication: Definition, research agenda, and ethical considerations

    Jeffrey T Hancock, Mor Naaman, and Karen Levy. Ai-mediated communication: Definition, research agenda, and ethical considerations. Journal of Computer-Mediated Communication, 25 0 (1): 0 89--100, 01 2020. ISSN 1083-6101. doi:10.1093/jcmc/zmz022. URL https://doi.org/10.1093/jcmc/zmz022

    doi:10.1093/jcmc/zmz022 2020

  8. [8]

    Artificial social influence via human-embodied ai agent interaction in immersive virtual reality (vr): Effects of similarity-matching during health conversations

    Sue Lim, Ralf Schmälzle, and Gary Bente. Artificial social influence via human-embodied ai agent interaction in immersive virtual reality (vr): Effects of similarity-matching during health conversations. Computers in Human Behavior: Artificial Humans, 5: 0 100172, 2025. ISSN 2949-8821. doi:10.1016/j.chbah.2025.100172. URL https://doi.org/10.1016/j.chbah.2...

    arXiv 2025

Show all 67 references
  1. [9]

    Embodied conversational agents in extended reality: A systematic review

    Fu-Chia Yang, Pedro Acevedo, Siqi Guo, Minsoo Choi, and Christos Mousas. Embodied conversational agents in extended reality: A systematic review. IEEE Access, 13: 0 79805--79824, 2025. doi:10.1109/ACCESS.2025.3566698

    2025

  2. [10]

    Image-to-video generation via 3d facial dynamics

    Xiaoguang Tu, Yingtian Zou, Jian Zhao, Wenjie Ai, Jian Dong, Yuan Yao, Zhikang Wang, Guodong Guo, Zhifeng Li, Wei Liu, and Jiashi Feng. Image-to-video generation via 3d facial dynamics. IEEE Transactions on Circuits and Systems for Video Technology, 32 0 (4): 0 1805--1819, 202...

    2022

  3. [11]

    Effects of a computer's human likeness on disclosure in psychosocial assessments: A randomised trial

    Tingting Zhu and Elizabeth Broadbent. Effects of a computer's human likeness on disclosure in psychosocial assessments: A randomised trial. Computers in Human Behavior, 169: 0 108683, 2025. ISSN 0747-5632. doi:https://doi.org/10.1016/j.chb.2025.108683. URL https://www.scienced...

    2025

  4. [12]

    Ziang Xiao, Michelle X. Zhou, Q. Vera Liao, Gloria Mark, Changyan Chi, Wenxi Chen, and Huahai Yang. Tell me about yourself: Using an ai-powered chatbot to conduct conversational surveys with open-ended questions. In ACM Transactions on Computer-Human Interaction (TOCHI), volum...

    2020 doi

  5. [13]

    Democratizing eye-tracking? appearance-based gaze estimation with improved attention branch

    Eduard Kuric, Peter Demcak, Jozef Majzel, and Giang Nguyen. Democratizing eye-tracking? appearance-based gaze estimation with improved attention branch. Engineering Applications of Artificial Intelligence, 149: 0 110494, 2025 a . ISSN 0952-1976. doi:10.1016/j.engappai.2025.110494

    2025

  6. [14]

    User Experience and User Experience Design, chapter 2, pages 11--28

    Monica Bordegoni, Marina Carulli, and Elena Spadoni. User Experience and User Experience Design, chapter 2, pages 11--28. Springer Nature Switzerland, Cham, 2023. ISBN 978-3-031-39683-0. doi:10.1007/978-3-031-39683-0\_2. URL https://doi.org/10.1007/978-3-031-39683-0\_2

    2023 doi

  7. [15]

    De Bleecker and R

    I. De Bleecker and R. Okoroji. Remote Usability Testing: Actionable insights in user behavior across geographies and time zones. Packt Publishing, 2018. ISBN 9781788990639. URL https://books.google.sk/books?id=YbZqDwAAQBAJ

    2018

  8. [16]

    What do usability test moderators say? 'mm hm', 'uh-huh', and beyond

    Morten Hertzum and Kristina Bonde Kristoffersen. What do usability test moderators say? 'mm hm', 'uh-huh', and beyond. In Proceedings of the 10th Nordic Conference on Human-Computer Interaction, NordiCHI '18, page 364–375, New York, NY, USA, 2018. Association for Computing Mac...

    2018

  9. [17]

    Krista D. Kleban. Adapting ux research for people living with alzheimer's, dementia, and mci: Improving the ux research process for participants. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS '24, pages 1--6, New York,...

    2024

  10. [18]

    Bias, Matthew Lease, and Rebecca Kuipers

    Di Liu, Randolph G. Bias, Matthew Lease, and Rebecca Kuipers. Crowdsourcing for usability testing. Proceedings of the American Society for Information Science and Technology, 49 0 (1): 0 1--10, 2012. doi:https://doi.org/10.1002/meet.14504901100. URL https://asistdl.onlinelibra...

    2012 doi

  11. [19]

    Chi, and Bongwon Suh

    Aniket Kittur, Ed H. Chi, and Bongwon Suh. Crowdsourcing user studies with mechanical turk. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '08, page 453–456, New York, NY, USA, 2008. Association for Computing Machinery. ISBN 9781605580111. d...

    2008

  12. [20]

    Web-based versus traditional paper questionnaires: A mixed-mode survey with a nordic perspective

    Lena Hohw \"u , Heidi Lyshol, Mika Gissler, Stefan Hrafn Jonsson, Max Petzold, and Carsten Obel. Web-based versus traditional paper questionnaires: A mixed-mode survey with a nordic perspective. J Med Internet Res, 15 0 (8): 0 e173, Aug 2013. ISSN 14388871. doi:10.2196/jmir.25...

    2013 doi

  13. [21]

    Do print and web surveys provide the same results? Computers in Human Behavior, 22 0 (3): 0 334--350, 2006

    Hsiu-Mei Huang. Do print and web surveys provide the same results? Computers in Human Behavior, 22 0 (3): 0 334--350, 2006. ISSN 0747-5632. doi:https://doi.org/10.1016/j.chb.2004.09.012. URL https://www.sciencedirect.com/science/article/pii/S0747563204001232

    2006 doi

  14. [22]

    Kristoffersen

    Morten Hertzum, Pia Borlund, and Kristina B. Kristoffersen. What do thinking-aloud participants say? a comparison of moderated and unmoderated usability sessions. International Journal of Human–Computer Interaction, 31 0 (9): 0 557--570, 2015. doi:10.1080/10447318.2015.1065691...

    2015

  15. [23]

    What you get is what you see: revisiting the evaluator effect in usability tests

    Morten Hertzum, Rolf Molich, and Niels Ebbe Jacobsen. What you get is what you see: revisiting the evaluator effect in usability tests. Behaviour & Information Technology, 33 0 (2): 0 144--162, 2014. doi:10.1080/0144929X.2013.783114. URL https://doi.org/10.1080/0144929X.2013.783114

    2014

  16. [24]

    Ward, Adam W

    M.K. Ward, Adam W. Meade, Christopher M. Allred, Gabriel Pappalardo, and J. William Stoughton. Careless response and attrition as sources of bias in online survey assessments of personality traits and performance. Computers in Human Behavior, 76: 0 417--430, 2017. ISSN 0747-56...

    2017 doi

  17. [25]

    Introduction to Chatbots, chapter 1, pages 1--11

    Rashid Khan and Anik Das. Introduction to Chatbots, chapter 1, pages 1--11. Apress, Berkeley, CA, 2018. ISBN 978-1-4842-3111-1. doi:10.1007/978-1-4842-3111-1\_1. URL https://doi.org/10.1007/978-1-4842-3111-1\_1

    2018 doi

  18. [26]

    Castro, Oscar Dieste, and Silvia T

    Ranci Ren, Mireya Zapata, John W. Castro, Oscar Dieste, and Silvia T. Acuña. Experimentation for chatbot usability evaluation: A secondary study. IEEE Access, 10: 0 12430--12464, 2022. doi:10.1109/ACCESS.2022.3145323

    2022

  19. [27]

    Clarifying the dialogue-level performance of gpt-3.5 and gpt-4 in task-oriented and non-task-oriented dialogue systems

    Shinya Iizuka, Shota Mochizuki, Atsumoto Ohashi, Sanae Yamashita, Ao Guo, and Ryuichiro Higashinaka. Clarifying the dialogue-level performance of gpt-3.5 and gpt-4 in task-oriented and non-task-oriented dialogue systems. Proceedings of the AAAI Symposium Series, 2 0 (1): 0 182...

    2024 doi

  20. [28]

    Emotionally intelligent chatbots: A systematic literature review

    Ghazala Bilquise, Samar Ibrahim, and Khaled Shaalan. Emotionally intelligent chatbots: A systematic literature review. Human Behavior and Emerging Technologies, 2022 0 (1): 0 9601630, 2022. doi:https://doi.org/10.1155/2022/9601630. URL https://onlinelibrary.wiley.com/doi/abs/1...

    2022 doi

  21. [29]

    Ramalingam, G

    Gokul Yenduri, M. Ramalingam, G. Chemmalar Selvi, Y. Supriya, Gautam Srivastava, Praveen Kumar Reddy Maddikunta, G. Deepti Raj, Rutvij H. Jhaveri, B. Prabadevi, Weizheng Wang, Athanasios V. Vasilakos, and Thippa Reddy Gadekallu. Gpt (generative pre-trained transformer)— a comp...

    2024

  22. [30]

    Submitting surveys via a conversational interface: An evaluation of user acceptance and approach effectiveness

    Irene Celino and Gloria Re Calegari . Submitting surveys via a conversational interface: An evaluation of user acceptance and approach effectiveness. International Journal of Human-Computer Studies, 139: 0 102410, 2020. ISSN 1071-5819. doi:https://doi.org/10.1016/j.ijhcs.2020....

    2020

  23. [31]

    Conversation-based hybrid ui for the repertory grid technique: A lab experiment into automation of qualitative surveys

    Yunxing Liu and Jean-Bernard Martens. Conversation-based hybrid ui for the repertory grid technique: A lab experiment into automation of qualitative surveys. International Journal of Human-Computer Studies, 184: 0 103227, 2024. ISSN 1071-5819. doi:https://doi.org/10.1016/j.ijh...

    2024

  24. [32]

    Unmoderated usability studies evolved: Can gpt ask useful follow-up questions? International Journal of Human–Computer Interaction, 0 0 (0): 0 1--18, 2024

    Eduard Kuric, Peter Demcak, and Matus Krajcovic. Unmoderated usability studies evolved: Can gpt ask useful follow-up questions? International Journal of Human–Computer Interaction, 0 0 (0): 0 1--18, 2024. doi:10.1080/10447318.2024.2427978. URL https://doi.org/10.1080/10447318....

    2024

  25. [33]

    Realizing multimodal behavior

    Michael Kipp, Alexis Heloir, Marc Schr \"o der, and Patrick Gebhard. Realizing multimodal behavior. In Jan Allbeck, Norman Badler, Timothy Bickmore, Catherine Pelachaud, and Alla Safonova, editors, Intelligent Virtual Agents, pages 57--63, Berlin, Heidelberg, 2010. Springer Be...

    2010

  26. [34]

    Birgit Lugrin, Catherine Pelachaud, and David Traum, editors. The Handbook on Socially Interactive Agents: 20 years of Research on Embodied Conversational Agents, Intelligent Virtual Agents, and Social Robotics Volume 2: Interactivity, Platforms, Application, volume 48. Associ...

    2022

  27. [35]

    for real

    Adélaïde Genay, Anatole Lécuyer, and Martin Hachet. Being an avatar “for real”: A survey on virtual embodiment in augmented reality. IEEE Transactions on Visualization and Computer Graphics, 28 0 (12): 0 5071--5090, 2022. doi:10.1109/TVCG.2021.3099290

    2022

  28. [36]

    Me, my avatar, and my sneakers: The effect of avatar customization on the psychological ownership of virtual fashion items

    Do Yuon Kim and Ju Yeun Jang. Me, my avatar, and my sneakers: The effect of avatar customization on the psychological ownership of virtual fashion items. Computers in Human Behavior, 168: 0 108653, 2025. ISSN 0747-5632. URL https://doi.org/10.1016/j.chb.2025.108653

    2025

  29. [37]

    Design features of embodied conversational agents in ehealth: a literature review

    Silke ter Stal , Lean Leonie Kramer, Monique Tabak, Harm op den Akker , and Hermie Hermens. Design features of embodied conversational agents in ehealth: a literature review. International Journal of Human-Computer Studies, 138: 0 102409, 2020. ISSN 1071-5819. doi:https://doi....

    2020

  30. [38]

    Ying Qu, Chris K. Y. Lo, and Eunsoo Baek. From humanoid to virtual humans: A systematic literature review of avatar marketing. International Journal of Human–Computer Interaction, 0 0 (0): 0 1--20, 2025. doi:10.1080/10447318.2025.2464889. URL https://doi.org/10.1080/10447318.2...

    2025

  31. [39]

    Scarborough, and Erica Robles

    Victoria Groom, Clifford Nass, Tina Chen, Alexia Nielsen, James K. Scarborough, and Erica Robles. Evaluating the effects of behavioral realism in embodied agents. International Journal of Human-Computer Studies, 67 0 (10): 0 842--849, 2009. ISSN 1071-5819. doi:https://doi.org/...

    2009 doi

  32. [40]

    Nagayama

    Jun'ichiro Seyama and Ruth S. Nagayama. The uncanny valley: Effect of realism on the impression of artificial human faces. Presence, 16 0 (4): 0 337--351, 2007. doi:10.1162/pres.16.4.337

    2007 doi

  33. [41]

    Jens Reinhardt, Luca Hillen, and Katrin Wolf. Embedding conversational agents into ar: Invisible or with a realistic human body? In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction, TEI '20, page 299–310, New York, NY, USA,...

    2020

  34. [42]

    Exploring the impact of avatar representations in ai chatbot tutors on learning experiences

    Chek Tien Tan, Indriyati Atmosukarto, Budianto Tandianus, Songjia Shen, and Steven Wong. Exploring the impact of avatar representations in ai chatbot tutors on learning experiences. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI '25, pages...

    2025

  35. [43]

    A meta-analysis of the impact of the inclusion and realism of human-like faces on user experiences in interfaces

    Nick Yee, Jeremy N Bailenson, and Kathryn Rickertsen. A meta-analysis of the impact of the inclusion and realism of human-like faces on user experiences in interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '07, page 1–10, New York, ...

    2007

  36. [44]

    A review of evaluation practices of gesture generation in embodied conversational agents

    Pieter Wolfert, Nicole Robinson, and Tony Belpaeme. A review of evaluation practices of gesture generation in embodied conversational agents. IEEE Transactions on Human-Machine Systems, 52 0 (3): 0 379--389, 2022. doi:10.1109/THMS.2022.3149173

    2022

  37. [45]

    Development and validation of a virtual agent to screen tobacco and alcohol use disorders

    Marc Auriacombe, Sarah Moriceau, Fuschia Serre, Cécile Denis, Jean-Arthur Micoulaud-Franchi, Etienne de Sevin , Emilien Bonhomme, Stéphanie Bioulac, Mélina Fatseas, and Pierre Philip. Development and validation of a virtual agent to screen tobacco and alcohol use disorders. Dr...

    2018 doi

  38. [46]

    Substance use screening using virtual agents: Towards automated screening, brief intervention, and referral to treatment (sbirt)

    Timothy Bickmore, Amy Rubin, and Steven Simon. Substance use screening using virtual agents: Towards automated screening, brief intervention, and referral to treatment (sbirt). In Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, IVA '20, page...

    2020

  39. [47]

    Charting the evolution and future of conversational agents: A research agenda along five waves and new frontiers

    Sofia Sch \"o bel, Anuschka Schmitt, Dennis Benner, Mohammed Saqr, Andreas Janson, and Jan Marco Leimeister. Charting the evolution and future of conversational agents: A research agenda along five waves and new frontiers. Information Systems Frontiers, 26 0 (2): 0 729--754, A...

    2024 doi

  40. [48]

    Revolutionising faculty development and continuing medical education through ai-generated videos

    Irene Contreras, Samia Hossfeld, Katharine de Boer, Jane Thorley Wiedler, and Monica Ghidinelli. Revolutionising faculty development and continuing medical education through ai-generated videos. Journal of CME, 13 0 (1): 0 2434322, 2024. doi:10.1080/28338073.2024.2434322. URL ...

    2024

  41. [49]

    Badawy, Kitiwat Khamwan, and Daniel Carrion

    Mohamed K. Badawy, Kitiwat Khamwan, and Daniel Carrion. A pilot study of generative ai video for patient communication in radiology and nuclear medicine. Health and Technology, 15 0 (2): 0 395--404, Mar 2025. ISSN 2190-7196. doi:10.1007/s12553-025-00945-z. URL https://doi.org/...

    2025 doi

  42. [50]

    The fine line between automation and augmentation in website usability evaluation

    Andrea Esposito, Giuseppe Desolda, and Rosa Lanzilotti. The fine line between automation and augmentation in website usability evaluation. Scientific Reports, 14 0 (1): 0 10129, May 2024. ISSN 2045-2322. doi:10.1038/s41598-024-59616-0. URL https://doi.org/10.1038/s41598-024-59616-0

    2024 doi

  43. [51]

    Mayer, Nicoletta Adamo-Villani, Christos Mousas, Minsoo Choi, Luchcha Lam, Magzhan Mukanova, and Klay Hauser

    Fangzheng Zhao, Richard E. Mayer, Nicoletta Adamo-Villani, Christos Mousas, Minsoo Choi, Luchcha Lam, Magzhan Mukanova, and Klay Hauser. Recognizing and relating to the race/ethnicity and gender of animated pedagogical agents. Journal of Educational Computing Research, 62 0 (3...

    2024 doi

  44. [52]

    Linking response quality to survey engagement: A combined random scale and latent variable approach

    Stephane Hess and Amanda Stathopoulos. Linking response quality to survey engagement: A combined random scale and latent variable approach. Journal of Choice Modelling, 7: 0 1--12, 2013. ISSN 1755-5345. doi:https://doi.org/10.1016/j.jocm.2013.03.005. URL https://www.sciencedir...

    2013 doi

  45. [53]

    Users' motivation to participate in online crowdsourcing platforms

    Mokter Hossain. Users' motivation to participate in online crowdsourcing platforms. In 2012 International Conference on Innovation Management and Technology Research, pages 310--315, 2012. doi:10.1109/ICIMTR.2012.6236409

    2012

  46. [54]

    Soto and Oliver P

    Christopher J. Soto and Oliver P. John. Short and extra-short forms of the big five inventory–2: The bfi-2-s and bfi-2-xs. Journal of Research in Personality, 68: 0 69--81, 2017. ISSN 0092-6566. doi:https://doi.org/10.1016/j.jrp.2017.02.004. URL https://www.sciencedirect.com/s...

    2017 doi

  47. [55]

    Factor structure and measurement invariance of the cognitive failures questionnaire across the adult life span

    Philippe Rast, Daniel Zimprich, Martin Van Boxtel, and Jellemer Jolles. Factor structure and measurement invariance of the cognitive failures questionnaire across the adult life span. Assessment, 16 0 (2): 0 145--158, 2009. doi:10.1177/1073191108324440. URL https://doi.org/10....

    2009 doi

  48. [56]

    Is usability testing valid with prototypes where clickable hotspots are highlighted upon misclick? Journal of Systems and Software, 226: 0 112446, 2025

    Matus Krajcovic, Peter Demcak, and Eduard Kuric. Is usability testing valid with prototypes where clickable hotspots are highlighted upon misclick? Journal of Systems and Software, 226: 0 112446, 2025. ISSN 0164-1212. doi:https://doi.org/10.1016/j.jss.2025.112446. URL https://...

    2025

  49. [57]

    Thomas and Scott Clifford

    Kyle A. Thomas and Scott Clifford. Validity and mechanical turk: An assessment of exclusion methods and interactive experiments. Computers in Human Behavior, 77: 0 184--197, 2017. ISSN 0747-5632. doi:https://doi.org/10.1016/j.chb.2017.08.038. URL https://www.sciencedirect.com/...

    2017 doi

  50. [58]

    rspeer/wordfreq: v3.0

    Robyn Speer. rspeer/wordfreq: v3.0. https://doi.org/10.5281/zenodo.7199437, 2022. v3.0.2

    2022 doi

  51. [59]

    A study of demographic embodiments of product recommendation agents in electronic commerce

    Lingyun Qiu and Izak Benbasat. A study of demographic embodiments of product recommendation agents in electronic commerce. International Journal of Human-Computer Studies, 68 0 (10): 0 669--688, 2010. ISSN 1071-5819. doi:https://doi.org/10.1016/j.ijhcs.2010.05.005. URL https:/...

    2010 doi

  52. [60]

    Rethinking feminized service bots: user responses to abstract and gender-ambiguous chatbot avatars in a large-scale interaction study

    Anna Aum \"u ller, Andreas Winklbauer, Beatrice Schreibmaier, Bernad Batinic, and Martina Mara. Rethinking feminized service bots: user responses to abstract and gender-ambiguous chatbot avatars in a large-scale interaction study. Personal and Ubiquitous Computing, 28 0 (6): 0...

    2024 doi

  53. [61]

    Face-to-face versus web surveying in a high-internet-coverage population: Differences in response quality

    Dirk Heerwegh and Geert Loosveldt. Face-to-face versus web surveying in a high-internet-coverage population: Differences in response quality. Public Opinion Quarterly, 72 0 (5): 0 836--846, 10 2008. ISSN 0033-362X. doi:10.1093/poq/nfn045. URL https://doi.org/10.1093/poq/nfn045

    2008 doi

  54. [62]

    Doherty and Paul Sorenson

    Rina A. Doherty and Paul Sorenson. Keeping users in the flow: Mapping system responsiveness with user experience. Procedia Manufacturing, 3: 0 4384--4391, 2015. ISSN 2351-9789. doi:10.1016/j.promfg.2015.07.436. URL https://doi.org/10.1016/j.promfg.2015.07.436. 6th Internationa...

    2015 doi

  55. [63]

    Measuring turn-taking offsets in human-human dialogues

    Rebecca Lunsford, Peter Heeman, and Emma Rennie. Measuring turn-taking offsets in human-human dialogues. In Proceedings of the 17th Annual Conference of the International Speech Communication Association, pages 2895--2899, San Francisco, 09 2016. doi:10.21437/Interspeech.2016-1350

    2016 doi

  56. [64]

    Investigating the impact of multimodal feedback on user-perceived latency and immersion with llm-powered embodied conversational agents in virtual reality

    Morad Elfleet and Mathieu Chollet. Investigating the impact of multimodal feedback on user-perceived latency and immersion with llm-powered embodied conversational agents in virtual reality. In Proceedings of the 24th ACM International Conference on Intelligent Virtual Agents,...

    2024

  57. [65]

    Naoki Mukawa, Hiroki Sasaki, and Atsushi Kimura. How do verbal/bodily fillers ease embarrassing situations during silences in conversations? In The 23rd IEEE International Symposium on Robot and Human Interactive Communication, pages 30--35, 2014. doi:10.1109/ROMAN.2014.6926226

    2014

  58. [66]

    Can behavioral features reveal lying in an online personality questionnaire? the impact of mouse dynamics and speech

    Eduard Kuric, Peter Demcak, Peter Smrecek, and Stefan Benus. Can behavioral features reveal lying in an online personality questionnaire? the impact of mouse dynamics and speech. Computers in Human Behavior Reports, 18: 0 100683, 2025 b . ISSN 2451-9588. doi:10.1016/j.chbr.202...

    2025

  59. [67]

    Modularization is better: Effective code generation with modular prompting, 2025

    Ruwei Pan and Hongyu Zhang. Modularization is better: Effective code generation with modular prompting, 2025. URL https://arxiv.org/abs/2503.12483

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.