Pith. sign in

REVIEW 5 major objections 5 minor 29 references

Conversations with Andrea: Visitors' Opinions on Android Robots in a Museum

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An android robot running fully autonomously in a public museum for six days was perceived by most visitors as an object of curiosity, not as uncanny.

desk verdict A genuinely novel field deployment of an autonomous LLM android, but the uncanniness conclusion overreaches the data. read the letter →

arxiv 2506.22466 v1 pith:TRHXXSEO submitted 2025-06-18 cs.RO cs.CY

Marcel Heisler , Christian Becker-Asano This is my paper
classification cs.ROcs.CY
keywords androidrobothuman-robotinteractionmuseumroboticsuncannyvalleylargelanguagemodelautonomousconversationfieldstudyvisitorinterviews
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper reports a field study in which the android robot Andrea held autonomous conversations with visitors at a public museum in Germany for six consecutive days. It argues that in this non-critical setting, even with no tele-operator in the loop, most visitors experienced the robot as an object of curiosity rather than as uncanny. Responses from 44 structured interviews and an automated cluster analysis of 4,436 logged requests converge: people chiefly want the robot to provide information about exhibits, to speak languages other than English, and to answer faster. Changing Andrea's wig and voice to vary perceived gender did not significantly change its overall positive evaluation. If the results are right, a fully autonomous LLM-driven android can be an acceptable public-facing information device in a museum.

What carries the argument

The mechanism that carries the study is the autonomous conversation pipeline: a push-to-talk button starts recording, Whisper transcribes speech into English, ChatGPT 3.5-turbo generates a reply using the maintained chat history, VITS synthesizes the spoken answer, and a lip-sync model animates the face while 52 pneumatic actuators move the upper body. A pose-estimation model on the eye camera steers the robot's gaze toward a detected nose keypoint. This pipeline is what makes the claim 'fully autonomous' meaningful, and the logged request categories it produces are the objective counterpart to the interview data; the convergence between the two is the paper's central evidence.

What would settle it

Instrument the system so that each conversation carries a visitor identifier and reset the chat history after every interaction; if the category proportions in the logged tables shift or reorder once repeated requests from the same visitor are collapsed, the claimed convergence between the logs and the interview themes would not survive.

Watch

Extended reading notes

Core claim

The authors conclude that in a non-critical museum scenario an android robot is not perceived as uncanny but as an object of curiosity by most visitors, even when it is run fully autonomously and not tele-operated. The evidence has two strands. In the interviews, curiosity was the most common reason for interacting, nine visitors mistook Andrea for real at first, and the mean usefulness rating was around 7 out of 10 in all three gender-cue conditions, significantly above the midpoint. In the logs, personal questions and conversational phrases account for the largest share of requests, followed by museum context and requests to switch language, while the most wanted improvement was multilingual support and the measured median response time was 5.43 seconds. The paper takes the agreement between interview themes and log categories as the main support for its conclusion.

Load-bearing premise

The load-bearing assumption is that each of the 4,436 logged requests is a usable independent observation, even though the paper concedes that the chat history was not always reset between interlocutors and there is no definitive way to tell different conversations apart.

Editorial extensions

If this is right

  • A fully autonomous LLM-driven android can operate acceptably in a public museum day after day without tele-operation.
  • Museum visitors' primary demand is exhibit information, so an information-agent role is the natural deployment target.
  • Multi-language support and faster responses, with a measured median latency of 5.43 seconds, are the concrete improvements that deployment requires.
  • Changing voice and hair cues to signal gender did not shift overall usefulness ratings, with all condition means near 7 out of 10.
  • Interview themes and logged request categories align, so log analysis can serve as a check on self-report data in human-robot interaction field studies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper leaves implicit is to run the same autonomous platform in a role where a wrong answer has a real cost, such as giving directions or citing exhibit facts, to see whether curiosity-based acceptance survives accountability.
  • A testable extension is to return to the same museum after the novelty has worn off and measure whether the share of uncanny or negative reactions rises as repeat exposure increases.
  • Because several visitors mistook Andrea for human at first, the setup could be repurposed to measure how long a visitor must interact before the android is classified as non-human.
  • The null gender effect is worth probing with a within-subjects design; the paper's between-days comparison may be too weak to detect a real effect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper reports a six-day field deployment of 'Andrea', a fully autonomous LLM-driven android robot, in the Mercedes-Benz Museum in Germany. Data come from 44 structured visitor interviews and from system logs of 4,436 user requests. The authors report that visitors were generally positive, that the most desired future use is providing exhibit information, that multi-language support and faster response times are the most requested improvements, and that a gender manipulation (wig and voice) had no significant effect on overall evaluation. Topic clustering of the logged requests is presented as converging with interview themes. The paper concludes that, in a non-critical museum scenario, an autonomous android is perceived not as uncanny but as an object of curiosity by most visitors.

Significance. The study provides rare real-world evidence on a fully autonomous, LLM-based android in a public museum, with independently collected interview and log data. The detailed description of the hardware/software pipeline and the practical improvement targets (multi-language support, response latency, lip-sync) are useful for the HRI field. The authors are transparent about several limitations, including the subjective nature of the BERTopic reduction and the acknowledged selection bias in interviews. If the conclusions are appropriately scaled to the actual data, the paper is a valuable exploratory field report; the current overreach in the central uncanny claim and in the log-based proportion claims needs correction.

major comments (5)
  1. [Section VI / Abstract] The conclusion that 'an android robot is not perceived as uncanny but as an object of curiosity by most visitors' is not supported by the reported data. The interview guide in Section IV-A contains no direct question about eeriness or the uncanny valley; the only supporting evidence is the open-ended mention of 'felt uncanny' by 5 of 14 non-interactors in Table II and of a 'strange feeling' by 2 interviewees in Table III. Given the acknowledged selection bias in Section V-C and the absence of a reported response rate, the absence of the word 'uncanny' cannot support a population-level claim. The conclusion should be restricted to the interviewed visitors, and the lack of a direct uncanny measure should be stated explicitly.
  2. [Abstract / Section V-B, Table VII] The abstract's claim that 'after chitchat and personal questions, most of the 4436 collected requests asked for information related to the museum and to converse in a different language' misstates the logged counts. In Table VII, personal questions account for 1,608 requests and conversational phrases for 818, while context accounts for 580 and language for 462; together context and language are 1,042 of 4,436 requests, or about 23%. They become a majority only if the 914-request 'other' category is also excluded. The sentence should be rewritten to describe the proportions accurately and to specify that the comparison is among the categorized non-chitchat, non-personal requests.
  3. [Section IV-B / Section V-B] The log analysis treats all 4,436 requests as independent observations even though the authors state in Section IV-B that 'Since the chat history was not always reset after each interlocutor, there is no definitive way to differentiate between different conversations.' A single visitor could therefore account for many requests, so the category counts in Tables VII and VIII are request-level statistics, not visitor-level statistics. Claims that the logs show what 'visitors' asked for should be rephrased as what 'requests' contained, and the segmentation caveat should be repeated where these counts support the discussion.
  4. [Section V-A, evaluation item (2)] The abstract's statement that changing the voice and wig 'did not have a significant impact' overstates the ANOVA result (F = 0.237, df1 = 2, df2 = 15.9, p > 0.79). With per-condition sample sizes of 12, 23, and 7, the test has very low power, the conditions are confounded with calendar day, and no manipulation check confirms that visitors perceived the intended gender. The paper should say that no significant effect was found in this small sample, and should avoid implying that the null hypothesis of no effect is established.
  5. [Section V-A / Section V-B] The claimed convergence between interview themes and log categories is weakened by shared subjective judgment: the BERTopic target number of topics (15) and MMR diversity (0.1) were selected through subjective exploration in Section IV-B, and both the manual grouping of clusters and the manual interview-topic extraction in Section V-A depend on the same authors' judgment. The triangulation should be presented as exploratory convergence rather than independent confirmation, and the potential for coding expectations to influence both channels should be acknowledged.
minor comments (5)
  1. [Section IV-B] There are small typos in the text: 'Outliers are reduces' should be 'Outliers are reduced', and 'ANOV A' in Section V-A should be 'ANOVA'.
  2. [Section V-A, demographic data] The demographic reporting would be clearer if the text specified how the 'undefined' gender category interacts with the missing values; with 27 male, 10 female, and 5 undefined respondents, the reader cannot immediately reconcile the statements about four missing gender/age/nationality and one missing interaction information.
  3. [Section III-C / Figure 3] The statement 'There was deliberately no advance notice' should be made more precise: the sign in front of the push-to-talk button instructs visitors to press and hold it, so the no-advance-notice refers to the absence of public announcements about the robot, not to the absence of usage instructions.
  4. [Section V-B, Table IX] In Table IX, the column header 'Lip' should be defined in the caption as the lip-sync model, and the 'Dependencies' column would benefit from a sentence clarifying that In(s), Out(c), and Out(s) refer to input audio duration, output character length, and output speech duration, respectively.
  5. [Section V-C] The limitations paragraph would benefit from explicitly stating that no response rate for the interviews is available, since this directly affects how cautiously the 'most visitors' phrasing should be interpreted.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; interviews and logs are independent evidence, and the uncanny-avoidance conclusion is an interpretive claim rather than a derivation from its inputs.

full rationale

Walking the paper's derivation chain: the central claims are (i) qualitative interview themes about motivations, use-cases, and improvements, and (ii) proportions of logged request categories. These two evidence streams are collected independently (structured interviews vs. system logs) and neither is fitted to the other. The conclusion that the robot was 'not perceived as uncanny but as an object of curiosity by most visitors' is an interpretation of open-ended interview answers (e.g., 5 of 14 non-interactors reporting 'felt uncanny'), not a quantity derived from the interview protocol; the absence of a direct uncanniness question is a validity limitation, not circularity. The only self-citations, e.g., [16] and [19], are technical references for the software pipeline (STT, TTS, chat, lip-sync) and are not used to justify the empirical conclusions. The log-count limitation, namely that 'there is no definitive way to differentiate between different conversations,' is a data-quality caveat, not a case where an output is equivalent to an input by construction. Therefore no circular step is exhibited.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claims rest on interview data and log clustering, not on a theoretical derivation. The only fitted model parameters are BERTopic hyperparameters, and no new theoretical entities are introduced.

free parameters (2)
  • BERTopic target number of topics = 15
    Section IV-B states: 'After subjectively exploring some values, 15 was chosen as a fixed number of topics to reduce to.' This manual hyperparameter choice shapes all cluster and category counts in Tables VII and VIII.
  • Maximal Marginal Relevance diversity = 0.1
    Section IV-B sets diversity to 0.1 for representative phrase extraction, which affects which phrases are manually examined and grouped into categories.
assumptions (4)
  • domain assumption Whisper in translate mode yields adequate English transcripts of all visitor speech, including non-English requests
    Section III-B: STT runs 'in translate mode, so that the model transcribes everything into English'. If translation distorts non-English requests, categories such as 'language' could be underestimated.
  • domain assumption Push-to-talk captures one complete utterance per press, and the logs are sufficiently complete despite unlogged thinking interruptions
    Section IV-B notes that interruptions of the thinking process were not logged and interruption information was only logged from the third day onward.
  • standard math The statistical tests, Welch ANOVA and Wilcoxon test on the evaluation ratings, are valid for the small group sizes
    Section V-A2 reports one-way ANOVA with unequal variances and a Wilcoxon test on pooled ratings; group sizes of 7 to 23 make the null result about gender conditions weak but not invalid.
  • domain assumption Agglomerative reduction to 15 BERTopic clusters preserves semantically meaningful categories
    Section IV-B: after manually exploring values, 15 topics were chosen; the clustering output is then manually grouped into six categories, so the final categories depend on both the algorithm and the authors' judgment.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Conversations with Andrea: Visitors' Opinions on Android Robots in a Museum." pith.science (2026). https://pith.science/paper/TRHXXSEO

@misc{pith2026250622466,
  author       = {Pith},
  title        = {Pith review of: Conversations with Andrea: Visitors' Opinions on Android Robots in a Museum},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TRHXXSEO}},
  note         = {Machine review of arXiv:2506.22466}
}
read the original abstract

The android robot Andrea was set up at a public museum in Germany for six consecutive days to have conversations with visitors, fully autonomously. No specific context was given, so visitors could state their opinions regarding possible use-cases in structured interviews, without any bias. Additionally the 44 interviewees were asked for their general opinions of the robot, their reasons (not) to interact with it and necessary improvements for future use. The android's voice and wig were changed between different days of operation to give varying cues regarding its gender. This did not have a significant impact on the positive overall perception of the robot. Most visitors want the robot to provide information about exhibits in the future, while opinions on other roles, like a receptionist, were both wanted and explicitly not wanted by different visitors. Speaking more languages (than only English) and faster response times were the improvements most desired. These findings from the interviews are in line with an analysis of the system logs, which revealed, that after chitchat and personal questions, most of the 4436 collected requests asked for information related to the museum and to converse in a different language. The valuable insights gained from these real-world interactions are now used to improve the system to become a useful real-world application.

Figures

Figures reproduced from arXiv: 2506.22466 by the authors.

Figure 1
Figure 1. A view from the back of the setup showing the interaction of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Still images of the two animations: (left) while recording and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. A sketch of the setup of Andrea during its public demonstration in [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Top: distribution of age groups over gender; bottom: distribution [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Descriptives of the visitors’ evaluations (top) and box-plots (bottom) [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

29 extracted references · 24 canonical work pages

  1. [1]

    Not that Uncanny After All? An Ethnographic Study on Android Robots Perception of Older Adults in Germany and Japan,

    F. Carros, B. B ¨urvenichet al., “Not that Uncanny After All? An Ethnographic Study on Android Robots Perception of Older Adults in Germany and Japan,” inSocial Robotics. Cham: Springer Nature Switzerland, 2022, vol. 13818, pp. 574–586, series Title: Lecture Notes in Computer Science

    work page 2022

  2. [2]

    Nadine the Social Robot: Three Case Studies in Everyday Life,

    N. M. Thalmann, N. Mishra, and G. Tulsulkar, “Nadine the Social Robot: Three Case Studies in Everyday Life,” inSocial Robotics. Cham: Springer International Publishing, 2021, pp. 107–116

    work page 2021

  3. [3]

    Design of an android robot head for stage performances,

    D. Choi, D.-W. Leeet al., “Design of an android robot head for stage performances,”Artif Life Robotics, vol. 16, no. 3, pp. 315–317, Dec

    work page

  4. [4]

    Intelligent Conversational Android ERICA Applied to Attentive Listening and Job Interview

    T. Kawahara, K. Inoue, and D. Lala, “Intelligent Conversational Android ERICA Applied to Attentive Listening and Job Interview,” May 2021. [Online]. Available: http://arxiv.org/abs/2105.00403

    work page Pith review arXiv 2021

  5. [5]

    Social Robots and Digital Humans as Job Interviewers: A Study of Human Reactions Towards a More Nat- uralistic Interaction,

    E. Baka, N. Mishraet al., “Social Robots and Digital Humans as Job Interviewers: A Study of Human Reactions Towards a More Nat- uralistic Interaction,” inHuman-Computer Interaction. Technological Innovation, M. Kurosu, Ed. Cham: Springer International Publishing, 2022, pp. 455–474

    work page 2022

  6. [6]

    Deploying social robots in museum settings: A quasi-systematic review exploring purpose and acceptability,

    N. Gasteiger, M. Hellou, and H. S. Ahn, “Deploying social robots in museum settings: A quasi-systematic review exploring purpose and acceptability,”International Journal of Advanced Robotic Systems, vol. 18, no. 6, p. 17298814211066740, Nov. 2021, publisher: SAGE Publications. [Online]. Available: https: //doi.org/10.1177/17298814211066740

    doi:10.1177/17298814211066740 2021

  7. [7]

    Public Perceptions of the Appropriateness of Robots in Museums and Galleries,

    C. Webster and S. Ivanov, “Public Perceptions of the Appropriateness of Robots in Museums and Galleries,”Journal of Smart Tourism, vol. 2, no. 1, pp. 33–39, Mar. 2022. [Online]. Available: https://doi.org/10.52255/SMARTTOURISM.2022.2.1.4

    work page doi:10.52255/smarttourism.2022.2.1.4 2022

  8. [8]

    Androide in der Kunst: Sind Roboter die neuen Hipster?

    Boris Pofalla, “Androide in der Kunst: Sind Roboter die neuen Hipster?”FAZ.NET, Dec. 2016. [Online]. Avail- able: https://www.faz.net/aktuell/feuilleton/kunst-und-architektur/ androide-in-der-kunst-sind-roboter-die-neuen-hipster-14579292.html

    work page 2016

Show all 29 references
  1. [9]

    Humanoid co-workers: How is it like to work with a robot?

    A. Vishwanath, A. Singhet al., “Humanoid co-workers: How is it like to work with a robot?” in2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO- MAN), Oct. 2019, pp. 1–6, iSSN: 1944-9437. [Online]. Available: https://ieeexplore.ieee.org/ab...

    2019

  2. [10]

    The uncanny in the wild. analysis of unscripted human–android interaction in the field,

    A. M. Rosenthal-von der P ¨utten, N. C. Kr ¨ameret al., “The uncanny in the wild. analysis of unscripted human–android interaction in the field,”International Journal of Social Robotics, vol. 6, pp. 67–83, 2014

    2014

  3. [11]

    Exploring the uncanny valley with Geminoid HI-1 in a real-world application,

    C. Becker-Asano, K. Ogawaet al., “Exploring the uncanny valley with Geminoid HI-1 in a real-world application,” inProceedings of IADIS International conference interfaces and human computer interaction, 2010, pp. 121–128

    2010

  4. [12]

    The Uncanny Valley [From the Field],

    M. Mori, K. F. MacDorman, and N. Kageki, “The Uncanny Valley [From the Field],”IEEE Robotics & Automation Magazine, vol. 19, no. 2, pp. 98–100, Jun. 2012, conference Name: IEEE Robotics & Automation Magazine

    2012

  5. [13]

    Science fiction reduces the eeriness of android robots: A field experiment,

    M. Mara and M. Appel, “Science fiction reduces the eeriness of android robots: A field experiment,”Computers in Human Behavior, vol. 48, pp. 156–162, Jul. 2015. [Online]. Available: https: //www.sciencedirect.com/science/article/pii/S0747563215000199

    2015

  6. [14]

    Crafting a museum guide using chatgpt4,

    G. Trichopoulos, M. Konstantakiset al., “Crafting a museum guide using chatgpt4,”Big Data and Cognitive Computing, vol. 7, no. 3,

  7. [15]

    A Conversational Agent as Museum Guide – Design and Evaluation of a Real-World Application,

    S. Kopp, L. Gesellensetteret al., “A Conversational Agent as Museum Guide – Design and Evaluation of a Real-World Application,” in Intelligent Virtual Agents. Berlin, Heidelberg: Springer, 2005, pp. 329–343

    2005

  8. [16]

    An Android Robot Head as Embodied Conversational Agent,

    M. Heisler and C. Becker-Asano, “An Android Robot Head as Embodied Conversational Agent,” inISR Europe 2023; 56th International Symposium on Robotics, Sep. 2023, pp. 93–99. [Online]. Available: https://ieeexplore.ieee.org/document/10363058

    2023

  9. [17]

    Robust Speech Recognition via Large-Scale Weak Supervision,

    A. Radford, J. W. Kimet al., “Robust Speech Recognition via Large-Scale Weak Supervision,” Dec. 2022. [Online]. Available: http://arxiv.org/abs/2212.04356

    2022 arXiv

  10. [18]

    Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech,

    J. Kim, J. Kong, and J. Son, “Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech,” Jun. 2021. [Online]. Available: http://arxiv.org/abs/2106.06103

    2021 arXiv

  11. [19]

    Making an Android Robot Head Talk,

    M. Heisler, S. Kopp, and C. Becker-Asano, “Making an Android Robot Head Talk,” in2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Aug. 2023, pp. 1837–1842, iSSN: 1944-9437. [Online]. Available: https://ieeexplore.ieee.org/document/10309532

    2023

  12. [20]

    Bertopic: Neural topic modeling with a class-based tf-idf procedure,

    M. Grootendorst, “Bertopic: Neural topic modeling with a class-based tf-idf procedure,”arXiv preprint arXiv:2203.05794, 2022

    2022 arXiv

  13. [21]

    Advice to new human-robot interaction researchers,

    T. Belpaeme, “Advice to new human-robot interaction researchers,” Human-robot interaction: Evaluation methods and their standardiza- tion, pp. 355–369, 2020

    2020

  14. [22]

    How quickly should communication robots respond?

    T. Shiwa, T. Kandaet al., “How quickly should communication robots respond?” in2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2008, pp. 153–160

    2008

  15. [23]

    Museum robot guides or conventional audio guides? An experimental study,

    A.-M. Velentza, D. Heinke, and J. Wyatt, “Museum robot guides or conventional audio guides? An experimental study,”Advanced Robotics, vol. 34, pp. 1571–1580, Dec. 2020

    2020

  16. [24]

    Xtts: a massively multilingual zero-shot text-to-speech model,

    E. Casanova, K. Daviset al., “Xtts: a massively multilingual zero-shot text-to-speech model,” inInterspeech 2024, 2024, pp. 4978–4982

    2024

  17. [25]

    Facexhubert: Text-less speech- driven e(x)pressive 3d facial animation synthesis using self- supervised speech representation learning,

    K. I. Haque and Z. Yumak, “Facexhubert: Text-less speech- driven e(x)pressive 3d facial animation synthesis using self- supervised speech representation learning,” inINTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (ICMI ’23). New York, NY , USA: ACM, 2023. [Online]. Availab...

    2023

  18. [26]

    W ASABI for affect simulation in human-computer interaction,

    C. Becker-Asano, “W ASABI for affect simulation in human-computer interaction,” inProc. on Emotion Representations and Modelling for HCI Systems. Sydney, Australia: Springer, 2014

    2014

  19. [27]

    Affective computing with pri- mary and secondary emotions in a virtual human,

    C. Becker-Asano and I. Wachsmuth, “Affective computing with pri- mary and secondary emotions in a virtual human,”Autonomous Agents and Multi-Agent Systems, vol. 20, no. 1, pp. 32–49, January 2010

    2010

  20. [2011]

    Available: https://doi.org/10.1007/s10015-011-0938-4

    [Online]. Available: https://doi.org/10.1007/s10015-011-0938-4

    doi

  21. [2023]

    Available: https://www.mdpi.com/2504-2289/7/3/148

    [Online]. Available: https://www.mdpi.com/2504-2289/7/3/148

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.