Pith. sign in

REVIEW 4 major objections 5 minor 26 references

CARIS: A Context-Adaptable Robot Interface System for Personalized and Scalable Human-Robot Interaction

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CARIS is a single open-source Wizard-of-Oz web interface that bundles teleoperation, perception, LLM-driven dialogue, and multimodal logging so HRI researchers can steer a robot through very different tasks—here a room tour and a mental-hea

desk verdict A workmanlike WoZ system paper with a genuinely useful integration but an overextended adaptability claim and a too-thin evaluation; send to review but expect major revision. read the letter →

arxiv 2509.00660 v1 pith:FDEFKNRG submitted 2025-08-31 cs.RO cs.HC

Felipe Arias-Russi , Yuanchen Bai , Angelique Taylor This is my paper · ORCID
classification cs.ROcs.HC
keywords Wizard-of-Ozhuman-robotinteractionmodularrobotinterfaceteleoperationLLMdialoguepersontrackingmultimodaldataloggingcontextadaptability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces CARIS, a web-based Wizard-of-Oz control panel designed to be reused across human-robot interaction contexts without rebuilding the system. It bundles four modules—localization, perception, conversation, and the wizard interface—so that a human operator can teleoperate the robot, watch live camera and LIDAR data, track and name people, converse through an LLM with speech input and output, and log every interaction in real time. The authors show the same setup driving a mobile robot in two pilot scenarios, a room tour and a mental-health check, and use wizard feedback to refine the interface. If the adaptability claim holds, HRI teams could stop building bespoke WoZ rigs and instead configure one general-purpose platform per study.

What carries the argument

The core mechanism is the four-module architecture: Localization, Perception, Conversation, and the Wizard interface. Localization uses a 2D LIDAR and SLAM with movement commands relayed through ROSBridge; Perception runs YOLO-based detection and DeepSORT tracking in a backend separate from the robot; Conversation connects speech-to-text, LLMs (cloud or local), and text-to-speech; and the Wizard Module ties them together as a web-based command center that also logs multimodal data. The separation of robot-side and backend-side processing is what lets each module be swapped or reconfigured for a new context.

What would settle it

Run CARIS on a second ground robot with different LIDAR, camera, and driver software; if the Wizard Module cannot issue movement commands or receive perception data without changing module internals, the adaptability claim is false. Alternatively, in a mental-health scenario, if the wizard cannot simultaneously move and converse, the ease-of-use claim weakens.

Watch

Extended reading notes

Core claim

The central claim is that CARIS's four-module decomposition—Localization, Perception, Conversation, and Wizard—decouples the robot's hardware from the higher-level Wizard-of-Oz decision loop, making the same interface adaptable to different contexts and robots. The Wizard Module acts as the command center: it sends movement commands, displays live video and LIDAR maps, lets the operator name and group tracked individuals, edits LLM prompts, and writes text, image, and JSON logs. The system was evaluated in a tour-guide scenario and a mental-health-check scenario with seven pilot participants and four main-study participants. In both contexts, CARIS successfully WoZ-controlled the robot, and

Load-bearing premise

The claim that CARIS adapts to most ground-based robots rests on the assumption that its modules—especially the movement relay and the separate backend for perception and conversation—are portable beyond the single TIAGo Base setup with its specific camera and computer; that portability was never tested.

Editorial extensions

If this is right

  • A researcher could take CARIS from a mental-health companion study to a tour-guide study by changing prompts and role settings instead of writing a new WoZ system.
  • The built-in multimodal logging—movement commands, TTS usage, photos, and LLM exchanges—would let a single experiment produce synchronized data for training data-driven HRI models.
  • Because perception and conversation run in a backend separate from the robot's control loop, swapping in a different camera or language model is a module change, not a system rewrite.
  • The study feedback—keyboard control, dropdown model selection, clearer notes—shows the interface can be iterated on quickly between pilot and main study.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the module boundaries hold, CARIS could extend to telepresence and remote caregiving, where a remote operator needs the same mix of movement, perception, and dialogue.
  • The pairing of person re-identification with interaction history suggests a path to longitudinal HRI studies, but privacy-sensitive contexts would need on-device storage and consent controls.
  • A direct stress-test of the adaptability claim would be deploying CARIS on a different ground robot base with different LIDAR and camera hardware and verifying that only configuration changes are needed.
  • The 'recommended prompts' and one-click communication suggestions point toward a semi-autonomous wizard where the LLM proposes actions and the human approves, cutting wizard workload in future studies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents CARIS, a modular Wizard-of-Oz (WoZ) platform for teleoperating social robots. CARIS combines a web interface (Svelte) with a FastAPI backend for perception (YOLO v8, DeepSORT) and LLM-based dialogue, plus a ROSBridge link to robot localisation and movement. The authors report two pilot studies: a mental-health companion scenario and a tour-guide scenario, with 7 participants in the pilot and 4 in a follow-up 'main study'. They report SUS scores and thematic analysis of interviews, and they argue that CARIS is context-adaptable, modular, and publicly available for HRI researchers.

Significance. If correctly supported, CARIS would be a useful open-source contribution: it combines teleoperation, perception, LLM dialogue, and multimodal data logging in a single web interface, and Table I suggests it offers a broader feature set than prior WoZ tools. The modular architecture (Localization, Perception, Conversation, Wizard) is a sensible design. However, the evidence is thin: the two pilots use the same hardware stack, the SUS means are far below the standard acceptability threshold, and no inferential statistics are provided. The central claim of adaptability across 'robotic platforms' is not demonstrated by the reported experiments.

major comments (4)
  1. [Sec. III-E and Abstract] The claim that CARIS 'can be adapted to most ground-based mobile robots with basic localization and speech' is unsupported by the evidence. Only one hardware stack (TIAGo Base, Oak-D camera on Raspberry Pi 4, FastAPI backend on Apple Silicon) was integrated and tested; both pilot scenarios use this same stack. The modules rely on specific ROSBridge topics (e.g., /cmd_vel) and camera drivers, but the paper does not demonstrate that these interfaces are robot-agnostic. Because cross-platform adaptability is a central contribution, either integrate a second mobile platform or provide a detailed mapping of generic module interfaces with justification of portability.
  2. [Secs. V and VII] The SUS results do not support the stated conclusions. Pilot SUS scores (M=40, SD=37.36) and main-study scores (M=43.75, SD=19.63) are both below the conventional 68-point threshold for acceptable usability, yet Sec. VII concludes 'These results demonstrate moderate usability CARIS.' No inferential statistics, confidence intervals, or effect sizes are reported, and no comparison between pilot and main study is made, so the claimed iterative improvement is not demonstrated. Please report appropriate analyses and explicitly interpret the low scores as a significant limitation.
  3. [Sec. IV (Participants)] The participant flow is unclear and likely underpowered for the claims. The text says 'we aimed to collect eight data points (two users and two wizards for each scenario),' but only four SUS scores are reported from the pilot. It is not stated how many sessions used a researcher as the wizard, nor whether the four SUS responses correspond to four distinct participants or to four wizard sessions. In the main study, n=4 is extremely small. Please clarify the exact number of participant-wizard sessions and report demographics for all participants, not only the 4 of 7 who completed the survey.
  4. [Abstract and Sec. I] The abstract and Sec. I state that CARIS is 'publicly available' and 'open-source,' but no repository URL, package name, or license appears anywhere in the manuscript. This is a load-bearing omission for a systems contribution: readers cannot verify or reuse the tool. Provide a code-availability statement with a stable link (e.g., Zenodo) and license, or remove the claim.
minor comments (5)
  1. [Sec. IV] Several typographical and grammatical errors: 'Futhermore' should be 'Furthermore'; 'inheritantly' should be 'inherently'; 'It’s suitability' should be 'its suitability'; 'ages aged' should be 'aged'; 'is HRI scenarios' likely means 'in HRI scenarios'; the abstract's 'enhance the usability wizard control' should be 'wizard control usability'.
  2. [Secs. IV and VII] Scenario naming is inconsistent: 'Room tour', 'lab tour', and 'tour guide' are used interchangeably. Use a single consistent name throughout.
  3. [Sec. VII] In the SUS quote, 'I'II needed to learn a lot of things...' appears to be a typo; should likely be 'I'd' or 'I'll'. Also, the discussion of 'average positive SUS scores' and 'average negative SUS range' is confusing; clarify that these are item-level means and explain reverse scoring for negative items.
  4. [Sec. IV] The IRB approval number is given as '#XX'; this placeholder should be replaced with the actual approval number before publication.
  5. [Sec. III-E] The statement 'CARIS can be adapted to most ground-based mobile robots' is presented without any architectural analysis or reference to external interface standards. At minimum, provide a list of the ROS topics and services that the Localization Module actually subscribes to/publishes, and note which of these are TIAGo-specific.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CARIS is an empirical systems paper with no derivation chain; the cross-robot adaptability claim is unsupported by evidence but not circular.

full rationale

This paper does not contain a derivation chain, fitted parameters, or equations, so the equation-level circularity patterns do not apply. The central claim is that CARIS is a modular, context-adaptable Wizard-of-Oz platform, supported by two pilot studies and a main study using the authors' own implementation. No claim in the paper is justified by a prior result from the same authors: the reference list contains no self-citations, and no uniqueness theorem or ansatz is imported from the authors' earlier work. The only passage resembling an overclaim is in Section III-E: 'Although we used a TIAGo Base, CARIS can be adapted to most ground-based mobile robots with basic localization and speech.' This is an unsupported generalization because only one robot integration was tested and the two pilot scenarios vary context but keep the same hardware/software stack. However, this is an evidence gap or correctness risk, not circularity: the adaptability claim is an empirical assertion about future portability, and it is not equivalent to any input by construction. Similarly, the SUS scores and thematic coding are self-reported usability data, which may be weak evidence for the system's general value, but they are not a prediction that reduces to its fitting inputs. Therefore, no circular step can be exhibited, and the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No fitted numerical constants are used in this paper: the system consumes off-the-shelf YOLOv8n, DeepSORT, Gemini/Llama/LLaVA models and reports no hand-fit parameters. The load-bearing assumptions are domain assumptions about WoZ validity, cross-platform portability, and the adequacy of small self-report samples.

assumptions (3)
  • domain assumption A human wizard operating a Wizard-of-Oz interface can simulate autonomous robot behavior well enough to yield valid HRI usability findings.
    Adopted from prior WoZ literature (Dow et al., Riek) and used to justify the usability evaluations in Sections IV-VII; no autonomous baseline is tested.
  • domain assumption The ROSBridge/FastAPI modular decomposition is portable across ground mobile robots with minimal reconfiguration.
    This is the core of the 'context-adaptable' claim, but only the TIAGo Base with an Oak-D camera on a Raspberry Pi 4 and an Apple Silicon backend is integrated (Section III-E).
  • domain assumption Self-report SUS scores and thematic coding from seven and four convenience-sampled participants are sufficient evidence of usability and adaptability.
    Used in Sections V and VII; no power analysis, no inferential statistics, and no comparison condition.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CARIS: A Context-Adaptable Robot Interface System for Personalized and Scalable Human-Robot Interaction." pith.science (2026). https://pith.science/paper/FDEFKNRG

@misc{pith2026250900660,
  author       = {Pith},
  title        = {Pith review of: CARIS: A Context-Adaptable Robot Interface System for Personalized and Scalable Human-Robot Interaction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FDEFKNRG}},
  note         = {Machine review of arXiv:2509.00660}
}
read the original abstract

The human-robot interaction (HRI) field has traditionally used Wizard-of-Oz (WoZ) controlled robots to explore navigation, conversational dynamics, human-in-the-loop interactions, and more to explore appropriate robot behaviors in everyday settings. However, existing WoZ tools are often limited to one context, making them less adaptable across different settings, users, and robotic platforms. To mitigate these issues, we introduce a Context-Adaptable Robot Interface System (CARIS) that combines advanced robotic capabilities such teleoperation, human perception, human-robot dialogue, and multimodal data recording. Through pilot studies, we demonstrate the potential of CARIS to WoZ control a robot in two contexts: 1) mental health companion and as a 2) tour guide. Furthermore, we identified areas of improvement for CARIS, including smoother integration between movement and communication, clearer functionality separation, recommended prompts, and one-click communication options to enhance the usability wizard control of CARIS. This project offers a publicly available, context-adaptable tool for the HRI community, enabling researchers to streamline data-driven approaches to intelligent robot behavior.

Figures

Figures reproduced from arXiv: 2509.00660 by the authors.

Figure 1
Figure 1. The CARIS Web Interface is a Wizard-of-Oz platform with percep [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. CARIS architecture with LIDAR, RGB camera, audio inputs; [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Procedural flow of study scenarios in our pilot and main study where [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 24 canonical work pages

  1. [1]

    How context and design shape human-robot trust and attributions,

    H. Biermann, P. Brauner, and M. Ziefle, “How context and design shape human-robot trust and attributions,” Paladyn, Journal of Behav- ioral Robotics, vol. 12, no. 1, pp. 74–86, 2021

    work page 2021

  2. [2]

    Wizard of Oz support throughout an iterative design process,

    S. Dow, B. MacIntyre, J. Lee et al., “Wizard of Oz support throughout an iterative design process,” IEEE Pervasive Computing, 2005

    work page 2005

  3. [3]

    Wizard of Oz studies in HRI: a systematic review and new reporting guidelines,

    L. D. Riek, “Wizard of Oz studies in HRI: a systematic review and new reporting guidelines,” J. Hum.-Robot Interact. , 2012

    work page 2012

  4. [4]

    WoZ4U: An Open- Source Wizard-of-Oz Interface for Easy, Efficient and Robust HRI Experiments,

    F. Rietz, A. Sutherland, S. Bensch et al. , “WoZ4U: An Open- Source Wizard-of-Oz Interface for Easy, Efficient and Robust HRI Experiments,” Frontiers in Robotics and AI , vol. 8, 2021

    work page 2021

  5. [5]

    A Design Flow Based on Docker and Kubernetes for ROS-based Robotic Software Applications,

    F. Lumpp, M. Panato, N. Bombieri, and F. Fummi, “A Design Flow Based on Docker and Kubernetes for ROS-based Robotic Software Applications,” ACM Trans. Embed. Comput. Syst. , 2024

    work page 2024

  6. [6]

    Do As I Can, Not As I Say: Grounding Language in Robotic Affordances,

    M. Ahn, A. Brohan, N. Brown et al. , “Do As I Can, Not As I Say: Grounding Language in Robotic Affordances,” 2022, arXiv:2204.01691

    arXiv 2022

  7. [7]

    Al- chemist: LLM-Aided End-User Development of Robot Applications,

    U. B. Karli, J.-T. Chen, V . N. Antony, and C.-M. Huang, “Al- chemist: LLM-Aided End-User Development of Robot Applications,” in ACM/IEEE Intern. Conf. on Human-Robot Interaction , 2024

    work page 2024

  8. [8]

    ToD4IR: A Humanised Task-Oriented Dialogue System for Industrial Robots,

    C. Li, X. Zhang, D. Chrysostomou, and H. Yang, “ToD4IR: A Humanised Task-Oriented Dialogue System for Industrial Robots,” IEEE Access, vol. 10, pp. 91 631–91 649, 2022

    work page 2022

Show all 26 references
  1. [9]

    Incremental Learning of Humanoid Robot Behavior from Natural Interaction and Large Language Models,

    L. B ¨armann, R. Kartmann, F. Peller-Konrad et al. , “Incremental Learning of Humanoid Robot Behavior from Natural Interaction and Large Language Models,” 2024, arXiv:2309.04316

    2024 arXiv

  2. [10]

    Understanding Large-Language Model (LLM)-powered Human-Robot Interaction,

    C. Y . Kim, C. P. Lee, and B. Mutlu, “Understanding Large-Language Model (LLM)-powered Human-Robot Interaction,” in Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, 2024, pp. 371–380, arXiv:2401.03217

    2024 arXiv

  3. [11]

    Scarecrows in Oz: The Use of Large Language Models in HRI,

    T. Williams, C. Matuszek, R. Mead, and N. Depalma, “Scarecrows in Oz: The Use of Large Language Models in HRI,” ACM Transactions on Human-Robot Interaction , vol. 13, no. 1, pp. 1–11, 2024

    2024

  4. [12]

    SCOUT: A Situated and Multi-Modal Human-Robot Dialogue Corpus,

    S. M. Lukin, C. Bonial, M. Marge et al. , “SCOUT: A Situated and Multi-Modal Human-Robot Dialogue Corpus,” in Proc. of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation , 2024, pp. 14 445–14 458

    2024

  5. [13]

    OpenWoZ: A Runtime-Configurable Wizard-of-Oz Framework for Human-Robot Interaction,

    G. Hoffman, “OpenWoZ: A Runtime-Configurable Wizard-of-Oz Framework for Human-Robot Interaction,” 2016

    2016

  6. [14]

    RISE: an open-source archi- tecture for interdisciplinary and reproducible human–robot interaction research,

    A. Groß, C. Sch ¨utze, M. Brandt et al., “RISE: an open-source archi- tecture for interdisciplinary and reproducible human–robot interaction research,” Frontiers in Robotics and AI , 2023

    2023

  7. [15]

    FLEX-SDK: An Open-Source Software Development Kit for Creating Social Robots,

    P. Alves-Oliveira, K. Mihata, R. Karim, E. A. Bjorling, and M. Cak- mak, “FLEX-SDK: An Open-Source Software Development Kit for Creating Social Robots,” in Proc. of the 35th Annual ACM Symposium on User Interface Software and Technology , 2022, pp. 1–10

    2022

  8. [16]

    An Open and Reconfigurable User Interface to Manage Complex ROS-based Robotic Systems,

    P. Malvido Fresnillo, S. Vasudevan, J. A. Perez Garcia, and J. L. Martinez Lastra, “An Open and Reconfigurable User Interface to Manage Complex ROS-based Robotic Systems,” IEEE Access, 2024

    2024

  9. [17]

    How can social robot use cases in healthcare be pushed - with an interoperable programming interface,

    R. Glauser, J. Holm, M. Bender, and T. B ¨urkle, “How can social robot use cases in healthcare be pushed - with an interoperable programming interface,” BMC Medical Informatics and Decision Making , 2023

    2023

  10. [18]

    Jocher, A

    G. Jocher, A. Chaurasia, and J. Qiu, “YOLOv8,” 2023. [Online]. Available: https://docs.ultralytics.com/models/yolov8/

    2023

  11. [19]

    Simple Online and Realtime Tracking with a Deep Association Metric,

    N. Wojke, A. Bewley, and D. Paulus, “Simple Online and Realtime Tracking with a Deep Association Metric,” 2017

    2017

  12. [20]

    Gemini 1.5 Flash API,

    Google, “Gemini 1.5 Flash API,” 2025. [Online]. Available: https://ai.google.dev/api/generate-content

    2025

  13. [21]

    Llama 3.1,

    Meta, “Llama 3.1,” 2024. [Online]. Available: https://huggingface.co/ meta-llama/Llama-3.1-8B

    2024

  14. [22]

    Visual Instruction Tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual Instruction Tuning,” 2023, arXiv:2304.08485

    2023 arXiv

  15. [23]

    TIAGo Base | Customisable AMR for Logistics and Research

    “TIAGo Base | Customisable AMR for Logistics and Research.” [Online]. Available: https://pal-robotics.com/robot/tiago-base/

  16. [24]

    FastAPI

    “FastAPI.” [Online]. Available: https://fastapi.tiangolo.com/

  17. [25]

    SUS: A quick and dirty usability scale,

    J. Brooke, “SUS: A quick and dirty usability scale,” Usability Eval. Ind., vol. 189, Nov. 1995

    1995

  18. [26]

    Thematic analysis,

    V . Clarke and V . Braun, “Thematic analysis,” The journal of positive psychology, vol. 12, no. 3, pp. 297–298, 2017

    2017

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.