Pith. sign in

REVIEW 4 major objections 5 minor 30 references

Gaze-supported Large Language Model Framework for Bi-directional Human-Robot Interaction

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A gaze- and speech-aware LLM interface lets assistive robots adapt to ambiguous tasks, and in a lab test it raised perceived confidence over a scripted pipeline while keeping usability and gaze metrics unchanged.

desk verdict A useful gaze+speech LLM systems paper whose abstract overclaims; the real result is one significant self-report item among mostly nulls. read the letter →

arxiv 2507.15729 v1 pith:WYGR33OB submitted 2025-07-21 cs.RO cs.HC

Jens V. Rüppel , Andrey Rudenko , Tim Schreiter , Martin Magnusson , Achim J. Lilienthal This is my paper
classification cs.ROcs.HC
keywords human-robotinteractionlargelanguagemodelsgazetrackingspeechinterfaceassistiveroboticsmultimodalchain-of-thoughtreasoningtaskadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a modular, gaze- and speech-informed interface built around a large language model can support bi-directional human-robot interaction in open-ended tasks, and that this approach is a genuine alternative to traditional scripted interaction. In a two-condition lab study with a mobile robot performing a six-step pick-and-place task, the LLM condition produced significantly higher perceived confidence (M = 5.6 vs 4.4, p = .039, d = 0.81) while all usability and gaze metrics were statistically indistinguishable from the scripted condition. The authors conclude that the choice depends on the task: scripted pipelines remain better suited to well-defined, repetitive steps, whereas the LLM framework pays off when instructions are ambiguous and users ask questions. If the claim holds, robot designers get a transferable architecture and a concrete tradeoff between adaptability and deployment cost.

What carries the argument

The load-bearing mechanism is the interaction loop's reasoning module: a prompt engineered in six parts that feeds a Python-dictionary state representation into GPT-4o-mini with chain-of-thought prompting (a technique that asks the model to reason step by step), letting the model generate executable Python code and robot API calls rather than selecting from a fixed action menu. Around it sits an input fusion module that combines speech transcription, mobile eye-tracker gaze directed at detected objects, and vision from open-vocabulary object detection, segmentation, and scene captioning, all condensed into the dictionary that grounds each LLM call. The action module executes the generated code through high-level robot SDK wrappers, which is what makes the framework transferable across robots: only the wrapper functions change. The language-based interaction state representation is what keeps the system real-time and local, avoiding heavy image-based perception in the reasoning loop.

What would settle it

A replication that lets the scripted condition answer clarifying questions from a predefined menu, or that reports the per-condition balance of the question-asking subsample, would test whether the confidence advantage survives; if the effect disappears, it is an artifact of the script's rigidity.

Watch

Extended reading notes

Core claim

The central claim is that an LLM-driven HRI framework with gaze, speech, and 3D vision input matches a traditional scripted pipeline on efficiency, usability, and engagement, while being more adaptable to ambiguous situations and significantly increasing participants' perceived confidence in their own task performance. This is a new application rather than a new mechanism: the paper's contribution is showing, with quantitative and qualitative user data, that an LLM backbone can substitute for hand-coded interaction logic without degrading measured HRI outcomes on well-defined tasks, and with a measurable confidence benefit when the scenario contains ambiguity. The one significant quantitative result is the perceived-confidence difference (p = .039, effect size d = 0.81); the gaze metrics, fixation times, saccades, and cognitive load showed no significant differences, and energy consumption was higher in the LLM condition (1972 Wh vs 1784 Wh).

Load-bearing premise

The comparison treats a scripted condition that simply repeated the predefined instruction and could not answer questions as a fair representative of traditional scripted HRI, and it depends on the 21 participants who asked at least one question being balanced across conditions.

Editorial extensions

If this is right

  • LLM-driven HRI can absorb ambiguous or unexpected user questions without robot reprogramming, because the reasoning module generates new responses and robot actions on the fly.
  • For well-defined, repetitive tasks, scripted pipelines remain the cheaper and more predictable choice, since the LLM condition in this study consumed more energy and produced occasional redundant output.
  • The significant perceived-confidence gain (d = 0.81) suggests that adaptive dialogue can improve how sure users feel about their own performance even when objective task metrics do not change.
  • The framework's modular interface means a different robot can be supported by swapping only the high-level SDK wrappers for speech, signaling, and motion.
  • Researchers and designers should treat task ambiguity as the deciding variable when choosing between LLM and scripted interaction, not general preference for one technology.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the confidence effect replicates, gaze-plus-LLM interfaces could be used where user self-assurance matters more than speed, such as training or assisting novices on unfamiliar procedures.
  • The null gaze and usability results leave open whether the confidence gain comes from the LLM's ability to answer questions or from the gaze channel itself; an ablation with gaze removed would separate these.
  • A hybrid controller that runs scripted routines by default and escalates to the LLM only when the user asks a question or the task state becomes ambiguous could deliver both efficiency and adaptability.
  • Because the analysis covers only participants who asked at least one question, the result may not generalize to users who complete tasks without interacting; that untouched baseline group deserves its own comparison.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents a modular, locally deployable human-robot interaction framework that fuses speech, gaze, and multi-camera vision through an LLM backbone (GPT-4o-mini) with chain-of-thought prompting, and it evaluates the framework in a lab study (N=29) against a scripted interaction pipeline on a multi-step pick-and-place task with an ambiguous box-selection step. The quantitative results show no significant differences in PSSUQ usability, gaze engagement metrics, cognitive-load measures, or task-completion time; the only significant difference is a single perceived-confidence item favoring the LLM condition (p=.039, d=0.81), while LLM energy consumption is higher. The manuscript concludes that an LLM-based approach enhances adaptability and marginally improves user engagement and task-execution metrics, and that scripted pipelines remain suitable for more straightforward tasks.

Significance. The architectural contribution is genuine and potentially useful: the framework is modular, transferable across robot SDKs, runs locally, and integrates real-time speech, mobile eye tracking, and open-vocabulary object detection into a language-based state representation. The authors also report several dissemination events that informed design choices, and the lab study uses standard instruments (PSSUQ, Tobii gaze metrics) with an external scripted baseline. However, the central comparative claim is not currently supported by the reported statistics: all direct engagement and task-execution measures are null, and the single positive effect comes from a post-hoc subset of participants with no reported per-condition breakdown and no explicit multiple-comparison correction. With these empirical issues addressed, the systems contribution would be publishable; as it stands, the conclusion needs substantial revision.

major comments (4)
  1. [Abstract; §V-A] The abstract's claim that the LLM approach 'marginally improves user engagement and task execution metrics' is not supported by the results reported in §V-A. PSSUQ subscales, fixation and saccade metrics, cognitive-load measures, and task-execution time are all reported as non-significant, with only a 'weak trend' for task duration. The sole significant difference is perceived confidence on one 7-point item (M=5.6 vs. 4.4, p=.039, d=0.81). The conclusion should either be restricted to 'perceived confidence was higher in the LLM condition' or be supported by additional pre-specified analyses.
  2. [§IV-B; §V-A] The comparison is based on the post-hoc subset of n=21 participants who asked at least one question and received a vocal response, but no per-condition breakdown of this subset is given. Because the scripted condition 'simply repeated the predefined instruction' while the LLM condition answered dynamically, the probability of being included in the subset is likely condition-dependent; if, for example, the subset is predominantly from the LLM condition, the significant confidence effect could reflect selection rather than the interaction mode. The authors should report the number of participants per condition in the subset, compare baseline characteristics, and analyze the full sample or justify the subset restriction before the study.
  3. [§IV-B; §V-A] It is unclear whether the perceived-confidence p-value received any multiple-comparison correction. The manuscript states that MANOVA with Games-Howell post-hoc tests was used for questionnaire data and Bonferroni-corrected pairwise comparisons for gaze data, but the single confidence item is reported only with its uncorrected p=.039. Given that many measures were collected, this p-value is weak evidence on its own. The authors should state explicitly which tests were corrected and present adjusted p-values or confidence intervals.
  4. [§IV-B; §V-C] The interpretation of the comparison is constrained by the design asymmetry: the scripted condition is unable to answer clarifying questions by construction, while the LLM condition can. This makes the conclusion that 'the suitability of an LLM-based system depends on the nature of the task' partly a consequence of the baseline's limited capability. The authors should discuss whether a deployed scripted system would include a small set of clarification responses, or frame the conclusion as specifically about a non-adaptive baseline rather than scripted systems in general.
minor comments (5)
  1. [Abstract; §IV-B] The abstract refers to 'two lab studies,' but Section IV-B describes one new experiment; the first study appears to be prior work [4]. Please clarify whether the paper reports one or two studies.
  2. [Figure 8] The legend 'Duration of interval (in s)' should specify which interval is meant (interaction phase, execution phase, or total task), since the paper defines two phases in Section IV-B.
  3. [§IV-B] The sentence 'Participants were assigned in a randomized order to one of the conditions' should state the exact allocation procedure (e.g., alternating, block randomization) and how many participants were in each condition.
  4. [§V-A] The energy-consumption numbers (1972 Wh vs. 1784 Wh) are reported without a statistical test or the measurement window; please add this information.
  5. [Throughout] Minor formatting issues include 'V . Ruppel' in the author block, 'MANOV A' in Section IV-B, and the redundant phrase 'PSSUQ questionnaire'; these should be corrected.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the central comparison is anchored to external instruments (PSSUQ, Tobii gaze metrics, timing, energy), self-citations are motivational rather than load-bearing, and the disclosed scripted-baseline asymmetry is a study-validity concern, not a derivation that reduces to its inputs.

full rationale

This systems paper makes no first-principles derivation, so the classical circularity patterns mostly do not apply. The central empirical claim that the LLM-based framework 'enhances adaptability and marginally improves user engagement and task execution metrics' (abstract) is anchored to an external comparison with a scripted pipeline and to standard, framework-independent instruments: the PSSUQ usability questionnaire, Tobii gaze metrics, task-execution timing, and measured energy consumption. No parameter is fitted to a data subset and then renamed a prediction. The sole significant difference, perceived confidence (M = 5.6 vs 4.4, p = .039, d = 0.81, Section V-A), is a reported measurement, and the null PSSUQ and gaze results are reported alongside it. Self-citations are present but not load-bearing: prior work [4] supplies the hypothesis 'a scripted routine might be more effective for repetitive actions, while the LLM condition is more suited for tasks that require dynamic adaptations' (Section IV-B), which the present study independently tests with new participants (N = 29), and the conclusions in Section V-C rest on the new data. The one partially constructed element is explicitly disclosed rather than hidden: 'In the scripted condition, the system's response simply repeated the predefined instruction, whereas in the LLM condition, the response was dynamically generated based on the content of a provided Python dictionary' (Section IV-B). The scripted baseline therefore cannot resolve the deliberately ambiguous box-selection step by construction, so part of the 'adaptability' advantage is intrinsic to the manipulation. That asymmetry, together with the post-hoc n = 21 analysis subset without a reported per-condition breakdown and the single uncorrected p-value, is a fairness and statistical-robustness threat to the generalization 'complex and multi-step realistic interactions benefit from our LLM framework' (Section V-C), and should be weighed as correctness risk. It is not a circular derivation in the sense of a fitted input renamed as a prediction or a result forced by a self-citation chain, and the manuscript's self-disclosure and external anchoring keep it at the minor end: no significant circularity, score 2 for the minor non-load-bearing self-citation and the disclosed baseline design.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

This is a systems integration paper with no formal derivation and no fitted statistical model, so the free-parameter ledger contains engineering thresholds rather than fitted constants. The listed thresholds directly shape what reaches the LLM and what the gaze analysis reports. The axioms are the domain assumptions about perception, LLM reliability, and measurement validity that the central claims depend on; several are flagged as fragile by the authors themselves. No new physical or theoretical entities are introduced; the ARMoD robot concept and task design come from the authors' cited prior work.

free parameters (3)
  • phrase-completion pause threshold = 3 s
    Section III-B.1: speech is considered complete after 3 seconds of silence; a hand-chosen latency/robustness tradeoff that determines which utterances reach the LLM.
  • gaze filter classification threshold = 100 deg/s
    Section IV-B: standard Tobii I-VT Attention filter threshold; directly shapes the reported gaze metrics, all of which came out null.
  • LLM sampling temperature = 0
    Section III-B.2: chosen to reduce variance in generated code; part of prompt engineering rather than a quantity fit to data, but it affects the reported reliability.
assumptions (4)
  • domain assumption The proprietary LLM (GPT-4o-mini at temperature 0), given the engineered prompt, reliably produces executable Python plans that map to the robot SDK.
    Section III-B.2 relies on this for every robot action; Section V-C concedes the LLM's non-determinism is "surprisingly resistant to prompt engineering", so the assumption is acknowledged as fragile even by the authors.
  • domain assumption The perception stack (YOLO-World detection, SAM2 segmentation, BLIP-2 captions, Tobii gaze mapping) accurately grounds objects and gaze targets in the lab setting.
    Section IV-A documents failures during public dissemination (skewed gaze angles, distant small objects, hallucinated speech in noise); the lab results are interpreted assuming these do not bias the comparison.
  • domain assumption Self-reported questionnaire responses (PSSUQ usability, 7-point confidence) and the Tobii I-VT gaze filter provide valid measures of the constructs claimed.
    Section IV-B: all engagement and usability conclusions rest on these instruments, with no validation against task outcomes beyond a weak duration trend.
  • domain assumption The energy measurements (1972 Wh vs 1784 Wh) are comparable across conditions.
    Section V-A reports single point estimates with no measurement window or variance, yet energy cost is a stated part of the tradeoff conclusion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Gaze-supported Large Language Model Framework for Bi-directional Human-Robot Interaction." pith.science (2026). https://pith.science/paper/WYGR33OB

@misc{pith2026250715729,
  author       = {Pith},
  title        = {Pith review of: Gaze-supported Large Language Model Framework for Bi-directional Human-Robot Interaction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WYGR33OB}},
  note         = {Machine review of arXiv:2507.15729}
}
read the original abstract

The rapid development of Large Language Models (LLMs) creates an exciting potential for flexible, general knowledge-driven Human-Robot Interaction (HRI) systems for assistive robots. Existing HRI systems demonstrate great progress in interpreting and following user instructions, action generation, and robot task solving. On the other hand, bi-directional, multi-modal, and context-aware support of the user in collaborative tasks still remains an open challenge. In this paper, we present a gaze- and speech-informed interface to the assistive robot, which is able to perceive the working environment from multiple vision inputs and support the dynamic user in their tasks. Our system is designed to be modular and transferable to adapt to diverse tasks and robots, and it is capable of real-time use of language-based interaction state representation and fast on board perception modules. Its development was supported by multiple public dissemination events, contributing important considerations for improved robustness and user experience. Furthermore, in two lab studies, we compare the performance and user ratings of our system with those of a traditional scripted HRI pipeline. Our findings indicate that an LLM-based approach enhances adaptability and marginally improves user engagement and task execution metrics but may produce redundant output, while a scripted pipeline is well suited for more straightforward tasks.

Figures

Figures reproduced from arXiv: 2507.15729 by the authors.

Figure 1
Figure 1. Bi-directional gaze- and speech-supported commu [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our framework operates as a task-oriented interaction [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Overview of the framework architecture. The system includes an input fusion module (purple) that integrates audio [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The prompt of the reasoning module with a modular [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Dialogue between a user and the robot mediated by [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Experiment setup. In steps I., II., IV. and VI. the [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 8
Figure 8. Figure 8: Summary of the gaze metrics (means and standard deviations). [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

30 extracted references · 24 canonical work pages

  1. [4]

    Evaluating Efficiency and Engagement in Scripted and LLM-Enhanced Human-Robot Interactions

    T. Schreiter, J. V . R ¨uppel, R. Hazra, A. Rudenko, M. Magnus- son, and A. J. Lilienthal, “Evaluating Efficiency and Engagement in Scripted and LLM-Enhanced Human-Robot Interactions,” arXiv preprint arXiv:2501.12128, 2025

    work page Pith review arXiv 2025

  2. [1]

    Closed-loop open-vocabulary mobile manipulation with gpt-4v,

    P. Zhi, Z. Zhang, M. Han, Z. Zhang, Z. Li, Z. Jiao, B. Jia, and S. Huang, “Closed-loop open-vocabulary mobile manipulation with gpt-4v,” arXiv preprint arXiv:2404.10220 , 2024

    arXiv 2024

  3. [2]

    LaMI: Large language models for multi-modal human-robot interaction,

    C. Wang, S. Hasler, D. Tanneberg, F. Ocker, F. Joublin, A. Ceravola, J. Deigmoeller, and M. Gienger, “LaMI: Large language models for multi-modal human-robot interaction,” in Extended Abstracts of the CHI Conf. on Human Factors in Computing Systems , 2024

    work page 2024

  4. [3]

    Comparing apples to oranges: LLM-powered multimodal intention prediction in an object catego- rization task,

    H. Ali, P. Allgeuer, and S. Wermter, “Comparing apples to oranges: LLM-powered multimodal intention prediction in an object catego- rization task,” arXiv preprint arXiv:2404.08424 , 2024

    arXiv 2024

  5. [5]

    Anticipatory robot control for efficient human-robot collaboration,

    C.-M. Huang and B. Mutlu, “Anticipatory robot control for efficient human-robot collaboration,” in 2016 11th ACM/IEEE international conference on human-robot interaction (HRI) . IEEE, 2016

    work page 2016

  6. [6]

    The Effect of Anthropomorphism on Trust in an Industrial Human-Robot Interaction,

    T. Schreiter, L. Morillo-Mendez, R. T. Chadalavada, A. Rudenko, E. A. Billing, and A. J. Lilienthal, “The Effect of Anthropomorphism on Trust in an Industrial Human-Robot Interaction,” 2022 IEEE Int. Conf. on Robot and Human Interactive Communication (RO-MAN) , 2022

    work page 2022

  7. [7]

    Advan- tages of Multimodal versus Verbal-Only Robot-to-Human Communi- cation with an Anthropomorphic Robotic Mock Driver,

    T. Schreiter, L. Morillo-Mendez, R. T. Chadalavada, A. Rudenko, E. Billing, M. Magnusson, K. O. Arras, and A. J. Lilienthal, “Advan- tages of Multimodal versus Verbal-Only Robot-to-Human Communi- cation with an Anthropomorphic Robotic Mock Driver,” in 2023 Int. Conf. on Robot and Human Interactive Communication (RO-MAN) . IEEE, 2023

    work page 2023

  8. [8]

    TH ¨OR-MAGNI: A large-scale indoor motion capture recording of human movement and robot interaction,

    T. Schreiter, T. Rodrigues de Almeida, Y . Zhu, E. Gutierrez Maestro, L. Morillo-Mendez, A. Rudenko, L. Palmieri, T. P. Kucner, M. Mag- nusson, and A. J. Lilienthal, “TH ¨OR-MAGNI: A large-scale indoor motion capture recording of human movement and robot interaction,” The International Journal of Robotics Research , 2024

    work page 2024

Show all 30 references
  1. [9]

    Leveraging Large Language Models in Human-Robot Interaction: A Critical Analysis of Potential and Pitfalls,

    J. Atuhurra, “Leveraging Large Language Models in Human-Robot Interaction: A Critical Analysis of Potential and Pitfalls,” arXiv preprint arXiv:2405.00693, 2024

    2024 arXiv

  2. [10]

    Chatgpt for robotics: Design principles and model abilities,

    S. H. Vemprala, R. Bonatti, A. Bucker, and A. Kapoor, “Chatgpt for robotics: Design principles and model abilities,” Ieee Access, 2024

    2024

  3. [11]

    To help or not to help: LLM-based attentive support for human-robot group interactions,

    D. Tanneberg, F. Ocker, S. Hasler, J. Deigmoeller, A. Belardinelli, C. Wang, H. Wersing, B. Sendhoff, and M. Gienger, “To help or not to help: LLM-based attentive support for human-robot group interactions,” in 2024 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS)....

    2024

  4. [12]

    How to Communicate Robot Motion Intent: A Scoping Review,

    M. Pascher, U. Gruenefeld, S. Schneegass, and J. Gerken, “How to Communicate Robot Motion Intent: A Scoping Review,” in Proc. of the 2023 CHI Conf. on Human Factors in Computing Systems , 2023

    2023

  5. [13]

    Bi-directional navigation intent communication us- ing spatial augmented reality and eye-tracking glasses for improved safety in human–robot interaction,

    R. T. Chadalavada, H. Andreasson, M. Schindler, R. Palm, and A. J. Lilienthal, “Bi-directional navigation intent communication us- ing spatial augmented reality and eye-tracking glasses for improved safety in human–robot interaction,” Robotics and Computer-Integrated Manufactu...

    2020

  6. [14]

    When robots get chatty: Grounding multimodal human-robot conversation and collaboration,

    P. Allgeuer, H. Ali, and S. Wermter, “When robots get chatty: Grounding multimodal human-robot conversation and collaboration,” in International Conf. on Artificial Neural Networks . Springer, 2024

    2024

  7. [15]

    The need for verbal robot explanations and how people would like a robot to explain itself,

    Z. Han, E. Phillips, and H. A. Yanco, “The need for verbal robot explanations and how people would like a robot to explain itself,” ACM Trans. on Human-Robot Interaction (THRI), vol. 10, no. 4, 2021

    2021

  8. [16]

    Spoken language interaction with robots: Recommendations for fu- ture research,

    M. Marge, C. Espy-Wilson, N. G. Ward, A. Alwan, Y . Artzi, M. Bansal, G. Blankenship, J. Chai, H. Daum ´e III, D. Dey et al. , “Spoken language interaction with robots: Recommendations for fu- ture research,” Computer Speech & Language , vol. 71, 2022

    2022

  9. [17]

    Human Gaze and Head Rotation during Navigation, Exploration and Object Manipulation in Shared Environments with Robots,

    T. Schreiter, A. Rudenko, M. Magnusson, and A. J. Lilienthal, “Human Gaze and Head Rotation during Navigation, Exploration and Object Manipulation in Shared Environments with Robots,” arXiv preprint arXiv:2406.06300, 2024

    2024 arXiv

  10. [18]

    Meet me where i’m gazing: how shared attention gaze affects human-robot handover timing,

    A. Moon, D. M. Troniak, B. Gleeson, M. K. Pan, M. Zheng, B. A. Blumer, K. MacLean, and E. A. Croft, “Meet me where i’m gazing: how shared attention gaze affects human-robot handover timing,” in Proc. of the 2014 ACM/IEEE Int. Conf. on Human-robot interaction , 2014

    2014

  11. [19]

    Perceptive Recommendation Robot: Enhancing Receptivity of Product Suggestions Based on Customers’ Nonverbal Cues,

    M. Iwasaki, A. Yamazaki, K. Yamazaki, Y . Miyazaki, T. Kawamura, and H. Nakanishi, “Perceptive Recommendation Robot: Enhancing Receptivity of Product Suggestions Based on Customers’ Nonverbal Cues,” Biomimetics, vol. 9, no. 7, 2024

    2024

  12. [20]

    Using human eye gaze patterns as indicators of need for assistance from a socially assistive robot,

    U. Kurylo and J. R. Wilson, “Using human eye gaze patterns as indicators of need for assistance from a socially assistive robot,” in Social Robotics: 11th International Conf., ICSR 2019, Madrid, Spain, November 26–29, 2019 . Springer, 2019

    2019

  13. [21]

    Enhancing ai interpretation and decision-making: Integrating cognitive computational models with deep learning for advanced uncertain reasoning systems,

    F. A. Alijoyo, S. Janani, K. Santosh, S. N. Shweihat, N. Alshammry, J. V . N. Ramesh, and Y . A. B. El-Ebiary, “Enhancing ai interpretation and decision-making: Integrating cognitive computational models with deep learning for advanced uncertain reasoning systems,” Alexandria ...

    2024

  14. [22]

    To- ward human-aware robot task planning

    R. Alami, A. Clodic, V . Montreuil, E. A. Sisbot, and R. Chatila, “To- ward human-aware robot task planning.” in AAAI spring symposium: to boldly go where no human-robot team has gone before , 2006

    2006

  15. [23]

    Fam- hri: Foundation-model assisted multi-modal human-robot interaction combining gaze and speech,

    Y . Lai, S. Yuan, B. Zhang, B. Kiefer, P. Li, and A. Zell, “Fam- hri: Foundation-model assisted multi-modal human-robot interaction combining gaze and speech,” arXiv preprint arXiv:2503.16492, 2025

    2025 arXiv

  16. [24]

    Semanticscanpath: Combining gaze and speech for situated human-robot interaction using llms,

    E. Menendez, M. Gienger, S. Mart ´ınez, C. Balaguer, and A. Belar- dinelli, “Semanticscanpath: Combining gaze and speech for situated human-robot interaction using llms,” arXiv preprint arXiv:2503.16548, 2025

    2025 arXiv

  17. [25]

    Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation,

    J. Li, D. Li, C. Xiong, and S. Hoi, “Blip: Bootstrapping language- image pre-training for unified vision-language understanding and generation,” in Int. Conf. on Machine Learning . PMLR, 2022

    2022

  18. [26]

    Yolo- world: Real-time open-vocabulary object detection,

    T. Cheng, L. Song, Y . Ge, W. Liu, X. Wang, and Y . Shan, “Yolo- world: Real-time open-vocabulary object detection,” in Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition , 2024

    2024

  19. [27]

    Sam 2: Segment anything in images and videos,

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. R ¨adle, C. Rolland, L. Gustafson et al., “Sam 2: Segment anything in images and videos,” arXiv preprint arXiv:2408.00714 , 2024

    2024 arXiv

  20. [28]

    Robust speech recognition via large-scale weak super- vision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak super- vision,” in International conference on machine learning . PMLR, 2023, pp. 28 492–28 518

    2023

  21. [29]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhou et al. , “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems, vol. 35, 2022

    2022

  22. [30]

    Perceived usability evaluation of edu- cational technology using the post-study system usability questionnaire (pssuq): a systematic review,

    P. Vlachogianni and N. Tselios, “Perceived usability evaluation of edu- cational technology using the post-study system usability questionnaire (pssuq): a systematic review,” Sustainability, vol. 15, no. 17, 2023

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.