Pith. sign in

REVIEW 4 major objections 6 minor 67 references

LLM-Driven Augmented Reality Puppeteer: Controller-Free Voice-Commanded Robot Teleoperation

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A voice-commanded AR headset can puppeteer a real robot arm without a physical controller.

desk verdict A competent system-integration demo whose abstract oversells validation; the architecture is clear and the limitations are honestly listed, but the evidence is anecdotal. read the letter →

arxiv 2502.09142 v1 pith:CMF6LHVG submitted 2025-02-13 cs.HC cs.RO

Yuchong Zhang , Bastian Orthmann , Michael C. Welle , Jonne Van Haastregt , Danica Kragic This is my paper
classification cs.HCcs.RO
keywords LLM-drivencontroller-freeARpuppeteeraugmentedrealityrobotteleoperationvoicecommandshuman-robotinteractionspeechrecognition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a way to teleoperate a robot arm without a physical controller: the user wears an augmented-reality headset, sees a virtual copy of the robot, spawns it with a hand gesture, and tells it where to go by voice. A large language model running on a local computer interprets the spoken command, the virtual arm moves to a color-coded target, and the real robot repeats the motion. The authors' central claim is that this controller-free loop makes robotic control more intuitive and accessible than the controller-based AR puppeteering approach it builds on. They support the claim with a small informal demonstration and state explicitly that a structured user study has not yet been done.

What carries the argument

The central object is the virtual-robot-as-puppet loop: a virtual robot arm in the AR headset is the shared representation linking the user to the physical arm. The mechanism carrying the argument is the command-validation pipeline — a wakeword-triggered speech recognizer, a quick syntactic check for 'move to [color]', and a round-robin pair of locally hosted language models that accept or reject the command and return a JSON target — which turns a spoken sentence into a trajectory. The local hosting choice makes the loop low-latency and keeps speech data on the operator's machine, while the AR view lets the user watch the intended motion before the real robot commits to it.

What would settle it

Run a controlled test in which speakers with varied accents issue the exact wakeword-plus-color command many times in a noisy room and record how often the physical robot reaches the correct target; if accuracy drops well below a usable threshold, the claimed validation is falsified. The paper's own Section 5.3 already reports command misinterpretations in exactly those conditions, so the result is directly checkable.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that replacing the physical controller of an AR puppeteering system with natural-language voice commands still preserves the leader-follower loop: the user's voice becomes the leader signal and the physical robot is the follower. The implemented pipeline starts with a wakeword ('Blueberry'), transcribes speech with a lightweight real-time speech-to-text model, checks for the template 'move to [color]', and then sends the candidate to two locally hosted Llama 3.2 1B language models used in round-robin to validate it and emit a JSON target. After validation the target travels over UDP to the robot, which moves its end-effector from the start pose to the selected color-coded area while the virtual arm in the headset shows the same motion. The authors report that this loop worked in a preliminary demonstration with a small group of users from their local network, and they position the system as the first controller-free LLM-driven AR puppeteering system, extending the controller-based AR puppeteering system of [48].

Load-bearing premise

The load-bearing premise is that a short, informal demonstration with a small group of the authors' local acquaintances is enough to show the system functions reliably.

Editorial extensions

If this is right

  • If the loop works as demonstrated, a person with no robotics training can move a real robot arm by saying a short command such as 'Blueberry, move to orange.'
  • The LLM validation step filters out unclear or invalid commands before anything reaches the robot, which is a safety benefit over sending raw speech straight to the controller.
  • Because speech recognition and the language models run locally, the interaction can work with low latency and without sending voice data to a remote server.
  • In its current form the system handles only single point-to-point moves to predefined color targets; the authors list multi-step motion and richer gestures as future work.
  • The paper's own limitations make clear that no structured user study has measured usability or accuracy, so these consequences are conditional on such a study.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: the LLM in this pipeline is a validator for a rigid 'move to [color]' template; a stronger test of the 'LLM-driven' claim would be to let users phrase commands freely and see whether the model can still extract the target.
  • Editorial: the same virtual-puppet architecture could be used as a rehearsal space, where users plan and verify movements in AR before the real robot executes them, extending the safety decoupling the paper already emphasizes.
  • Editorial: a direct head-to-head comparison against the controller-based AR puppeteering baseline, using the same users and tasks, would reveal whether voice control improves accessibility or merely trades a controller for a speech-recognition error rate.
  • Editorial: the two-model validation suggests a scaling path; as vocabulary grows beyond colors, the quick syntactic check would need to yield to full natural-language parsing, and the local small models might then become the bottleneck.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This manuscript describes an AR-based robot teleoperation system in which a user wearing a Meta Quest 3 headset sees a virtual Franka robot arm, can spawn the virtual robot through controller-free hand tracking, and issues voice commands of the form "Blueberry, move to [color]" that are transcribed with RealtimeSTT, validated by a simple rule and two locally hosted Llama 3.2 1B models, and sent over OSC/UDP to move both the virtual and physical robot to one of seven color-coded targets. The paper claims that a preliminary demonstration with a small group of users from the authors' local network successfully validated the system's functionality and that the physical robot precisely replicates the virtual robot's trajectory in real time. The authors explicitly acknowledge limitations: voice misinterpretations, lack of gesture-based direct manipulation, single-trajectory motion, and no comprehensive user study.

Significance. The proposed integration is technically coherent and builds transparently on the group's prior controller-based AR puppeteering system [48]; using locally hosted LLMs for command validation is a reasonable design choice for privacy and latency. The paper provides a concrete component list, configuration parameters, and an honest limitations section. However, the central evidential claim—that the system was "successfully validated"—is not supported by any measured data: no participant count, task protocol, success rate, trajectory error, latency, or comparison to the controller-based baseline is reported. As a system description the paper may be of interest to the HRI/AR community, but as an evaluation it is currently at the level of an informal demonstration. The manuscript would be suitable for publication only if the claims are scaled back to a feasibility demonstration or if a structured user study with quantitative results is added.

major comments (4)
  1. [Abstract; Section 3.2; Section 5.3] The abstract states that "a preliminary user demonstration successfully validated the system's functionality," but Section 3.2 only reports an unspecified "small group of users from the authors' local network" and Section 5.3 explicitly admits that "the system misinterpreted voice commands, especially under noisy conditions" and that "we have not yet performed a comprehensive user study with structured task design and systematic user evaluation." With no participant count, no task description, no success/failure counts, and no raw recordings, the validation claim is not reproducible and is contradicted by the paper's own limitation statement. This claim must either be removed or be replaced by actual data.
  2. [Section 3.1; Section 4.3] The paper's framing claims real-time "precisely replicating" trajectory puppeteering, but the implemented pipeline, as described in Section 4.3 and Section 3.1, only supports a single command form—"move to [color]" with a fixed set of seven colors—and the robot moves from its initial position to one predefined target position. No end-to-end latency, trajectory-tracking error, or command success rate is reported, so the words "real time" and "precisely" are not quantified. Please either provide measurements supporting these terms or soften the claims to "the virtual robot's commanded target is mirrored by the physical robot."
  3. [Section 4.2; Section 4.3] The LLM's role is smaller than the "LLM-driven" label implies. The quick-validation step in Section 4.3(a) already accepts any utterance containing "move to" plus a predefined color, and the LLM validation in Section 4.3(b) is only a fallback for commands that the rule does not catch. The paper should report how often the LLM stage actually changes the outcome, give examples of commands that are accepted by the LLM but not the rule, and specify the prompt template. Without this information, the contribution could be read as a regular expression plus a speech recognizer rather than LLM-driven natural-language control.
  4. [Section 3.2; Section 4.3] The user demonstration is described as moving to an "orange" area, but the command grammar is limited to seven preset colors (red, blue, orange, yellow, purple, green, black). This makes the demonstrated behavior a single-target color-navigation episode, not the general "manipulating its virtual counterpart" interaction suggested by the abstract. If the authors intend to claim general puppeteering, they need to show multiple trajectories, varied targets, and user-controlled virtual-robot manipulation; otherwise the contribution should be scoped as a voice-commanded target-selection system.
minor comments (6)
  1. [Section 4.1] The wakeword is written as ""Blueberry"" with doubled quotation marks; please fix the typographical formatting and state the sensitivity scale for the wake_words_sensitivity parameter.
  2. [Section 2.2] Reference [48] is referred to as "Van et al." in the text, but the reference list gives "Van Haastregt, J., Welle, M.C., Zhang, Y., Kragic, D."; please correct the citation name to "Van Haastregt et al."
  3. [Section 4.3] The "round-robin approach" for the two LLM instances is not defined; please clarify how the instances are alternated and what happens if one instance fails or times out.
  4. [Figure 3] The caption refers to "left-side sub-figures" and "right-side sub-figures," but the figure has no subfigure labels; adding labels such as (a)–(c) would make the description easier to follow.
  5. [General] No link to code, a video, or supplementary data is provided; including these would make the demonstration reproducible and would substantially strengthen the paper.
  6. [Section 5.1] Phrases such as "fostering greater confidence, precision, and engagement" describe user outcomes that were not measured; please mark these as intended effects rather than demonstrated results, or remove them.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is a system description whose claims rest on an informal demonstration, not on a derivation that reduces to its own inputs.

full rationale

This manuscript is an engineering system description with no equations, no fitted parameters, and no derivation chain that could reduce to its own inputs. The core claim is that a voice-commanded AR puppeteering system was demonstrated with a small group of users. That claim is supported only by anecdotal demonstration and is explicitly qualified in Section 5.3 ('we have not yet performed a comprehensive user study with structured task design and systematic user evaluation'), but lack of rigorous validation is a correctness/evidence concern, not circularity. The paper builds on the authors' prior controller-based puppeteering system [48], and several references are self-citations, but none of these citations is used to forbid alternatives, justify a uniqueness theorem, or define a fitted quantity as a prediction. The voice/LLM pipeline is an independent implementation described in Sections 4.1-4.3. No step in the paper exhibits the required pattern of a prediction or result being equivalent to an input by construction, a fitted parameter being renamed as a prediction, or a load-bearing assumption validated only by a self-citation. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The reported functionality depends on several unvalidated component-level assumptions: speech transcription accuracy, LLM command classification, AR tracking and hand-spawning stability, virtual-to-physical trajectory mirroring, and real-time latency. The paper provides configuration values (wake word sensitivity, model choices, color vocabulary) but no measurements for any of them.

free parameters (4)
  • Wake word sensitivity = 0.9 (RealtimeSTT parameter)
    Hand-set threshold in Section 4.1; determines whether the wakeword 'Blueberry' triggers transcription. No sensitivity analysis is provided, and it directly affects whether voice commands reach the LLM.
  • Speech recognition model = tiny.en
    Section 4.1 selects the lightweight tiny.en model for transcription; this choice trades accuracy for speed, and the paper itself notes misrecognition in challenging conditions (Section 5.3).
  • Allowed target color set = red, blue, orange, yellow, purple, green, black (tutorial only)
    Section 4.3 restricts command execution to 'move to <color>' with these values; this defines the entire command vocabulary and limits the claimed generality of LLM-driven understanding.
  • Number of LLM instances = 2 (round-robin)
    Section 4.2 runs two local Llama 3.2 1B-Instruct Q6 instances; the count is an implementation choice to manage latency and privacy, with no reported load or failure behavior.
assumptions (5)
  • domain assumption RealtimeSTT transcription is sufficiently accurate in real conditions to convert voice commands into text.
    Invoked in Sections 4.1 and 4.3; the paper itself reports misinterpretation under noise and diverse accents (Section 5.3), so this premise is load-bearing and not established.
  • domain assumption The two locally hosted Llama 3.2 1B-Instruct Q6 instances correctly classify valid, invalid, and uncertain move-commands and emit JSON.
    Section 4.3 relies on LLM validation to decide whether to execute commands; no accuracy, latency, or failure rates are reported.
  • domain assumption The virtual robot's trajectory in Meta Quest 3 maps correctly and safely to the physical Franka robot.
    Section 3.1 assumes the physical robot replicates the virtual trajectory; this mapping is inherited from prior work [48] and is not independently validated here.
  • domain assumption Hand-tracking spawning and AR alignment are stable enough for the intended interaction.
    Section 3.2 says the virtual robot is spawned alongside the user's hand via mid-air interactions; no tracking stability or alignment error data are given.
  • domain assumption The system operates in real time with acceptable latency.
    The abstract and Section 5.1 claim real-time interaction; no latency measurements are reported, though local hosting is asserted to maintain responsiveness (Section 4.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLM-Driven Augmented Reality Puppeteer: Controller-Free Voice-Commanded Robot Teleoperation." pith.science (2026). https://pith.science/paper/CMF6LHVG

@misc{pith2026250209142,
  author       = {Pith},
  title        = {Pith review of: LLM-Driven Augmented Reality Puppeteer: Controller-Free Voice-Commanded Robot Teleoperation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CMF6LHVG}},
  note         = {Machine review of arXiv:2502.09142}
}
read the original abstract

The integration of robotics and augmented reality (AR) presents transformative opportunities for advancing human-robot interaction (HRI) by improving usability, intuitiveness, and accessibility. This work introduces a controller-free, LLM-driven voice-commanded AR puppeteering system, enabling users to teleoperate a robot by manipulating its virtual counterpart in real time. By leveraging natural language processing (NLP) and AR technologies, our system -- prototyped using Meta Quest 3 -- eliminates the need for physical controllers, enhancing ease of use while minimizing potential safety risks associated with direct robot operation. A preliminary user demonstration successfully validated the system's functionality, demonstrating its potential for safer, more intuitive, and immersive robotic control.

Figures

Figures reproduced from arXiv: 2502.09142 by the authors.

Figure 1
Figure 1. An overview of our proposed LLM-driven, controller-free, voice-commanded AR robotic puppeteering system. The system operates within an AR environment, utilizing the Meta Quest 3 HMD. Users can interact seamlessly with the virtual robot, modeled identically after the real Franka robot arm using only hand gestures, while voice commands are integrated to enable intuitive control of the virtual robot, which in turn gove… view at source ↗
Figure 2
Figure 2. The overview of the controller-based baseline AR robotic puppeteering system proposed by [48]. Building on these advancements, this work introduces the LLM-driven AR Puppeteer system, a novel teleoperation framework that combines the power of LLMs and AR to enable voice-commanded, controller-free interaction with robots, as shown in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. A schematic overview of the proposed system with a user in action. The virtual Franka robot arm is rendered within the AR view (the middle one). Users can interact with the virtual robot through controller-free mid-air interaction by hands and voice commands signaled by a ZOOM H2N microphone (the left sub-figures), enabling precise manipulation of its movements (the right sub-figures). Simultaneously, the physical r… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

67 extracted references · 63 canonical work pages

  1. [48]

    Van Haastregt, J., Welle, M.C., Zhang, Y., Kragic, D.: Puppeteer your robot: Aug- mentedrealityleader-followerteleoperation.In:2024IEEE-RAS23rdInternational Conference on Humanoid Robots (Humanoids). pp. 1019–1026. IEEE (2024)

    work page 2024

  2. [1]

    Journal of endourology 21(3), 285–287 (2007)

    Albani, J.M., Lee, D.I.: Virtual reality-assisted robotic surgery simulation. Journal of endourology 21(3), 285–287 (2007)

    work page 2007

  3. [2]

    In: Pro- ceedings of the 2021 CHI conference on human factors in computing systems

    Arevalo Arboleda, S., Rücker, F., Dierks, T., Gerken, J.: Assisting manipulation and grasping in robot teleoperation with augmented reality visual cues. In: Pro- ceedings of the 2021 CHI conference on human factors in computing systems. pp. 1–14 (2021)

    work page 2021

  4. [3]

    In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

    Baytas,M.A.,Çay,D.,Zhang,Y.,Obaid,M.,Yantaç,A.E.,Fjeld,M.:Thedesignof social drones: A review of studies on autonomous flyers in inhabited environments. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. pp. 1–13 (2019)

    work page 2019

  5. [4]

    Springer handbook of robotics pp

    Breazeal, C., Dautenhahn, K., Kanda, T.: Social robotics. Springer handbook of robotics pp. 1935–1972 (2016) Title Suppressed Due to Excessive Length 13

    work page 2016

  6. [5]

    IEEE Transactions on Human-Machine Systems 48(2), 197–206 (2017)

    Brizzi, F., Peppoloni, L., Graziano, A., Di Stefano, E., Avizzano, C.A., Ruffaldi, E.: Effectsofaugmentedrealityontheperformanceofteleoperatedindustrialassembly tasks in a robotic embodiment. IEEE Transactions on Human-Machine Systems 48(2), 197–206 (2017)

    work page 2017

  7. [6]

    Journal of rehabilita- tion medicine 43(6), 493–499 (2011)

    Brütsch, K., Koenig, A., Zimmerli, L., Mérillat-Koeneke, S., Riener, R., Jäncke, L., van Hedel, H.J., Meyer-Heim, A.: Virtual reality for enhancement of robot- assisted gait training in children with central gait disorders. Journal of rehabilita- tion medicine 43(6), 493–499 (2011)

    work page 2011

  8. [7]

    Applied Sciences 10(2), 486 (2020)

    Burghardt, A., Szybicki, D., Gierlak, P., Kurc, K., Pietruś, P., Cygan, R.: Program- ming of industrial robots using virtual reality and digital twins. Applied Sciences 10(2), 486 (2020)

    work page 2020

Show all 67 references
  1. [8]

    ACM Transactions on Human-Robot Interaction (THRI)11(3), 1–19 (2022)

    Chan, W.P., Hanks, G., Sakr, M., Zhang, H., Zuo, T., Van der Loos, H.M., Croft, E.: Design and evaluation of an augmented reality head-mounted display interface for human robot teams collaborating in physically shared manufacturing tasks. ACM Transactions on Human-Robot Intera...

    2022

  2. [9]

    IEEE Robotics and Automation Letters (2024)

    Chen, L., Lei, Y., Jin, S., Zhang, Y., Zhang, L.: Rlingua: Improving reinforcement learning sample efficiency in robotic manipulations with large language models. IEEE Robotics and Automation Letters (2024)

    2024

  3. [10]

    arXiv preprint arXiv:2311.02379 (2023)

    Chu, K., Zhao, X., Weber, C., Li, M., Wermter, S.: Accelerating reinforcement learning of robotic manipulations via feedback from large language models. arXiv preprint arXiv:2311.02379 (2023)

    2023 arXiv

  4. [11]

    Journal of medical Internet research 20(7), e10410 (2018)

    Cresswell, K., Cunningham-Burley, S., Sheikh, A.: Health care robotics: qualitative exploration of key challenges and future directions. Journal of medical Internet research 20(7), e10410 (2018)

    2018

  5. [12]

    In: 2019 IEEE International Work Conference on Bioinspired Intelligence (IWOBI)

    De Franco, A., Lamon, E., Balatti, P., De Momi, E., Ajoudani, A.: An intuitive augmented reality interface for task scheduling, monitoring, and work performance improvement in human-robot collaboration. In: 2019 IEEE International Work Conference on Bioinspired Intelligence (I...

    2019

  6. [13]

    Computers & Industrial En- gineering 149, 106806 (2020)

    De Pace, F., Manuri, F., Sanna, A., Fornaro, C.: A systematic review of augmented reality interfaces for collaborative industrial robots. Computers & Industrial En- gineering 149, 106806 (2020)

    2020

  7. [14]

    Applied Sciences12(1), 135 (2021)

    Dzedzickis, A., Subači¯ ut˙ e-Žemaitien˙ e, J., Šutinys, E., Samukait˙ e-Bubnien˙ e, U., Bučinskas, V.: Advanced applications of industrial robotics: New trends and pos- sibilities. Applied Sciences12(1), 135 (2021)

    2021

  8. [15]

    Robotics and Computer-Integrated Manufacturing28(2), 227–237 (2012)

    Fang, H., Ong, S., Nee, A.: Interactive robot trajectory planning and simulation using augmented reality. Robotics and Computer-Integrated Manufacturing28(2), 227–237 (2012)

    2012

  9. [16]

    In: 2007 IEEE 10th International Confer- ence on Rehabilitation Robotics

    Frisoli, A., Borelli, L., Montagner, A., Marcheschi, S., Procopio, C., Salsedo, F., Bergamasco, M., Carboncini, M.C., Tolaini, M., Rossi, B.: Arm rehabilitation with a robotic exoskeleleton in virtual reality. In: 2007 IEEE 10th International Confer- ence on Rehabilitation Rob...

    2007

  10. [17]

    In: 2024 IEEE International Conference on Robotics and Automation (ICRA)

    Gao, J., Sarkar, B., Xia, F., Xiao, T., Wu, J., Ichter, B., Majumdar, A., Sadigh, D.: Physically grounded vision-language models for robotic manipulation. In: 2024 IEEE International Conference on Robotics and Automation (ICRA). pp. 12462– 12469. IEEE (2024)

    2024

  11. [18]

    In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Sys- tems

    Ghiringhelli, F., Guzzi, J., Di Caro, G.A., Caglioti, V., Gambardella, L.M., Giusti, A.: Interactive augmented reality for understanding and analyzing multi-robot sys- tems. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Sys- tems. pp. 1195–1201. IEEE (2014)

    2014

  12. [19]

    A Roadmap to Industry 4.0: Smart Production, Sharp Business and Sustainable Development pp

    Goel, R., Gupta, P.: Robotics and industry 4.0. A Roadmap to Industry 4.0: Smart Production, Sharp Business and Sustainable Development pp. 157–169 (2020) 14 Zhang and Orthmann et al

    2020

  13. [20]

    International journal of advanced robotic systems5(1), 1 (2008)

    Green, S.A., Billinghurst, M., Chen, X., Chase, J.G.: Human-robot collaboration: A literature review and augmented reality approach in design. International journal of advanced robotic systems5(1), 1 (2008)

    2008

  14. [21]

    In: The 21st International Conference on Artificial Reality and Telexistence, Proceedings of ICAT2011

    Hashimoto, S., Ishida, A., Inami, M., Igarashi, T.: Touchme: An augmented re- ality based remote robot manipulation. In: The 21st International Conference on Artificial Reality and Telexistence, Proceedings of ICAT2011. vol. 2 (2011)

    2011

  15. [22]

    IEEE/CAA Journal of Automatica Sinica4(4), 602–609 (2017)

    He, W., Li, Z., Chen, C.P.: A survey of human-centered intelligent robots: issues and challenges. IEEE/CAA Journal of Automatica Sinica4(4), 602–609 (2017)

    2017

  16. [23]

    In: Proceedings of the 2018 ACM/IEEE International Confer- ence on Human-Robot Interaction

    Hedayati, H., Walker, M., Szafir, D.: Improving collocated robot teleoperation with augmented reality. In: Proceedings of the 2018 ACM/IEEE International Confer- ence on Human-Robot Interaction. pp. 78–86 (2018)

    2018

  17. [24]

    Cognitive Robotics1, 58–75 (2021)

    Javaid, M., Haleem, A., Singh, R.P., Suman, R.: Substantial capabilities of robotics in enhancing industry 4.0 implementation. Cognitive Robotics1, 58–75 (2021)

    2021

  18. [25]

    In: Proceedings of the 2024 ACM/IEEE In- ternational Conference on Human-Robot Interaction

    Kim, C.Y., Lee, C.P., Mutlu, B.: Understanding large-language model (llm)- powered human-robot interaction. In: Proceedings of the 2024 ACM/IEEE In- ternational Conference on Human-Robot Interaction. pp. 371–380 (2024)

    2024

  19. [26]

    In: Proceedings of the 36th international symposium on automation and robotics in construction, ISARC

    Kyjanek, O., Al Bahar, B., Vasey, L., Wannemacher, B., Menges, A.: Implementa- tion of an augmented reality ar workflow for human robot collaboration in timber prefabrication. In: Proceedings of the 36th international symposium on automation and robotics in construction, ISARC...

    2019

  20. [27]

    Lee, J.Y., Mucksavage, P., Kerbl, D.C., Huynh, V.B., Etafy, M., McDougall, E.M.: Validation study of a virtual reality robotic simulator—role as an assessment tool? The Journal of urology187(3), 998–1002 (2012)

    2012

  21. [28]

    arXiv preprint arXiv:2308.16529 (2023)

    Lee, Y.K., Jung, Y., Kang, G., Hahn, S.: Developing social robots with empathetic non-verbal cues using large language models. arXiv preprint arXiv:2308.16529 (2023)

    2023 arXiv

  22. [29]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Li, X., Zhang, M., Geng, Y., Geng, H., Long, Y., Shen, Y., Zhang, R., Liu, J., Dong, H.: Manipllm: Embodied multimodal large language model for object-centric robotic manipulation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 18061...

    2024

  23. [30]

    In: Proceedings 2002 IEEE Interna- tional Conference on Robotics and Automation (Cat

    Marin, R., Sanz, P.J., Sánchez, J.S.: A very high level interface to teleoperate a robot via web including augmented reality. In: Proceedings 2002 IEEE Interna- tional Conference on Robotics and Automation (Cat. No. 02CH37292). vol. 3, pp. 2725–2730. IEEE (2002)

    2002

  24. [31]

    International Journal on Interactive Design and Manufacturing (IJIDeM)11, 139–153 (2017)

    Matsas, E., Vosniakos, G.C.: Design of a virtual reality training system for human– robot collaboration in manufacturing tasks. International Journal on Interactive Design and Manufacturing (IJIDeM)11, 139–153 (2017)

    2017

  25. [32]

    Procedia CIRP41, 370–375 (2016)

    Michalos, G., Karagiannis, P., Makris, S., Tokçalar, Ö., Chryssolouris, G.: Aug- mented reality (ar) applications for supporting human-robot interactive coopera- tion. Procedia CIRP41, 370–375 (2016)

    2016

  26. [33]

    Stroke 40(1), 169–174 (2009)

    Mirelman, A., Bonato, P., Deutsch, J.E.: Effects of training with a robot-virtual reality system compared with a robot alone on the gait of individuals after stroke. Stroke 40(1), 169–174 (2009)

    2009

  27. [34]

    Industrial Robot: An International Journal32(6), 505–511 (2005)

    Norberto Pires, J.: Robot-by-voice: experiments on commanding an industrial robot using the human voice. Industrial Robot: An International Journal32(6), 505–511 (2005)

    2005

  28. [35]

    Nowak, A., Zhang, Y., Romanowski, A., Fjeld, M.: Augmented reality with in- dustrial process tomography: to support complex data analysis in 3d space. In: Title Suppressed Due to Excessive Length 15 Adjunct Proceedings of the 2021 ACM International Joint Conference on Perva- s...

    2021

  29. [36]

    In: Proceedings of SOUTHEASTCON’94

    Ojha, A.K.: An application of virtual reality in rehabilitation. In: Proceedings of SOUTHEASTCON’94. pp. 4–6. IEEE (1994)

    1994

  30. [37]

    Robotics and Computer-Integrated Manufacturing71, 102167 (2021)

    Pan, Y., Chen, C., Li, D., Zhao, Z., Hong, J.: Augmented reality-based robot teleoperation system using rgb-d imaging and attitude teaching device. Robotics and Computer-Integrated Manufacturing71, 102167 (2021)

    2021

  31. [38]

    In: 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob)

    Pulikottil, T.B., Caimmi, M., D’Angelo, M.G., Biffi, E., Pellegrinelli, S., Tosatti, L.M.: A voice control system for assistive robotic arms: preliminary usability tests on patients. In: 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob)...

    2018

  32. [39]

    Communications of the ACM60(11), 68–78 (2017)

    Riek, L.D.: Healthcare robotics. Communications of the ACM60(11), 68–78 (2017)

    2017

  33. [40]

    Sensors 22(23), 9520 (2022)

    Rogowski, A.: Scenario-based programming of voice-controlled medical robotic sys- tems. Sensors 22(23), 9520 (2022)

    2022

  34. [41]

    In: 2024 IEEE Conference on Telepresence

    Sahin, M., Subramanian, K., Sahin, F.: Using augmented reality to enhance worker situational awareness in human robot interaction. In: 2024 IEEE Conference on Telepresence. pp. 217–224. IEEE (2024)

    2024

  35. [42]

    Robotics and au- tonomous systems 146, 103902 (2021)

    Sarker, S., Jamal, L., Ahmed, S.F., Irtisam, N.: Robotics and artificial intelligence in healthcare during covid-19 pandemic: A systematic review. Robotics and au- tonomous systems 146, 103902 (2021)

    2021

  36. [43]

    Journal of endourology23(3), 503–508 (2009)

    Sethi, A.S., Peine, W.J., Mohammadi, Y., Sundaram, C.P.: Validation of a novel virtual reality robotic simulator. Journal of endourology23(3), 503–508 (2009)

    2009

  37. [44]

    The Inter- national Journal of Advanced Manufacturing Technology111, 1077–1097 (2020)

    Solanes, J.E., Muñoz, A., Gracia, L., Martí, A., Girbés-Juan, V., Tornero, J.: Tele- operation of industrial robot manipulators based on augmented reality. The Inter- national Journal of Advanced Manufacturing Technology111, 1077–1097 (2020)

    2020

  38. [45]

    IEEE Robotics and Automation Letters (2024)

    Song, D., Liang, J., Payandeh, A., Raj, A.H., Xiao, X., Manocha, D.: Vlm-social- nav: Socially aware robot navigation through scoring using vision-language models. IEEE Robotics and Automation Letters (2024)

    2024

  39. [46]

    IEEE robotics & automation magazine14(1), 35–42 (2007)

    Tapus, A., Mataric, M.J., Scassellati, B.: Socially assistive robotics [grand chal- lenges of robotics]. IEEE robotics & automation magazine14(1), 35–42 (2007)

    2007

  40. [47]

    Procedia CIRP 100, 133–138 (2021)

    Togias, T., Gkournelos, C., Angelakis, P., Michalos, G., Makris, S.: Virtual reality environment for industrial robot control and path design. Procedia CIRP 100, 133–138 (2021)

    2021

  41. [49]

    Wade, E., Winstein, C.J.: Virtual reality and robotics for stroke rehabilitation: where do we go from here? Topics in stroke rehabilitation18(6), 685–700 (2011)

    2011

  42. [50]

    In: Proceedings of the 2018 ACM/IEEE International Con- ference on Human-Robot Interaction

    Walker, M., Hedayati, H., Lee, J., Szafir, D.: Communicating robot motion intent with augmented reality. In: Proceedings of the 2018 ACM/IEEE International Con- ference on Human-Robot Interaction. pp. 316–324 (2018)

    2018

  43. [51]

    In: 2019 14th ACM/IEEE International Conference on Human- Robot Interaction (HRI)

    Walker, M.E., Hedayati, H., Szafir, D.: Robot teleoperation with augmented reality virtual surrogates. In: 2019 14th ACM/IEEE International Conference on Human- Robot Interaction (HRI). pp. 202–210. IEEE (2019)

    2019

  44. [52]

    In: Proceedings of the 12th International Conference on Human-Agent Interaction

    Wallace, B., van Otterdijk, M., Zhang, Y., Rajabi, N., Marin-Bucio, D., Kragic, D., Torresen, J.: Imitation or innovation? translating features of expressive motion from humans to robots. In: Proceedings of the 12th International Conference on Human-Agent Interaction. pp. 296–...

    2024

  45. [53]

    In: Robotics Research: The 18th International Symposium ISRR

    Whitney, D., Rosen, E., Phillips, E., Konidaris, G., Tellex, S.: Comparing robot grasping teleoperation across desktop and virtual reality with ros reality. In: Robotics Research: The 18th International Symposium ISRR. pp. 335–350. Springer (2019)

    2019

  46. [54]

    Nature Communications15(1), 1760 (2024)

    Xia, H., Zhang, Y., Rajabi, N., Taleb, F., Yang, Q., Kragic, D., Li, Z.: Shaping high-performance wearable robots for human motor and sensory reconstruction and enhancement. Nature Communications15(1), 1760 (2024)

    2024

  47. [55]

    arXiv preprint arXiv:2311.07226 (2023)

    Zeng, F., Gan, W., Wang, Y., Liu, N., Yu, P.S.: Large language models for robotics: A survey. arXiv preprint arXiv:2311.07226 (2023)

    2023

  48. [56]

    Biomimetic Intelligence and Robotics p

    Zhang, C., Chen, J., Li, J., Peng, Y., Mao, Z.: Large language models for human- robot interaction: A review. Biomimetic Intelligence and Robotics p. 100131 (2023)

    2023

  49. [57]

    In: ProceedingsoftheInternationalConferenceonMobileandUbiquitousMultimedia

    Zhang, Y., Kassem, K., Gong, Z., Mo, F., Ma, Y., Kirjavainen, E., Häkkilä, J.: Human-centered ai technologies in human-robot interaction for social settings. In: ProceedingsoftheInternationalConferenceonMobileandUbiquitousMultimedia. pp. 501–505 (2024)

    2024

  50. [58]

    In: Adjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction

    Zhang, Y., Ma, Y., Kragic, D.: Vision beyond boundaries: An initial design space of domain-specific large vision models in human-robot interaction. In: Adjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction. pp. 1–8 (2024)

    2024

  51. [59]

    In: International Conference on Human-Computer Interaction

    Zhang, Y., Nowak, A., Rao, G., Romanowski, A., Fjeld, M.: Is industrial tomogra- phy ready for augmented reality? a need-finding study of how augmented reality can be adopted by industrial tomography experts. In: International Conference on Human-Computer Interaction. pp. 523–...

    2023

  52. [60]

    In: Proceedings of the 2022 International Conference on Advanced Visual Interfaces

    Zhang, Y., Nowak, A., Romanowski, A., Fjeld, M.: On-site or remote working?: An initial solution on how covid-19 pandemic may impact augmented reality users. In: Proceedings of the 2022 International Conference on Advanced Visual Interfaces. pp. 1–3 (2022)

    2022

  53. [61]

    In: 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

    Zhang, Y., Nowak, A., Xuan, Y., Romanowski, A., Fjeld, M.: See or hear? exploring the effect of visual/audio hints and gaze-assisted instant post-task feedback for visual search tasks in ar. In: 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). pp. 1113...

    2023

  54. [62]

    Sensors21(19), 6515 (2021)

    Zhang, Y., Omrani, A., Yadav, R., Fjeld, M.: Supporting visualization analysis in industrial process tomography by using augmented reality—a case study of an industrial microwave drying system. Sensors21(19), 6515 (2021)

    2021

  55. [63]

    arXiv preprint arXiv:2403.06186 (2024)

    Zhang, Y., Rajabi, N., Taleb, F., Matviienko, A., Ma, Y., Björkman, M., Kragic, D.: Mind meets robots: A review of eeg-based brain-robot interaction systems. arXiv preprint arXiv:2403.06186 (2024)

    2024 arXiv

  56. [64]

    In: International Conference on Human-Computer Interaction

    Zhang, Y., Vasco, M., Björkman, M., Kragic, D.: Will you participate? exploring the potential of robotics competitions on human-centric topics. In: International Conference on Human-Computer Interaction. pp. 240–255. Springer (2024)

    2024

  57. [65]

    In: IFIP Conference on Human-Computer Interaction

    Zhang, Y., Xuan, Y., Yadav, R., Omrani, A., Fjeld, M.: Playing with data: An augmented reality approach to interact with visualizations of industrial process tomography. In: IFIP Conference on Human-Computer Interaction. pp. 123–144. Springer (2023)

    2023

  58. [66]

    Robotics and Autonomous Systems96, 93–101 (2017)

    Zhao, Z., Huang, P., Lu, Z., Liu, Z.: Augmented reality for enhancing tele-robotic system with force feedback. Robotics and Autonomous Systems96, 93–101 (2017)

    2017

  59. [67]

    Zhong, X., Liu, P., Georganas, N.D., Boulanger, P.: Designing a vision-based col- laborative augmented reality application for industrial training. it Inf. Technol. 45(1), 7–19 (2003)

    2003

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.