pith. sign in

arxiv: 2606.18189 · v1 · pith:HSGDURSPnew · submitted 2026-06-16 · 💻 cs.RO

Beyond Failure Recovery: An Engagement-Aware Human-in-the-loop Framework for Robotic Systems

Jiaying Fang , Joyce Yang , Zhanxin Wu , Bohan Yang , Tapomayukh Bhattacharjee This is my paper

Pith reviewed 2026-06-27 00:21 UTC · model grok-4.3

classification 💻 cs.RO
keywords human-robot interactionmodel predictive controluser engagementhuman-in-the-looprobotic caregivingworkload constraintsbite acquisition
0
0 comments X

The pith

Engagement-aware MPC plans interactions proactively to hold user engagement at a target level while respecting workload limits.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Conventional human-in-the-loop robotics calls on people only after failures occur, which can leave users disengaged during long stretches of autonomous operation. The paper introduces E-MPC, which instead uses a model of how engagement changes with the frequency and type of interaction to decide when and how to involve the user throughout a task. The controller optimizes interaction choices inside a model-predictive loop so that engagement stays near a user-specified target without violating workload bounds. Evaluation in simulation across user personas and a real-world study on a bite-acquisition robot with emulated mobility limits shows maintained task success alongside improved reported experience.

Core claim

E-MPC is a model-predictive controller that incorporates an explicit user-interaction dynamics model; the model predicts how engagement evolves as a function of interaction frequency and type, and the planner selects interaction actions at each step to drive predicted engagement toward a desired setpoint while enforcing a workload constraint and preserving task completion.

What carries the argument

Engagement-aware Model Predictive Control (E-MPC) that embeds a dynamics model of engagement evolution driven by interaction frequency and type.

If this is right

  • Robots can schedule user input at regular intervals rather than only on failure, keeping users involved in decision-making for the entire task duration.
  • Task performance metrics remain comparable to failure-triggered baselines while subjective engagement and workload scores improve.
  • The same planning structure can be tuned to different target engagement setpoints for users who prefer more or less involvement.
  • In physical caregiving, users with mobility constraints avoid long passive intervals without incurring extra fatigue from constant requests.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method could be tested in domains outside caregiving, such as shared-control manufacturing or navigation assistance, where sustained attention matters.
  • Learning the engagement dynamics parameters online from individual users would allow the planner to adapt without a pre-specified model.
  • Adding explicit uncertainty bounds on the engagement predictions would let the controller trade off engagement risk against task risk in a more principled way.

Load-bearing premise

The dynamics model that maps interaction frequency and type to future engagement levels is accurate enough for the optimizer to reliably reach the target engagement without violating workload limits.

What would settle it

A follow-up study that measures actual engagement trajectories under E-MPC plans and finds statistically significant, systematic deviations from the model's predictions that cause either sustained disengagement or workload violations.

Figures

Figures reproduced from arXiv: 2606.18189 by Bohan Yang, Jiaying Fang, Joyce Yang, Tapomayukh Bhattacharjee, Zhanxin Wu.

Figure 1
Figure 1. Figure 1: We propose Engagement-aware MPC (E-MPC), a human-in-the-loop framework for robotic systems that explicitly models user engagement. E-MPC involves users in the robot’s decision-making not only when user’s assistance is needed, but also when interaction is desired to sustain user’s preferred engagement level throughout the task. Abstract—Conventional human-in-the-loop approaches typically involve users only … view at source ↗
Figure 2
Figure 2. Figure 2: E-MPC framework. E-MPC proactively issues user queries to regulate engagement gt toward a user-specified desired level gdes throughout task execution. By predicting the engagement effect of different query types, E-MPC selects interactions that provide engagement boosts while keeping user workload wt bounded. When task success is at risk (low confidence C(ϕ i t , ot) and retries exhausted), the controller … view at source ↗
Figure 3
Figure 3. Figure 3: Simulation Results. (a) Performance of methods across varying pre-query skill success rates and user personas. Results are averaged over workload thresholds τw ∈ {0.25, 0.5, 0.75, 1.0}. For WorkloadAware baseline, we set the value of the hyperparameter γscale as the value that maximizes the Satisfaction metrics in each setting. (b) Performance of methods across varying workload thresholds τw with high-enga… view at source ↗
Figure 4
Figure 4. Figure 4: Effect of “Fake” Queries and Heuristic Baselines. (a) Satisfaction for three personas under E-MPC and its No-Fake-Queries ablation. (b) Satisfaction for three personas under E-MPC and two heuristic baselines: Random and Periodic. Results are averaged over pre-query skill success rates psuccess ∈ {0.6, 0.7, 0.8, 0.9, 1.0} and workload thresholds τw ∈ {0.25, 0.5, 0.75, 1.0}. ∗ indicates statistical significa… view at source ↗
Figure 5
Figure 5. Figure 5: Study Setup. (left) User study setup. (middle) Plates used in the study: a Thanksgiving plate and a fruit salad plate. (right) Participants in the user study with emulated mobility limitations using resistance bands (faces shown with full permission). Ask Draw 𝑔𝑑𝑒𝑠 𝑔𝑑𝑒𝑠 𝑔𝑑𝑒𝑠 𝑔𝑑𝑒𝑠 𝑔𝑑𝑒𝑠 𝑔𝑑𝑒𝑠 Ask Draw Ask MCQ Lower Engagement Ask MCQ Lower Engagement No Query No Query (a) (b) [PITH_FULL_IMAGE:figures/full_fi… view at source ↗
Figure 6
Figure 6. Figure 6: User Study Results. (a) Visualization of a rollout from the user study. The user initially set the gdes = 0.8. During the process, the user lowered gdes twice. E-MPC adapts to the updated gdes. (b) Participants ratings on four subjective metrics. * indicates statistical significance (p < 0.05), determined via a Mann-Whitney U test. help the execution of the skill. During each study session, the robot attem… view at source ↗
read the original abstract

Conventional human-in-the-loop approaches typically involve users only when a robot encounters failure or uncertainty, treating humans primarily as tools for improving robot performance. However, in many human-centered robotics settings, interaction should support engagement by keeping users involved in decision-making rather than limiting them to failure-driven interventions. This is particularly compelling in physical caregiving, where mobility limitations can reduce users' ability to intervene or modulate the robot's behavior in the moment. As a result, failure-driven interaction policies may relegate users to passive observers for long stretches of the task. For example, a user with mobility limitations may feel less engaged when being continuously and passively fed by a robot. At the same time, overly frequent interaction can be tiring and increase the user's workload. To address this trade-off, we propose Engagement-aware MPC (E-MPC), a user-engagement-aware method that plans interaction to maintain engagement while respecting a workload constraint. E-MPC leverages a user interaction dynamics model that captures how user engagement evolves as a function of both the frequency and type of interaction. Rather than requesting input only when difficulties arise during task execution, the robot proactively considers the user's preferred level of engagement throughout the task, balancing autonomy and interaction while ensuring task success. We evaluate E-MPC in simulation with several ablations and baseline comparisons. Results demonstrate the effectiveness of our approach across diverse user personas. In addition, we conduct a real-world user study with participants with emulated mobility limitations on a robot-assisted bite acquisition system, showing that E-MPC improves user experience while maintaining task success.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes Engagement-aware MPC (E-MPC), an extension of standard MPC for human-in-the-loop robotic systems (e.g., caregiving for mobility-limited users). E-MPC uses a user interaction dynamics model that maps interaction frequency and type to evolving engagement levels; the planner proactively selects interactions to maintain a target engagement while respecting workload constraints and preserving task success. Evaluation consists of simulation ablations and baseline comparisons across user personas plus a real-world user study on a robot-assisted bite acquisition task with emulated mobility limitations, claiming improved user experience without loss of task performance.

Significance. If the engagement dynamics model is shown to be accurate and generalizable, the work offers a principled shift from reactive failure-recovery to proactive engagement maintenance in HRI. This could matter for assistive robotics where prolonged passivity harms user well-being; the combination of MPC with an explicit engagement state and the dual simulation-plus-real-user evaluation would be a concrete contribution if the model holds.

major comments (2)
  1. [Abstract / User Interaction Dynamics Model section] The central claim that proactive engagement planning improves UX rests on the accuracy of the user interaction dynamics model (Abstract). The manuscript must demonstrate that this model was derived or validated independently of the final user-study outcomes; otherwise the reported UX gains cannot be attributed to the engagement-aware component rather than other MPC elements or study design.
  2. [Evaluation section] Simulation ablations and the real-world study are presented as evidence across personas, yet the abstract supplies no quantitative metrics (e.g., engagement scores, workload ratings, statistical tests, sample size). Without these, it is impossible to judge whether the workload constraint is actually satisfied or whether task success is maintained at a level comparable to baselines.
minor comments (2)
  1. [Methods] Clarify whether the engagement dynamics model contains any free parameters fitted to the same user-study data used for final evaluation.
  2. [Abstract] The abstract states that E-MPC 'maintains task success'; explicit comparison tables or figures showing success rates versus baselines would strengthen this.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on the validation of the engagement model and the need for quantitative metrics in the abstract. We address each major comment below and will incorporate revisions where appropriate to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Abstract / User Interaction Dynamics Model section] The central claim that proactive engagement planning improves UX rests on the accuracy of the user interaction dynamics model (Abstract). The manuscript must demonstrate that this model was derived or validated independently of the final user-study outcomes; otherwise the reported UX gains cannot be attributed to the engagement-aware component rather than other MPC elements or study design.

    Authors: The user interaction dynamics model is grounded in prior HRI literature on engagement and workload, with its structure and parameters initially specified and ablated in simulation studies performed independently of the real-world user study. The simulation results across personas serve as the primary validation of the model's predictive accuracy for engagement evolution. The user study evaluates the end-to-end E-MPC framework rather than fitting the model itself. To address the concern directly, we will add an explicit subsection detailing the literature basis, simulation-based derivation, and pre-study validation steps so that readers can clearly separate model development from the final evaluation. revision: yes

  2. Referee: [Evaluation section] Simulation ablations and the real-world study are presented as evidence across personas, yet the abstract supplies no quantitative metrics (e.g., engagement scores, workload ratings, statistical tests, sample size). Without these, it is impossible to judge whether the workload constraint is actually satisfied or whether task success is maintained at a level comparable to baselines.

    Authors: We agree that the abstract would benefit from quantitative results to allow readers to assess the claims at a glance. We will revise the abstract to include key metrics from both the simulation (e.g., engagement maintenance rates, workload constraint satisfaction percentages) and the user study (e.g., mean engagement scores, NASA-TLX workload ratings, task success rates, statistical comparisons to baselines, and participant sample size). revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation relies on independent dynamics model and external evaluations

full rationale

The paper's central claim rests on proposing E-MPC that uses a user interaction dynamics model to evolve engagement based on interaction frequency and type, then optimizes under workload constraints. This model is introduced as a modeling choice rather than derived from the target result, and effectiveness is shown via simulation ablations plus a real-user study with emulated mobility limitations. No equations or claims reduce the predictions to fitted inputs by construction, no self-citation chains justify uniqueness, and no ansatz or renaming is smuggled in. The load-bearing element is the model's accuracy (an empirical assumption), not a definitional loop.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review; no explicit free parameters, axioms, or invented entities are detailed beyond the existence of an engagement dynamics model and workload constraint.

pith-pipeline@v0.9.1-grok · 5826 in / 1075 out tokens · 35847 ms · 2026-06-27T00:21:54.171251+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 2 canonical work pages

  1. [1]

    To ask or not to ask: Human-in-the-loop contextual bandits with applica- tions in robot-assisted feeding

    Rohan Banerjee, Rajat Kumar Jenamani, Sidharth Va- sudev, Amal Nanavati, Katherine Dimitropoulou, Sarah Dean, and Tapomayukh Bhattacharjee. To ask or not to ask: Human-in-the-loop contextual bandits with applica- tions in robot-assisted feeding. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 1378–1384. IEEE, 2025

    2025

  2. [2]

    A human-in-the-loop confidence- aware failure recovery framework for modular robot policies

    Rohan Banerjee, Krishna Palempalli, Bohan Yang, Jiaying Fang, Alif Abdullah, Tom Silver, Sarah Dean, and Tapo- mayukh Bhattacharjee. A human-in-the-loop confidence- aware failure recovery framework for modular robot policies. InProceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2026

    2026

  3. [3]

    Is more autonomy always better? exploring preferences of users with mobility impairments in robot-assisted feeding

    Tapomayukh Bhattacharjee, Ethan K Gordon, Rosario Scalise, Maria E Cabrera, Anat Caspi, Maya Cakmak, and Siddhartha S Srinivasa. Is more autonomy always better? exploring preferences of users with mobility impairments in robot-assisted feeding. InProceedings of the 2020 ACM/IEEE international conference on human- robot interaction, pages 181–190, 2020

    2020

  4. [4]

    Laura Esther Bijkerk, Anke Oenema, Nicole Geschwind, and Mark Spigt. Measuring engagement with mental health and behavior change interventions: an integrative review of methods and instruments.International journal of behavioral medicine, 30(2):155–166, 2023

    2023

  5. [5]

    Assessing student engagement from facial behavior in on-line learning.Multimedia Tools and Applications, 82(9):12859– 12877, 2023

    Paolo Buono, Berardina De Carolis, Francesca D’Errico, Nicola Macchiarulo, and Giuseppe Palestra. Assessing student engagement from facial behavior in on-line learning.Multimedia Tools and Applications, 82(9):12859– 12877, 2023

    2023

  6. [6]

    Sam 3: Segment anything with concepts, 2025

    Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoub- hik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, An- drew Huang, Jie Lei, Tengyu Ma, Baishan Guo, Arpit Kalla, Markus Marks, Joseph Greer, Meng Wang, Peize Sun, Roman R ¨adle, Triantafyllos Afouras, Effrosyni Mavroudi, Katherine Xu, Tsung-Han Wu, Yu Zhou, Li...

    Pith/arXiv arXiv 2025

  7. [7]

    The sense of agency in assistive robotics using shared autonomy

    Maggie A Collier, Rithika Narayan, and Henny Admoni. The sense of agency in assistive robotics using shared autonomy. In2025 20th ACM/IEEE International Confer- ence on Human-Robot Interaction (HRI), pages 880–888. IEEE, 2025

    2025

  8. [8]

    The sense of agency in emerging technologies for human–computer integration: A review

    Patricia Cornelio, Patrick Haggard, Kasper Hornbaek, Orestis Georgiou, Joanna Bergstr¨om, Sriram Subramanian, and Marianna Obrist. The sense of agency in emerging technologies for human–computer integration: A review. Frontiers in Neuroscience, 16:949138, 2022

    2022

  9. [9]

    Learning on the job: Long-term behavioural adaptation in human-robot interactions.IEEE Robotics and Automation Letters, 7 (3):6934–6941, 2022

    Francesco Del Duchetto and Marc Hanheide. Learning on the job: Long-term behavioural adaptation in human-robot interactions.IEEE Robotics and Automation Letters, 7 (3):6934–6941, 2022

    2022

  10. [10]

    Are you still with me? continuous engagement assessment from a robot’s point of view.Frontiers in Robotics and AI, 7:116, 2020

    Francesco Del Duchetto, Paul Baxter, and Marc Hanheide. Are you still with me? continuous engagement assessment from a robot’s point of view.Frontiers in Robotics and AI, 7:116, 2020

    2020

  11. [11]

    Engagement in hci: conception, theory and measurement.ACM computing surveys (CSUR), 51(5):1–39, 2018

    Kevin Doherty and Gavin Doherty. Engagement in hci: conception, theory and measurement.ACM computing surveys (CSUR), 51(5):1–39, 2018

    2018

  12. [12]

    A policy- blending formalism for shared control.The International Journal of Robotics Research, 32(7):790–805, 2013

    Anca D Dragan and Siddhartha S Srinivasa. A policy- blending formalism for shared control.The International Journal of Robotics Research, 32(7):790–805, 2013

    2013

  13. [13]

    Human autonomy and sense of agency in human-robot interaction: A systematic literature review

    Felix Glawe, Tim Schmeckel, Philipp Brauner, and Martina Ziefle. Human autonomy and sense of agency in human-robot interaction: A systematic literature review. arXiv preprint arXiv:2509.22271, 2025

    Pith/arXiv arXiv 2025

  14. [14]

    An adaptable, safe, and portable robot-assisted feeding system

    Ethan K Gordon, Rajat Kumar Jenamani, Amal Nanavati, Ziang Liu, Daniel Stabile, Xilai Dai, Tapomayukh Bhat- tacharjee, Tyler Schrenk, Jonathan Ko, Haya Bolotski, et al. An adaptable, safe, and portable robot-assisted feeding system. InCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, pages 74–76, 2024

    2024

  15. [15]

    Glas, Iolanda Leite, and Kheng Lee Koay

    Bahar Irfan, Aditi Ramachandran, Samuel Spaulding, Dylan F. Glas, Iolanda Leite, and Kheng Lee Koay. Personalization in long-term human-robot interaction. In2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 685–686, 2019. doi: 10.1109/HRI.2019.8673076

    work page doi:10.1109/hri.2019.8673076 2019

  16. [16]

    Shared autonomy via hindsight optimization

    Shervin Javdani, Siddhartha Srinivasa, and Andrew Bag- nell. Shared autonomy via hindsight optimization. In Proceedings of Robotics: Science and Systems, Rome, Italy, July 2015. doi: 10.15607/RSS.2015.XI.032

    work page doi:10.15607/rss.2015.xi.032 2015

  17. [17]

    Feel the bite: Robot-assisted inside-mouth bite transfer using robust mouth perception and physical interaction-aware control

    Rajat Kumar Jenamani, Daniel Stabile, Ziang Liu, Abrar Anwar, Katherine Dimitropoulou, and Tapomayukh Bhat- tacharjee. Feel the bite: Robot-assisted inside-mouth bite transfer using robust mouth perception and physical interaction-aware control. InProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, pages 313–322, 2024

    2024

  18. [18]

    Flair: Feeding via long-horizon acquisition of realistic dishes

    Rajat Kumar Jenamani, Priya Sundaresan, Maram Sakr, Tapomayukh Bhattacharjee, and Dorsa Sadigh. Flair: Feeding via long-horizon acquisition of realistic dishes. arXiv preprint arXiv:2407.07561, 2024

    arXiv 2024

  19. [19]

    Feast: A flexible mealtime-assistance system towards in-the-wild personalization.arXiv preprint arXiv:2506.14968, 2025

    Rajat Kumar Jenamani, Tom Silver, Ben Dodson, Shiqin Tong, Anthony Song, Yuting Yang, Ziang Liu, Ben- jamin Howe, Aimee Whitneck, and Tapomayukh Bhat- tacharjee. Feast: A flexible mealtime-assistance system towards in-the-wild personalization.arXiv preprint arXiv:2506.14968, 2025

    arXiv 2025

  20. [20]

    Hg-dagger: In- teractive imitation learning with human experts

    Michael Kelly, Chelsea Sidrane, Katherine Driggs- Campbell, and Mykel J Kochenderfer. Hg-dagger: In- teractive imitation learning with human experts. In2019 International Conference on Robotics and Automation (ICRA), pages 8077–8083. IEEE, 2019

    2019

  21. [21]

    Srt-h: A hierarchical framework for autonomous surgery via language-conditioned imitation learning.Science robotics, 10(104):eadt5254, 2025

    Ji Woong Kim, Juo-Tung Chen, Pascal Hansen, Lucy Xi- aoyang Shi, Antony Goldenberg, Samuel Schmidgall, Paul Maria Scheikl, Anton Deguet, Brandon M White, De Ru Tsai, et al. Srt-h: A hierarchical framework for autonomous surgery via language-conditioned imitation learning.Science robotics, 10(104):eadt5254, 2025

    2025

  22. [22]

    User experience of human-robot long-term interac- tions

    Guy Laban, Arvid Kappas, Val Morrison, and Emily S Cross. User experience of human-robot long-term interac- tions. InProceedings of the 10th international conference on human-agent interaction, pages 287–289, 2022

    2022

  23. [23]

    Interactive task planning with language models.arXiv preprint arXiv:2310.10645, 2023

    Boyi Li, Philipp Wu, Pieter Abbeel, and Jitendra Malik. Interactive task planning with language models.arXiv preprint arXiv:2310.10645, 2023

    arXiv 2023

  24. [24]

    Robot learning on the job: Human-in- the-loop autonomy and learning during deployment.The International Journal of Robotics Research, 44(10-11): 1727–1742, 2025

    Huihan Liu, Soroush Nasiriany, Lance Zhang, Zhiyao Bao, and Yuke Zhu. Robot learning on the job: Human-in- the-loop autonomy and learning during deployment.The International Journal of Robotics Research, 44(10-11): 1727–1742, 2025

    2025

  25. [25]

    Long-term interactions with social robots: Trends, insights, and recommendations.ACM Transactions on Human-Robot Interaction, 14(3):1–42, 2025

    Kayla Matheus, Rebecca Ramnauth, Brian Scassellati, and Nicole Salomons. Long-term interactions with social robots: Trends, insights, and recommendations.ACM Transactions on Human-Robot Interaction, 14(3):1–42, 2025

    2025

  26. [26]

    Meet me where i’m gazing: how shared attention gaze affects human-robot handover timing

    AJung Moon, Daniel M Troniak, Brian Gleeson, Matthew KXJ Pan, Minhua Zheng, Benjamin A Blumer, Karon MacLean, and Elizabeth A Croft. Meet me where i’m gazing: how shared attention gaze affects human-robot handover timing. InProceedings of the 2014 ACM/IEEE international conference on Human-robot interaction, pages 334–341, 2014

    2014

  27. [27]

    Jauwairia Nasir, Barbara Bruno, Mohamed Chetouani, and Pierre Dillenbourg. What if social robots look for productive engagement? automated assessment of goal- centric engagement in learning applications.International Journal of Social Robotics, 14(1):55–71, 2022

    2022

  28. [28]

    Acceptance of remote assistive robots with and without human-in-the-loop for healthcare applications.International Journal of Social Robotics, 16(6):1131–1150, 2024

    Simone Nertinger, Robin Jeanne Kirschner, Abdeldjallil Naceri, and Sami Haddadin. Acceptance of remote assistive robots with and without human-in-the-loop for healthcare applications.International Journal of Social Robotics, 16(6):1131–1150, 2024

    2024

  29. [29]

    Heather L O’Brien and Elaine G Toms. What is user engagement? a conceptual framework for defining user engagement with technology.Journal of the American society for Information Science and Technology, 59(6): 938–955, 2008

    2008

  30. [30]

    Engagement in human-agent interaction: An overview.Frontiers in Robotics and AI, 7:92, 2020

    Catharine Oertel, Ginevra Castellano, Mohamed Chetouani, Jauwairia Nasir, Mohammad Obaid, Catherine Pelachaud, and Christopher Peters. Engagement in human-agent interaction: An overview.Frontiers in Robotics and AI, 7:92, 2020

    2020

  31. [31]

    Robots that ask for help: Uncertainty alignment for large language model planners.arXiv preprint arXiv:2307.01928, 2023

    Allen Z Ren, Anushri Dixit, Alexandra Bodrova, Sumeet Singh, Stephen Tu, Noah Brown, Peng Xu, Leila Takayama, Fei Xia, Jake Varley, et al. Robots that ask for help: Uncertainty alignment for large language model planners.arXiv preprint arXiv:2307.01928, 2023

    arXiv 2023

  32. [32]

    Recognizing engagement in human- robot interaction

    Charles Rich, Brett Ponsler, Aaron Holroyd, and Can- dace L Sidner. Recognizing engagement in human- robot interaction. In2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 375–382. IEEE, 2010

    2010

  33. [33]

    A reduction of imitation learning and structured prediction to no-regret online learning

    St´ephane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. InProceedings of the fourteenth international conference on artificial intelligence and statistics, pages 627–635. JMLR Workshop and Confer- ence Proceedings, 2011

    2011

  34. [34]

    Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being.American psychologist, 55 (1):68, 2000

    Richard M Ryan and Edward L Deci. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being.American psychologist, 55 (1):68, 2000

    2000

  35. [35]

    Auto- matic analysis of affective postures and body motion to detect engagement with a game companion

    Jyotirmay Sanghvi, Ginevra Castellano, Iolanda Leite, Andr´e Pereira, Peter W McOwan, and Ana Paiva. Auto- matic analysis of affective postures and body motion to detect engagement with a game companion. InProceed- ings of the 6th international conference on Human-robot interaction, pages 305–312, 2011

    2011

  36. [36]

    Correcting robot plans with natural language feedback.arXiv preprint arXiv:2204.05186, 2022

    Pratyusha Sharma, Balakumar Sundaralingam, Valts Blukis, Chris Paxton, Tucker Hermans, Antonio Tor- ralba, Jacob Andreas, and Dieter Fox. Correcting robot plans with natural language feedback.arXiv preprint arXiv:2204.05186, 2022

    arXiv 2022

  37. [37]

    Yell at your robot: Improving on-the-fly from language corrections

    Lucy Xiaoyang Shi, Zheyuan Hu, Tony Z Zhao, Archit Sharma, Karl Pertsch, Jianlan Luo, Sergey Levine, and Chelsea Finn. Yell at your robot: Improving on-the-fly from language corrections. InRobotics: Science and Systems, 2024

    2024

  38. [38]

    From the definition to the automatic assessment of engagement in human–robot interaction: A systematic review.International Journal of Social Robotics, 16(7): 1641–1663, 2024

    Alessandra Sorrentino, Laura Fiorini, and Filippo Cavallo. From the definition to the automatic assessment of engagement in human–robot interaction: A systematic review.International Journal of Social Robotics, 16(7): 1641–1663, 2024

    2024

  39. [39]

    Sensing and handling engagement dynamics in human-robot interaction involving peripheral computing devices

    Mingfei Sun, Zhenjie Zhao, and Xiaojuan Ma. Sensing and handling engagement dynamics in human-robot interaction involving peripheral computing devices. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pages 556–567, 2017

    2017

  40. [40]

    Preserving sense of agency: User preferences for robot autonomy and user control across household tasks.arXiv preprint arXiv:2506.19202, 2025

    Claire Yang, Heer Patel, Max Kleiman-Weiner, and Maya Cakmak. Preserving sense of agency: User preferences for robot autonomy and user control across household tasks.arXiv preprint arXiv:2506.19202, 2025

    arXiv 2025

  41. [41]

    Morpheus: a multimodal one-armed robot-assisted peeling system with human users in-the-loop

    Ruolin Ye, Yifei Hu, Yuhan Anjelica Bian, Luke Kulm, and Tapomayukh Bhattacharjee. Morpheus: a multimodal one-armed robot-assisted peeling system with human users in-the-loop. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 9540–9547. IEEE, 2024

    2024

  42. [42]

    Distilling and retrieving generalizable knowledge for robot manipulation via language correc- tions

    Lihan Zha, Yuchen Cui, Li-Heng Lin, Minae Kwon, Montserrat Gonzalez Arenas, Andy Zeng, Fei Xia, and Dorsa Sadigh. Distilling and retrieving generalizable knowledge for robot manipulation via language correc- tions. In2024 IEEE international conference on robotics and automation (ICRA), pages 15172–15179. IEEE, 2024. APPENDIX A. Simulation results Across W...

    2024

  43. [43]

    [Show a demo of the hard drawing-based question] How engaged would you feel if you keep interacting with the robot by answering this type of drawing-based questions?

  44. [44]

    [Show a demo of the easy MCQ] How engaged would you feel if you keep interacting with the robot by answering this type of food shape questions?

  45. [45]

    How engaged do you want to be in this task? (Note: Your answer would change how often you are asked by the robot and the type of questions)

  46. [46]

    draw pick up points

    What is the largest workload you can accept in this task? (Note: 10% roughly corresponds to asking you one "draw pick up points" question in 5 minutes, 100% roughly corresponds to asking you "draw pick up points " questions 10 times in 5 minutes.) G. Post-Method Questions Asked in User Study

  47. [47]

    For the last method, how successful was it for keeping you engaged at your desired engagement level?

  48. [48]

    For the last method, is the workload you felt during the task acceptable?

  49. [49]

    For the last method, how satisfied are you with how successfully the robot completed the task?

  50. [50]

    For the last method, how satisfied are you with how the robot interacted with you?

  51. [51]

    Statistics of User Study Table A1 shows detailed statistics of the user study results

    For the last method, how satisfied are you with how in control/agency you felt during the task? H. Statistics of User Study Table A1 shows detailed statistics of the user study results. I. Eye-gaze Analysis The additional eye-gaze analysis is conducted using study videos by estimating head pose (using head direction as a proxy for gaze) and projecting it ...