Pith. sign in

REVIEW 5 major objections 5 minor 67 references

Effect of Adaptive Communication Support on LLM-powered Human-Robot Collaboration

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLM-powered robot teammates should adjust how often they speak to how hard the task is: users want frequent proactive help as tasks get harder, but when the task exceeds the LLM's ability, superactive feedback becomes noise and hurts team…

desk verdict Useful framework, but the feedback conditions are confounded with robot planning behavior, so the central claim is not yet supported. read the letter →

arxiv 2412.06808 v2 pith:WZ7C2PYE submitted 2024-11-26 cs.HC cs.AIcs.RO

Shipeng Liu , FNU Shrutika , Boshen Zhang , Zhehui Huang , Gaurav Sukhatme , Feifei Qian This is my paper · ORCID
classification cs.HCcs.AIcs.RO
keywords human-robotcollaborationlargelanguagemodelsfeedbackadaptivecommunicationtaskcomplexityuserstudyOvercookedproactiveassistance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish a design principle for how often an LLM-powered robot should talk to a human teammate. Through a user study in an Overcooked-style cooking game, the authors compare four feedback modes — inactive, passive, active, and superactive — across easy, medium, and hard layouts. They report that as tasks become harder relative to the human's skill, people prefer and benefit from more frequent, proactive robot communication. But when the task is hard enough that the LLM's own suggestions become unreliable, high-frequency feedback backfires: humans spend extra effort interpreting and responding to noisy messages, and team scores drop. If the principle holds, robot helpers should tune their communication frequency dynamically based on task complexity, human capability, and the language model's capability.

What carries the argument

The load-bearing mechanism is HRT-ML, a two-module framework that separates strategic from tactical communication. A Coordinator uses a directed acyclic graph of subtasks to hold low-frequency, high-level coordination conversations and can revise the graph structure, node attributes, or add temporary subtasks based on human input. A Manager performs high-frequency, subtask-level allocation, generating short instructions and updating graph status when an Interact action completes a subtask. Four feedback modes — Inactive, Passive, Active, and Superactive — vary only in when and how often the Manager's and Coordinator's language output reaches the human, so observed differences in score, trust, satisfaction, and preference are attributed to communication frequency and content.

What would settle it

Run the four agents on the same hard layout with a fifth condition in which the superactive agent's suggestions are filtered to be always correct and delivered at the same high rate; if team scores recover, the hard-layout penalty comes from inaccurate content, while if scores stay low, the penalty comes from communication volume itself. A complementary check is to keep content identical and vary only delivery rate.

Watch

Extended reading notes

Core claim

The central claim is that the value of language feedback in human-robot teams follows task complexity and model capability, not "more is better." The paper's data show trust and perceived intelligence rise monotonically as the robot becomes more active, yet team score does not: passive agents score highest on the easy layout, superactive agents score highest on the medium layout, and on the hard layout superactive feedback lowers performance below that of merely active feedback. The authors interpret the pattern through flow theory and cognitive load: support helps when it closes the gap between task complexity and human skill, but becomes harmful when the LLM's suggestions are inaccurate or so frequent that responding to them consumes the human's attention. They generalize this into a four-case adaptation rule comparing task complexity $T$, human capability $C_h$, and LLM capability $C_l$.

Load-bearing premise

The paper assumes that differences in score and ratings across the four agents are caused by the frequency and content of language feedback, because the same planning and action framework is shared across conditions; if the agents also differ in how subtasks are allocated or in the robot's behavior beyond what is spoken, the attribution to "feedback" would collapse.

Editorial extensions

If this is right

  • Robot teammates should not use a fixed communication policy; the preferred feedback level shifts from passive in easy tasks to active or superactive as task complexity grows relative to human skill.
  • When the task exceeds the LLM's capability, high-frequency feedback can reduce team efficiency and engagement, so less frequent or more passive communication becomes the safer default.
  • Human preference and objective team score can diverge: participants often preferred passive agents even when active agents scored higher, so satisfaction and performance need to be measured separately.
  • Trust and perceived intelligence improve monotonically with feedback activity, meaning humans can find a robot more credible and capable even when its chatter does not help them perform better.
  • The paper's adaptation rule — compare task complexity with human and LLM capability — provides a starting point for robots to select among Inactive, Passive, Active, and Superactive modes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own limitation notes that agents held a constant feedback level; a natural extension the authors leave open is real-time estimation of human cognitive load and LLM confidence to switch modes mid-task.
  • The four discrete modes imply an inverted-U relationship between feedback frequency and team performance, with the peak shifting right as difficulty increases; a continuous frequency sweep would test this directly.
  • The flow-theory explanation suggests some humans deliberately want to retain mastery in hard tasks, so an HRI system might need to let users set or signal an engagement preference rather than always maximizing score.
  • Since preference and performance diverge, designers may need separate optimization targets — objective efficiency and subjective flow — for LLM-powered teammates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes HRT-ML, a human-robot teaming framework with two LLM-based modules (Coordinator and Manager) that generate multi-modal language feedback at different frequencies and to different degrees of proactivity. The authors report a user study (n=16) in an Overcooked environment with three layout difficulties (easy, medium, hard) and four feedback modes (IFA, PFA, AFA, SFA), measuring game score, satisfaction, engagement, trust, perceived intelligence, and stated preference. The central empirical claim is that humans prefer more frequent, proactive support as task complexity increases relative to human capability, but that overly frequent feedback from an LLM whose capacity is exceeded can hurt team performance. The paper concludes with a set of qualitative conditions (Section VII) for when passive, active, or superactive feedback should be used, based on the relative magnitudes of task complexity, human capability, and LLM capability. The contribution is positioned as the first systematic study of LLM-based multimodal feedback frequency on human-robot teaming performance across task complexities.

Significance. If the central claim holds, the paper would provide a useful, actionable design principle for setting the frequency and proactivity of language feedback in LLM-powered human-robot teams, with clear relevance to HRI and human-AI collaboration. The study design is nontrivial: it introduces a flexible DAG-based coordination framework, a two-level (Coordinator/Manager) LLM architecture, and a user study with three difficulty levels and four feedback modes. The paper also ships unusually detailed prompt and survey material in the appendix, which aids reproducibility. The reported raw measurements (game scores, Likert ratings) are concrete and lend themselves to further analysis. However, the validity of the main causal claim is currently threatened by a confound between feedback mode and robot planning behavior, and by the absence of inferential statistics for the central performance comparisons. The paper's proposed adaptation principle in Section VII is plausible but is a post-hoc summary of the same experimental data rather than a validated rule.

major comments (5)
  1. [IV-E and VI-B] The four feedback conditions do not differ only in language feedback; they also differ in the robot's planning and coordination behavior. In IV-E, IFA is described as having only the Manager generate target subtasks for the greedy planner, with no coordination conversation, while AFA lets the GPT-4o Coordinator periodically suggest coordination strategies that can revise the subtask graph, add intermediate nodes, and modify target positions. Since the Manager allocates subtasks based on this graph (Section IV-C), the robot's actual action sequence and the human-robot division of labor change across conditions. Therefore, the differences in game scores in Fig. 5 and preferences in Fig. 6 cannot be attributed solely to the frequency or content of language feedback. The abstract's claim that 'noisy and inaccurate feedback' hinders performance presupposes that only feedback varies, which the current design does not establish. This confound is load-bearing and needs to be addressed, either by equating planning behavior across conditions or by adding a condition that controls for coordination behavior with feedback removed.
  2. [VI-B, Fig. 5] The paper makes strong comparative claims without reporting inferential statistics. For example, in VI-B it states that on the easy layout, PFA scored 83.6 on average and was 'significantly higher than IFA, AFA, and SFA,' but no t-test, ANOVA, nonparametric test, or multiple-comparison correction is reported anywhere in the results. Similar comparative statements about SFA outperforming IFA and PFA on the hard layout are made without supporting statistics. With n=16 participants and 12 trials each, the variability visible in the box plots makes these 'significant' claims unverifiable. The authors should either add the appropriate statistical analyses or explicitly soften the causal and comparative language to descriptive terms.
  3. [VI-C, Fig. 7] The statement that 'This strong correlation between satisfaction and engagement indicates that engagement is the cause of different satisfaction levels reported by participants' overstates what a correlation can show. A correlation coefficient of 0.93 with p=0.00275 establishes association, not causation. Moreover, satisfaction and engagement were both self-reported on Likert scales in the same post-trial survey, so shared method variance could inflate the correlation. The authors should rephrase this as a suggestive association and, if they want to support the engagement-as-cause story, use a mediation or path analysis or collect engagement as a behavioral measure.
  4. [VII] The adaptation principle in Section VII is derived entirely from the same experimental data used to reach the earlier conclusions; it is not a prediction validated on holdout data or a new experiment. For instance, the claim that when Ch < T and Cl < T a 'more passive (PFA) or relative infrequent (AFA) feedback style would result in better teaming performance' is a restatement of the hard-layout results in Fig. 5. The paper should explicitly label this section as a post-hoc interpretation rather than a tested design principle, and should state what independent evidence would be needed to validate it. Relatedly, the AFA feedback interval (~20 s) is a free parameter of the study; the sensitivity of the results to this interval is not examined.
  5. [IX and abstract] The limitations section states that robots were set to a constant active level and could not dynamically adjust their support during a task, yet the abstract and introduction frame the contribution as 'adaptive communication support' and 'dynamically adjust their levels and frequencies of communications.' The paper evaluates fixed feedback modes, not an adaptive policy. This gap between the framing and the actual experimental manipulation should be reconciled, either by renaming the contribution to 'selecting a feedback level' or by adding a real-time adaptation mechanism that is tested across changing task conditions.
minor comments (5)
  1. [Abstract and Introduction] There are several typos and grammatical slips, e.g., 'robot to adopt their roles' in the abstract, 'preceived intelligence' in Section VI-A, 'avaiable' in Appendix C, and 'onlyh be' in the Coordinator prompt. A thorough proofreading pass would improve readability.
  2. [Appendix B and Fig. 4] The survey questions in Appendix B use scales from 1 to 7, but Fig. 4 shows y-axis values from 0 to 6 and labels starting at 'Very Untrustworthy' at the bottom. Please make the scale anchors and the plotted axis range consistent across the text, appendix, and figures.
  3. [VI-B] The sentence 'Team with PFA performed best, scoring 83.6 in average' is ambiguous because Fig. 5 shows a range of scores up to 100 and the average score for the easy layout is reported earlier as 53.6 for all agent types. Clarify whether 83.6 is the mean, the median, or the mean for a subset of trials, and report the corresponding standard deviation or interquartile range.
  4. [VII] The notation T, Ch, and Cl is introduced informally in Section VII without formal definitions or thresholds. Even as a qualitative framework, the text should specify what quantities these symbols refer to and how a practitioner would estimate them, otherwise the proposed conditions are not actionable.
  5. [Related Work] The related-work section would benefit from a brief discussion of prior studies on feedback frequency or communication cost in human-robot interaction (beyond directionality), since the paper's contribution is specifically about frequency and proactivity. The current narrative focuses mostly on unidirectional vs. bidirectional communication.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claims are empirical summaries of external measurements, not derivations from their own inputs.

full rationale

The paper does not contain a derivation chain in which a predicted quantity is equivalent to its inputs by construction. The core results are game scores, Likert ratings, and preference counts collected from 16 participants across controlled trials; these measurements are external to the interpretation placed on them. The adaptation principle in Section VII is explicitly presented as "based on our results" and is a post-hoc summary of the observed patterns, not a fitted parameter renamed as a prediction and not a quantity derived from itself. No load-bearing self-citation appears: the authors' prior works [24,25] are cited only as examples of prior human-robot collaboration research and do not justify the central claim. There is no uniqueness theorem imported from the authors, no ansatz smuggled in via citation, and no renaming of a known result. The most serious concern is internal validity rather than circularity: Section IV-E defines the four agents by differences in whether the Coordinator may revise the subtask graph (e.g., IFA "Only the Manager generates target subtasks" while AFA's GPT-4o coordinator "suggest[s] coordination strategies" that can alter graph structure and thus robot planning). This means the manipulation may vary robot coordination behavior alongside language feedback, which threatens the causal attribution to feedback alone. That is a legitimate experimental-design criticism, but it is not a circularity, because the paper's measurements are not constructed to equal the conclusions. Accordingly, the circularity score is 0.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no free parameters in a formal sense. The main hand-chosen constant is the ~20s interval for AFA feedback. The framework relies on the assumed validity of the Overcooked environment, the teaming fluency metric, and GPT-4o's capabilities. No new physical entities are postulated.

free parameters (1)
  • AFA feedback interval = ~20s
    Chosen by authors as 'low frequency' for the Active Feedback Agent; not derived from data.
assumptions (4)
  • domain assumption Overcooked game score is a valid proxy for human-robot team performance.
    Used throughout Section VI to compare agent types.
  • domain assumption The teaming fluency metric (percentage of unobstructed free area) adequately differentiates task complexity.
    Appendix A; used to select easy/medium/hard layouts.
  • domain assumption The four feedback modes operationalize increasing levels of communication frequency and proactivity.
    Section IV-E; no direct measurement of message counts is reported.
  • domain assumption GPT-4o is capable of generating useful coordination and task instructions at the intended frequencies.
    The entire framework relies on GPT-4o; the paper does not verify the quality of its outputs beyond participant perceptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Effect of Adaptive Communication Support on LLM-powered Human-Robot Collaboration." pith.science (2026). https://pith.science/paper/WZ7C2PYE

@misc{pith2026241206808,
  author       = {Pith},
  title        = {Pith review of: Effect of Adaptive Communication Support on LLM-powered Human-Robot Collaboration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WZ7C2PYE}},
  note         = {Machine review of arXiv:2412.06808}
}
read the original abstract

Effective human-robot collaboration requires robot to adopt their roles and levels of support based on human needs, task requirements, and complexity. Traditional human-robot teaming often relies on a pre-determined robot communication scheme, restricting teamwork adaptability in complex tasks. Leveraging strong communication capabilities of Large Language Models (LLMs), we propose a Human-Robot Teaming Framework with Multi-Modal Language feedback (HRT-ML), a framework designed to enhance human-robot interaction by adjusting the frequency and content of language-based feedback. HRT-ML framework includes two core modules: a Coordinator for high-level, low-frequency strategic guidance, and a Manager for subtask-specific, high-frequency instructions, enabling passive and active interactions with human teammates. To assess the impact of language feedback in collaborative scenarios, we conducted experiments in an enhanced Overcooked environment with varying levels of task complexity (easy, medium, hard) and feedback frequency (inactive, passive, active, superactive). Our results show that as task complexity increases relative to human capabilities, human teammates exhibited a stronger preference towards robotic agents that can offer frequent, proactive support. However, when task complexities exceed the LLM's capacity, noisy and inaccurate feedback from superactive robotic agents can instead hinder team performance, as it requires human teammates to increase their effort to interpret and respond to a large number of communications, with limited performance return. Our results offer a general principle for robotic agents to dynamically adjust their levels and frequencies of communications to work seamlessly with humans and achieve improved teaming performance.

Figures

Figures reproduced from arXiv: 2412.06808 by the authors.

Figure 1
Figure 1. (A) Cooking process to complete an order. (B) The designed human [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Human-Robot Teaming Framework with Multi-Modal Feedback(HRT-ML). It contains two modules: the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of the human study procedure involving 16 participants, [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: Game scores of all participants paired with different robotic [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 4
Figure 4. Figure 4: Human perceived robot intelligence level (blue bar) and trust level [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: Participant preferences and satisfaction levels for different agent types [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Illustration of the relationship and variability between satisfaction [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Easy, medium, hard layouts. The red crosses represent the critical [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 49 canonical work pages

  1. [1]

    LLM- Coordination: Evaluating and Analyzing Multi-agent Co- ordination Abilities in Large Language Models

    Saaket Agashe, Yue Fan, and Xin Eric Wang. LLM- Coordination: Evaluating and Analyzing Multi-agent Co- ordination Abilities in Large Language Models. arXiv preprint arXiv:2310.03903, 2023

    arXiv 2023

  2. [2]

    Effects of communication directionality and AI agent differences in human-AI interaction

    Zahra Ashktorab, Casey Dugan, James Johnson, Qian Pan, Wei Zhang, Sadhana Kumaravel, and Murray Camp- bell. Effects of communication directionality and AI agent differences in human-AI interaction. In Proceed- ings of the 2021 CHI conference on human factors in computing systems, pages 1–15, 2021

    work page 2021

  3. [3]

    Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and pref- erences

    Erdem Bıyık, Dylan P Losey, Malayandi Palan, Nicholas C Landolfi, Gleb Shevchuk, and Dorsa Sadigh. Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and pref- erences. The International Journal of Robotics Research, 41(1):45–67, 2022

    work page 2022

  4. [4]

    Sparks of artificial general intelligence: Early experiments with gpt-4

    S ´ebastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712 , 2023

    arXiv 2023

  5. [5]

    On the utility of learning about humans for human-ai coordina- tion

    Micah Carroll, Rohin Shah, Mark K Ho, Tom Griffiths, Sanjit Seshia, Pieter Abbeel, and Anca Dragan. On the utility of learning about humans for human-ai coordina- tion. Advances in neural information processing systems, 32, 2019

    work page 2019

  6. [6]

    PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks

    Matthew Chang, Gunjan Chhablani, Alexander Clegg, Mikael Dallaire Cote, Ruta Desai, Michal Hlavac, Vladimir Karashchuk, Jacob Krantz, Roozbeh Mottaghi, Priyam Parashar, et al. PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks. arXiv preprint arXiv:2411.00081 , 2024

    arXiv 2024

  7. [7]

    Huili Chen, Sharifa Alghowinem, Cynthia Breazeal, and Hae Won Park. Integrating Flow Theory and Adaptive Robot Roles: A Conceptual Model of Dynamic Robot Role Adaptation for the Enhanced Flow Experience in Long-term Multi-person Human-Robot Interactions. In Proceedings of the 2024 ACM/IEEE International Con- ference on Human-Robot Interaction , pages 116–...

    work page 2024

  8. [8]

    Trust-aware decision making for human-robot collaboration: Model learning and plan- ning

    Min Chen, Stefanos Nikolaidis, Harold Soh, David Hsu, and Siddhartha Srinivasa. Trust-aware decision making for human-robot collaboration: Model learning and plan- ning. ACM Transactions on Human-Robot Interaction (THRI), 9(2):1–23, 2020

    work page 2020

Show all 67 references
  1. [9]

    An overview of assistive robotics and technologies for elderly care

    Eftychios G Christoforou, Andreas S Panayides, Sotiris Avgousti, Panicos Masouras, and Constantinos S Pat- tichis. An overview of assistive robotics and technologies for elderly care. In XV Mediterranean Conference on Medical and Biological Engineering and Computing– MEDICON 2...

    2019

  2. [10]

    The future of service: The power of emotion in human-robot inter- action

    Stephanie Hui-Wen Chuah and Joanne Yu. The future of service: The power of emotion in human-robot inter- action. Journal of Retailing and Consumer Services , 61: 102551, 2021

    2021

  3. [11]

    Beyond boredom and anxiety

    Mihaly Csikszentmihalyi. Beyond boredom and anxiety. Jossey-bass, 2000

    2000

  4. [12]

    Rethinking Conversational Agents in the Era of LLMs: Proactivity, Non-collaborativity, and Beyond

    Yang Deng, Wenqiang Lei, Minlie Huang, and Tat-Seng Chua. Rethinking Conversational Agents in the Era of LLMs: Proactivity, Non-collaborativity, and Beyond. In Proceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in ...

    2023

  5. [13]

    Human factors in cobot era: a review of modern production systems features

    Maurizio Faccio, Irene Granata, Alberto Menini, Mat- tia Milanese, Chiara Rossato, Matteo Bottin, Riccardo Minto, Patrik Pluchino, Luciano Gamberini, Giovanni Boschetti, et al. Human factors in cobot era: a review of modern production systems features. Journal of Intelligent M...

    2023

  6. [14]

    Openagi: When llm meets domain experts

    Yingqiang Ge, Wenyue Hua, Kai Mei, Juntao Tan, Shuyuan Xu, Zelong Li, Yongfeng Zhang, et al. Openagi: When llm meets domain experts. Advances in Neural Information Processing Systems , 36:5539–5568, 2023

    2023

  7. [15]

    Adaptive robot-assisted feeding: An online learning framework for acquiring previously unseen food items

    Ethan K Gordon, Xiang Meng, Tapomayukh Bhattachar- jee, Matt Barnes, and Siddhartha S Srinivasa. Adaptive robot-assisted feeding: An online learning framework for acquiring previously unseen food items. In 2020 IEEE/RSJ International Conference on Intelligent Robots and System...

    2020

  8. [16]

    Efficient Human-AI Coordination via Preparatory Language-based Convention

    Cong Guan, Lichao Zhang, Chunpeng Fan, Yichen Li, Feng Chen, Lihe Li, Yunjia Tian, Lei Yuan, and Yang Yu. Efficient Human-AI Coordination via Preparatory Language-based Convention. arXiv preprint arXiv:2311.00416, 2023

    2023 arXiv

  9. [17]

    Evaluating fluency in human–robot collaboration

    Guy Hoffman. Evaluating fluency in human–robot collaboration. IEEE Transactions on Human-Machine Systems, 49(3):209–218, 2019

    2019

  10. [18]

    Metagpt: Meta programming for multi-agent collaborative frame- work

    Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta programming for multi-agent collaborative frame- work. arXiv preprint arXiv:2308.00352 , 2023

    2023 arXiv

  11. [19]

    ” My agent understands me better”: Integrating Dynamic Human-like Memory Recall and Consolidation in LLM- Based Agents

    Yuki Hou, Haruki Tamoto, and Homei Miyashita. ” My agent understands me better”: Integrating Dynamic Human-like Memory Recall and Consolidation in LLM- Based Agents. In Extended Abstracts of the CHI Con- ference on Human Factors in Computing Systems , pages 1–7, 2024

    2024

  12. [20]

    Human-robot interactions in manufacturing: A survey of human behavior modeling

    Roohollah Jahanmahin, Sara Masoud, Jeremy Rickli, and Ana Djuric. Human-robot interactions in manufacturing: A survey of human behavior modeling. Robotics and Computer-Integrated Manufacturing, 78:102404, 2022

    2022

  13. [21]

    Cognitive burden of survey questions and response times: A psycholinguistic experiment

    Timo Lenzner, Lars Kaczmirek, and Alwine Lenzner. Cognitive burden of survey questions and response times: A psycholinguistic experiment. Applied cognitive psy- chology, 24(7):1003–1020, 2010

    2010

  14. [22]

    CoNav: A Benchmark for Human- Centered Collaborative Navigation

    Changhao Li, Xinyu Sun, Peihao Chen, Jugang Fan, Zixu Wang, Yanxia Liu, Jinhui Zhu, Chuang Gan, and Mingkui Tan. CoNav: A Benchmark for Human- Centered Collaborative Navigation. arXiv preprint arXiv:2406.02425, 2024

    2024 arXiv

  15. [23]

    Llm-powered hierarchi- cal language agent for real-time human-ai coordination

    Jijia Liu, Chao Yu, Jiaxuan Gao, Yuqing Xie, Qingmin Liao, Yi Wu, and Yu Wang. Llm-powered hierarchi- cal language agent for real-time human-ai coordination. arXiv preprint arXiv:2312.15224 , 2023

    2023 arXiv

  16. [24]

    Understanding Human Dy- namic Sampling Objectives to Enable Robot-assisted Sci- entific Decision Making

    Shipeng Liu, Cristina G Wilson, Bhaskar Krishna- machari, and Feifei Qian. Understanding Human Dy- namic Sampling Objectives to Enable Robot-assisted Sci- entific Decision Making. ACM Transactions on Human- Robot Interaction, 2023

    2023

  17. [25]

    Wilson, Zachary I

    Shipeng Liu, Cristina G. Wilson, Zachary I. Lee, and Feifei Qian. Modelling Experts’ Sampling Strategy to Balance Multiple Objectives During Scientific Explo- rations. In Proceedings of the 2024 ACM/IEEE Interna- tional Conference on Human-Robot Interaction, HRI ’24, page 452–...

    2024

  18. [26]

    Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance

    Yaxi Lu, Shenzhi Yang, Cheng Qian, Guirong Chen, Qinyu Luo, Yesai Wu, Huadong Wang, Xin Cong, Zhong Zhang, Yankai Lin, et al. Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance. arXiv preprint arXiv:2410.12361 , 2024

    arXiv

  19. [27]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information pro- cessing systems, 35:27...

    2022

  20. [28]

    Language-Model- Based Paired Variational Autoencoders for Robotic Lan- guage Learning

    Ozan ¨Ozdemir, Matthias Kerzel, Cornelius Weber, Jae Hee Lee, and Stefan Wermter. Language-Model- Based Paired Variational Autoencoders for Robotic Lan- guage Learning. IEEE Transactions on Cognitive and Developmental Systems, 15(4):1812–1824, 2022

    2022

  21. [29]

    Active robot-assisted feeding with a general-purpose mobile manipulator: Design, evaluation, and lessons learned

    Daehyung Park, Yuuna Hoshi, Harshal P Mahajan, Ho Keun Kim, Zackory Erickson, Wendy A Rogers, and Charles C Kemp. Active robot-assisted feeding with a general-purpose mobile manipulator: Design, evaluation, and lessons learned. Robotics and Autonomous Systems , 124:103344, 2020

    2020

  22. [30]

    Ai-augmented predictions: Llm assistants improve human forecasting accuracy

    Philipp Schoenegger, Peter S Park, Ezra Karger, Sean Trott, and Philip E Tetlock. Ai-augmented predictions: Llm assistants improve human forecasting accuracy. arXiv preprint arXiv:2402.07862 , 2024

    2024 arXiv

  23. [31]

    Correcting robot plans with natural language feedback

    Pratyusha Sharma, Balakumar Sundaralingam, Valts Blukis, Chris Paxton, Tucker Hermans, Antonio Tor- ralba, Jacob Andreas, and Dieter Fox. Correcting robot plans with natural language feedback. arXiv preprint arXiv:2204.05186, 2022

    2022 arXiv

  24. [32]

    Optimizing autonomous driving for safety: A human- centric approach with llm-enhanced rlhf

    Yuan Sun, Navid Salami Pargoo, Peter Jin, and Jorge Or- tiz. Optimizing autonomous driving for safety: A human- centric approach with llm-enhanced rlhf. In Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing , pages 76–80, 2024

    2024

  25. [33]

    To Help or Not to Help: LLM-based Attentive Support for Human-Robot Group Interactions

    Daniel Tanneberg, Felix Ocker, Stephan Hasler, Jo- erg Deigmoeller, Anna Belardinelli, Chao Wang, Heiko Wersing, Bernhard Sendhoff, and Michael Gienger. To Help or Not to Help: LLM-based Attentive Support for Human-Robot Group Interactions. arXiv preprint arXiv:2403.12533, 2024

    2024 arXiv

  26. [34]

    Autogen: Enabling next- gen llm applications via multi-agent conversation frame- work

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xi- aoyun Zhang, and Chi Wang. Autogen: Enabling next- gen llm applications via multi-agent conversation frame- work. arXiv preprint arXiv:2308.08155 , 2023

    2023 arXiv

  27. [35]

    A three-dimensional mapping and virtual reality-based human–robot interaction for collaborative space explo- ration

    Junhao Xiao, Pan Wang, Huimin Lu, and Hui Zhang. A three-dimensional mapping and virtual reality-based human–robot interaction for collaborative space explo- ration. International Journal of Advanced Robotic Sys- tems, 17(3):1729881420925293, 2020

    2020

  28. [36]

    Proagent: Building proactive cooperative ai with large language models

    Ceyao Zhang, Kaijie Yang, Siyi Hu, Zihao Wang, Guanghe Li, Yihang Sun, Cheng Zhang, Zhaowei Zhang, Anji Liu, Song-Chun Zhu, et al. Proagent: Building proactive cooperative ai with large language models. arXiv preprint arXiv:2308.11339 , 2023

    2023 arXiv

  29. [37]

    Building cooperative embodied agents modularly with large language models

    Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B Tenenbaum, Tianmin Shu, and Chuang Gan. Building cooperative embodied agents modularly with large language models. arXiv preprint arXiv:2307.02485, 2023

    2023 arXiv

  30. [38]

    COMBO: Compositional World Models for Embodied Multi-Agent Cooperation

    Hongxin Zhang, Zeyuan Wang, Qiushi Lyu, Zheyuan Zhang, Sunli Chen, Tianmin Shu, Yilun Du, and Chuang Gan. COMBO: Compositional World Models for Embodied Multi-Agent Cooperation. arXiv preprint arXiv:2404.10775, 2024

    2024 arXiv

  31. [39]

    Mutual theory of mind in human- ai collaboration: An empirical study with llm-driven ai agents in a real-time shared workspace task

    Shao Zhang, Xihuai Wang, Wenhao Zhang, Yongshan Chen, Landi Gao, Dakuo Wang, Weinan Zhang, Xinbing Wang, and Ying Wen. Mutual theory of mind in human- ai collaboration: An empirical study with llm-driven ai agents in a real-time shared workspace task. arXiv preprint arXiv:2409...

    2024 arXiv

  32. [40]

    User on-boarding (goals, rules, consent)

  33. [41]

    Participant demographics a) What is your age? b) What is your gender? c) How often do you play video games every week? d) Are you familiar with the usage of large language models? embodied agents?

  34. [42]

    Satisfaction and engagement a) Describe the satisfactory level of different agents in different layouts from 1 (Very unsatisfied) to 7 (very satisfied). b) Please describe the reason for the different level of satisfaction c) Describe the engaging level of you when you interac...

  35. [43]

    Preference a) In layout ”easy”, which agent do you prefer? b) In layout ”medium”, which agent do you prefer? c) In layout ”hard”, which agent do you prefer?

  36. [44]

    Perceived intelligence a) Please evaluate the intelligence level of three dif- ferent agents from 1 (very unintelligent) to 7 (very intelligent)

  37. [45]

    Perceived trust a) Please indicate the level of trust you have in three different agents to work with you in the Overcooked game from 1 (very trustworthy) to 7 (very untrustworthy)

  38. [46]

    In particular,

    Miscellaneous a) If you were to play with this agent in an Over- cooked game competition, what changes or im- provements would you suggest for the agent’s feedback? APPENDIX C PROMPT CONSTRUCTION We include the full content of the prompts used by the Coordinator and the Manage...

  39. [47]

    we show prompts for initial graph generation in Ap- pendix C-A

    Coordinator generates the initial node graph and revises it with a human player through natural language dialog. we show prompts for initial graph generation in Ap- pendix C-A. we also show examples of revising the node graph by adding intermediate nodes, human preferences thr...

  40. [48]

    We show the prompts used for Manager to actively assign subtasks and change subtask status in Ap- pendix C-B

    Manager assigns subtasks to players and changes player status. We show the prompts used for Manager to actively assign subtasks and change subtask status in Ap- pendix C-B. A. Coordinaotor: Node Graph Generation and Revision This section provides the prompts for all possible n...

  41. [52]

    PUTTING", 1:

    Put the soup to the serve location. Remember, you must put the all ingredients and ,→ correct ingredients exactly as specified in ,→ the recipe book, a important thing that you ,→ have to put the soup to the serve location Recipe book: {recipe_book} Kitchen state: {kitchen_ite...

  42. [53]

    Only use the information above (recipe, ,→ kitchen items, etc.), Analyze the state of ,→ the kitchen and items, as well as the recipe ,→

  43. [54]

    Decompose the recipe needed to finish ,→ cooking the soup into subtasks, with its ,→ subtask type, status, and all possible ,→ target locations

  44. [55]

    In this turn, please: - Generate a a Directed Acyclic Graph (DAG) of ,→ subtasks in the correct chronological order

    Arrange these subtasks in chronological ,→ order. In this turn, please: - Generate a a Directed Acyclic Graph (DAG) of ,→ subtasks in the correct chronological order. - Output these subtasks in the given structure: Graph Revision With Dialogue After the node graph is generated...

  45. [56]

    find which edge have a huge cost

  46. [57]

    find potential collaboration intermediate ,→ point, where by collaborating on this ,→ location, the cost would be reduced

  47. [58]

    Instructions:| The normal procedure to finish one soup is:

    Are there any tricks when assigning ,→ different subtasks to different agent that ,→ can avoid collision. Instructions:| The normal procedure to finish one soup is:

  48. [59]

    Pick up the required ingredients

  49. [60]

    Put ingredients into pots

  50. [61]

    pick up the soup after it is ready

  51. [62]

    Put the soup to the serve location. Remember, you must put the all ingredients and ,→ correct ingredients exactly as specified in ,→ the recipe book, a important thing that you ,→ have to put the soup to the serve location Recipe book: {recipe_book} Kitchen state: {kitchen_ite...

  52. [63]

    do not assigne same task to human and robot

  53. [64]

    Assign emergency subtask first

  54. [65]

    Prioritize assign subtask to robot, if the ,→ robot are free, do not wait human to finish

  55. [66]

    always handle the high priority subtasks and ,→ then consider the cost. Instructions:| Robot current state {robot_state} Human current state: {human_state} To finish the recipe, two agent (robot and ,→ human) are following a subtask graph, below ,→ are subtasks that are ready ...

  56. [67]

    note that there will be multiple repeated ,→ subtasks; you must make judgment based on ,→ the status change before and after robot ,→ interaction

  57. [68]

    if a subtask is finished, the object held in ,→ the robot and human should change,d or the ,→ pot state will changed

  58. [69]

    only check the current executing tasks if ,→ they are assigned

  59. [70]

    you should be smart, if other unexecuted ,→ subtaskbut not currently executed by robot ,→ or human has been finished, human might not ,→ following the assinged tasks. Instructions:| Robot States: {robot_prev_state} {robot_state} Human States: {human_prev_state} {human_state} R...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.