Pith. sign in

REVIEW 3 major objections 5 minor 32 references

A Human-in-the-loop Approach to Robot Action Replanning through LLM Common-Sense Reasoning

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A user can correct a one-shot, vision-generated robot plan by talking to an LLM, which edits a semantic behavior tree and decodes it back into executable waypoints.

desk verdict A neat semantic bridge for LLM-based plan repair, but the discrete label space quietly caps what users can actually request. read the letter →

arxiv 2507.20870 v1 pith:MFPDT75F submitted 2025-07-28 cs.RO

Elena Merlo , Marta Lagomarsino , Arash Ajoudani This is my paper
classification cs.RO
keywords human-in-the-loopinteractiveplanninglargelanguagemodelsfailuremitigationprogrammingbydemonstrationbehaviortreesnaturalfeedbackone-shotrobot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes that a non-expert can repair and personalize a robot plan that was automatically generated from a single RGB video by talking to an LLM. The vision-based plan is first translated into a semantic behavior tree with human-readable pose labels, the LLM edits that tree in response to natural-language requests, and the edited tree is decoded back into executable waypoints. The claimed advantage is that common vision errors such as incorrect contact heights or tilts can be fixed without extra demonstrations or programming, and the user keeps global oversight of the whole plan. Experimental evidence in the paper shows a pouring task and a tray-cleaning task where the vision-only plan failed and the LLM-refined plan succeeded.

What carries the argument

The carrier of the argument is a semantic round-trip through behavior trees. A Label Encoder converts each numeric target pose into a triplet of labels: the nearest interaction point on the background object, a three-state vertical relation (above, touching, or below), and a rotation described by an axis (side bending, tilting, or turning) plus an angle quantized to a 45-degree grid. A Plan Refiner feeds the resulting semantic behavior tree, object interaction-point labels, and structured XML guidelines to an LLM along with the user's request, and the LLM returns a modified tree. A Label Decoder then inverts the triplet back to a numeric transformation matrix for execution.

What would settle it

Show a user a vision-generated plan whose only defect is a required vertical offset between the touching and above thresholds, or a rotation of 30 degrees rather than the nearest 45-degree grid value, and ask them to fix it without a new demonstration; if the semantic label space cannot represent the needed correction and no rephrasing works, the framework's claim that it can repair user-correctable errors fails.

Watch

Extended reading notes

Core claim

The central claim is that a user can correct a one-shot, vision-generated robot plan using natural language routed through an LLM, so the final behavior tree avoids failures and matches the user's intent. The paper demonstrates this with pouring and cleaning tasks, where the user's verbal corrections fixed perception-derived pose errors, and where high-level requests such as 'pour less water' changed plan parameters like tilt angles and execution time. The paper further reports that in a ten-user study all participants obtained an executable plan that completed the cleaning task, requiring on average 2.4 refinement iterations, and that LLM hallucinations could be caught by the user through a roll-back mechanism.

Load-bearing premise

The whole repair channel depends on the assumption that the user's intent and the LLM's edits, once expressed as a nearest interaction point, a three-level height, and one of eight rotations, can still be decoded into a pose that actually achieves the task.

Editorial extensions

If this is right

  • Vision errors like wrong contact height can be corrected with one or two short spoken requests, without re-recording demonstrations.
  • High-level intent can be injected as parameter changes, for example reducing a pour tilt angle from -90 degrees to -60 degrees to pour less water.
  • Users can catch and revert LLM hallucinations, acting as a sanity loop over the LLM's edits.
  • The approach supports plan personalization and extension, such as adding waypoints to clean the right side of the tray.
  • Semantic labels keep non-experts in the loop because the plan remains human-readable throughout the refinement process.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 45-degree rotation quantization and the three-state vertical label bound the expressible corrections; tasks requiring intermediate poses or out-of-grid rotations would need a richer label space.
  • The same encoder-LLM-decoder round-trip could apply to other discrete plan representations, such as skill graphs or PDDL-like plans, as long as a faithful label encoder and decoder exist.
  • A testable extension is to measure the maximum pose error that can still be corrected through this channel, and to compare success rates when users phrase requests at different abstraction levels.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents a human-in-the-loop pipeline that converts a single RGB-video demonstration into a behavior-tree robot plan via information-theoretic analysis, translates the numeric plan into a semantic label-based BT, lets a user issue natural-language requests to an LLM (GPT-4o) that edits the semantic BT, and decodes the edited BT back into executable waypoints. The central claim is that this mechanism lets non-expert users correct vision-derived errors and personalize plans without additional demonstrations. Experiments include two pilot tasks (pouring, cleaning) with physical robot execution, a 10-subject usability study on the cleaning task, and a 20-request evaluation of LLM common-sense reasoning for adjusting the poured amount.

Significance. If the result holds, the paper offers a practical protocol for one-shot observational learning with language-based repair: it is end-to-end, uses interpretable behavior trees, and gives the user global pre-execution oversight. The paper's concrete strengths are the physical validation in the two pilot tasks, including a measured force reduction from over 60 N to low contact in the cleaning task, the inclusion of naive users, and the use of a deployed LLM with automatic speech recognition rather than a simulated oracle. The main risk is the expressiveness of the semantic label space, which constrains the class of corrections that can be honored; this limits the generality of the claimed capability unless the representation is extended or the claims are appropriately qualified.

major comments (3)
  1. [Sections II-B1 and II-B3] The semantic label space is a hard representational bottleneck that is load-bearing for the paper's central claim that users can correct and adapt the vision-based plan through natural language. In the encoder (Section II-B1), the vertical displacement is collapsed into three states (above, touching, below) with a fixed threshold, and the horizontal position is mapped to the closest manually defined interaction point; in the decoder (Section II-B3), these labels are converted back to z = zabove = 0.15 m, z = epsilon_z = 0, or z = zbelow = -0.15 m and to the x-y coordinates of the nearest IP. Consequently, any user instruction that implies an intermediate height (e.g., 'hold the sponge 3 cm above the tray') or a position that is not an IP (e.g., 'between the center and the left rim') is unrepresentable no matter how well the LLM understands the request. The experiments only exercise discrete transitions (below to above in pouring, above to touching in cleaning) and movement between predefined corners or edges, so the paper provides no evidence for finer-grained corrections. The Discussion's limitation list (Section V) mentions missing condition checks and action nodes but does not acknowledge this quantization gap. Note that the orientation dimension appears less restricted: Fig. 4-e shows the LLM outputting -60 degrees, which is outside the encoder's quantized set of eight angles, so the paper should clarify whether the decoder accepts arbitrary rotation angles and whether the stated three-label format is actually enforced.
  2. [Section III-B and Results B] The multi-subject experiment does not provide statistical support for the claimed robustness and usability. The results report average refinement iterations (N = 2.4), SUS question averages with error bars, and a per-request satisfaction bar chart, but no statistical tests, confidence intervals, or effect sizes are given. Moreover, the claim that 'all the subjects finally obtained a logical and executable plan that enabled the robot to complete the cleaning task successfully' is not backed by per-subject physical execution data; it is unclear whether each participant's final BT was actually executed on the robot or only inspected by the experimenter. Because the paper's contribution is specifically the interactive loop across users, this evidence gap weakens the generalization claim and should be addressed with execution logs or at least an explicit statement of which plans were physically executed.
  3. [Section III-C and Table I] The evaluation of LLM reasoning in plan adaptation relies on experimenter-only binary judgments (tick/cross) without predefined success criteria, a second annotator, or inter-rater reliability. The failure cases R19 and R20 are explained with post-hoc hypotheses ('likely due to confusion caused by the presence of the word less') that are not testable from the recorded data. This makes the claimed 18/20 success rate less robust than it appears. If the table is meant as an exploratory analysis, this should be stated; otherwise the evaluation needs an objective metric, such as predetermined criteria for what counts as 'fulfilled'.
minor comments (5)
  1. [Figure 4 caption] The phrase 'SemBT generated to encounter this input' should be 'SemBT generated to address this input' or 'to accommodate this input'.
  2. [Section III-A] The marker system is referred to as 'AruCo'; the standard spelling is 'ArUco'.
  3. [Section III-A] The phrase 'The LLM model we employed' contains a redundancy; use 'The LLM we employed'.
  4. [Section II-B2] The statement that the three-label format is 'implicitly' requested from the LLM is vague; because the decoder depends on that format, the constraint should be explicit and its enforcement should be tested, especially since Fig. 4-e contains an angle outside the encoder's quantized set.
  5. [Section III-B] The paper reports SUS question averages rather than the standard total SUS score (0-100); reporting the total score and its distribution would make the usability result more comparable with prior work.

Circularity Check

1 steps flagged · score 3.0 of 10

The user-satisfaction evaluation loop makes the headline 'successful LLM adaptation' partly self-fulfilling, but the LLM's per-turn edits are independently testable and do fail, so circularity is only partial.

  1. fitted input called prediction [Section III-B / IV-B (multi-subject robustness experiment); restore mechanism in Sec. II-B2]
    "At each iteration, users rated whether the LLM plan modifications matched their requests by selecting: Satisfied, Quite Satisfied, or Not Satisfied. ... Importantly, all the subjects finally obtained a logical and executable plan that enabled the robot to complete the cleaning task successfully, with an average of N = 2.4 iterations."

    The outcome variable is whether the LLM output matches the user's own request, and the protocol lets the user iterate, restore the previous semantic plan, and rephrase until satisfied (Sec. II-B2: 'If the user is unsatisfied with the outcome, they can restore the previous version SemBT j−1'). Therefore the final 'all subjects obtained an executable plan' result is guaranteed by the human-in-the-loop stopping rule rather than by the LLM's reasoning alone. This is partial circularity: the LLM's individual outputs were not all satisfactory (partial/unsatisfactory ratings occurred), so the per-turn edit capability retains independent content, but the headline effectiveness claim is partly self-fulfilling.

full rationale

The paper's central pipeline is an LLM instruction-following layer on top of a self-cited vision-plan generator [8]. The self-citation to [8] is not load-bearing for the new contribution: the LLM refinement is tested on actual failing plans, and the LLM demonstrably fails on some requests (R9, R19, R20). The semantic label encoder/decoder (Secs. II-B1/II-B3) is a codec: decoded poses are definitional inverses of labels, and the coarse z/position quantization limits expressiveness, but that is a representational limitation, not a circular derivation. The main circularity concern is the evaluation protocol: success is measured by the user's own rating of match to their request, with unlimited iterations and version restore, so eventual satisfaction is partly guaranteed by the protocol. This affects the strength of the usability/effectiveness claim but does not make the central 'LLM can translate feedback into plan edits' claim vacuous, since per-iteration failures are reported. Score 3 reflects one partial, evaluation-level circularity with an otherwise independent system demonstration.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central system relies on hand-chosen thresholds, manual interaction point definitions, and the assumption that a discrete semantic label space can carry the required corrections. These are not fitted to reproduce a benchmark, but they are free choices that affect the outcome.

free parameters (6)
  • zth = 0.01 m
    Threshold classifying vertical displacement into above, touching, or below labels in the label encoder (Section II-B1). Chosen by hand; affects semantics and decoded z coordinates.
  • z_above, z_below, epsilon_z = 0.15 m, -0.15 m, 0 m
    Mappings from semantic vertical labels to numeric z offsets in the Plan Synthetizer (Sections II-B3 and III-A). Chosen to ensure separation or contact; arbitrary choices.
  • orientation quantization set = 0, 45, 90, 135, 180, -135, -90, -45 degrees
    Nearest-angle mapping for the orientation label in Section II-B1. Discretizes rotation and loses continuous pose information.
  • stiffness levels = low=1000 N/m, medium=1500 N/m, high=2000 N/m
    Arm translational stiffness values for the ExecTrajectory node (Section III-A). Chosen by hand; the LLM selects among them.
  • d_oo threshold dth_oo = not reported
    Threshold for detecting object-object proximity when extracting the approach target pose (Section II-A1). The threshold's existence is assumed, but its value is not reported.
  • LLM-chosen pouring angle = -60 degrees for the 'pour less' request
    In Section IV-A the LLM reduced the tilt from -90 to -60 degrees. This value is generated by the model and is not derived from any stated physical principle.
assumptions (5)
  • domain assumption Mutual information between hand and object positions over a window indicates physical interaction, and local MI minima mark task-relevant key poses.
    Invoked in Section II-A1 and used to extract Target Poses for both pilot tasks. This is inherited from the authors' prior work and is not independently validated in this paper.
  • domain assumption The semantic label space, consisting of nearest interaction point, three-state vertical label, and eight quantized rotations, is sufficient to express user corrections and decode to feasible robot poses.
    The Plan Explainer and Plan Synthetizer (Sections II-B1 and II-B3) rely on this lossy representation. Any unrepresentable correction cannot be handled by the system.
  • domain assumption GPT-4o follows the XML structure guidelines and produces semantically appropriate modifications.
    Section II-B2 assumes this. The paper's own Table I shows three failures out of 20 requests, and users found logical hallucinations in 3 of 24 requests, so the assumption is only partially satisfied.
  • ad hoc to paper Interaction points for the objects are manually defined and sufficient for semantic labeling.
    IP lists for glass and tray are hand-specified in Section III-A. There is no automatic IP generation procedure, and the method's generality depends on this manual step.
  • domain assumption The robot execution environment matches the demonstration environment, with known object poses obtained from AruCo markers.
    Section III-A describes marker-based pose detection. The experiments do not test the method under markerless or unknown pose tracking.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Human-in-the-loop Approach to Robot Action Replanning through LLM Common-Sense Reasoning." pith.science (2026). https://pith.science/paper/MFPDT75F

@misc{pith2026250720870,
  author       = {Pith},
  title        = {Pith review of: A Human-in-the-loop Approach to Robot Action Replanning through LLM Common-Sense Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MFPDT75F}},
  note         = {Machine review of arXiv:2507.20870}
}
read the original abstract

To facilitate the wider adoption of robotics, accessible programming tools are required for non-experts. Observational learning enables intuitive human skills transfer through hands-on demonstrations, but relying solely on visual input can be inefficient in terms of scalability and failure mitigation, especially when based on a single demonstration. This paper presents a human-in-the-loop method for enhancing the robot execution plan, automatically generated based on a single RGB video, with natural language input to a Large Language Model (LLM). By including user-specified goals or critical task aspects and exploiting the LLM common-sense reasoning, the system adjusts the vision-based plan to prevent potential failures and adapts it based on the received instructions. Experiments demonstrated the framework intuitiveness and effectiveness in correcting vision-derived errors and adapting plans without requiring additional demonstrations. Moreover, interactive plan refinement and hallucination corrections promoted system robustness.

Figures

Figures reproduced from arXiv: 2507.20870 by the authors.

Figure 1
Figure 1. The system processes video demonstration frames to generate a behavior tree (gray block), converts it to a semantic version, and allows users to refine [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Experimental setup: the user consults the Graphical User Interface [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. (a) Hand-jug Mutual Information signal (orange/red) and jug-glass [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: (a) Hand-sponge MI signal and sponge-tray distance during human [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 7
Figure 7. Figure 7: User’s satisfaction with LLM-adapted plans across [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Average users’ ratings for SUS questions represented by orange [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 6
Figure 6. Figure 6: (a) User’s requests to modify the vision-based plan; (b) LLM-generated [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 23 canonical work pages

  1. [1]

    Ergonomic human-robot collaboration in industry: A review,

    M. Lorenzini, M. Lagomarsino, L. Fortini, S. Gholami, and A. Ajoudani, “Ergonomic human-robot collaboration in industry: A review,” Frontiers in Robotics and AI , vol. 9, p. 813907, 2023

    work page 2023

  2. [2]

    Survey: Robot programming by demonstration,

    A. Billard, S. Calinon, R. Dillmann, and S. Schaal, “Survey: Robot programming by demonstration,” Springer handbook of robotics , pp. 1371–1394, 2008

    work page 2008

  3. [3]

    Recent advances in robot learning from demonstration,

    H. Ravichandar, A. S. Polydoros, S. Chernova, and A. Billard, “Recent advances in robot learning from demonstration,” Annual review of control, robotics, and autonomous systems , vol. 3, pp. 297–330, 2020

    2020

  4. [4]

    Emulation, imitation, over-imitation and the scope of culture for child and chimpanzee,

    A. Whiten, N. McGuigan, S. Marshall-Pescini, and L. M. Hopper, “Emulation, imitation, over-imitation and the scope of culture for child and chimpanzee,” Philosophical Trans. of the Royal Society B: Biological Sciences, vol. 364, pp. 2417–2428, 2009

    work page 2009

  5. [5]

    Seeing to learn: Observational learning of robotic manipulation tasks,

    L. Pauly, “Seeing to learn: Observational learning of robotic manipulation tasks,” Ph.D. dissertation, University of Leeds, 2021

    work page 2021

  6. [6]

    Transferring skills to humanoid robots by extracting semantic representations from observations of human activities,

    K. Ramirez-Amaro, M. Beetz, and G. Cheng, “Transferring skills to humanoid robots by extracting semantic representations from observations of human activities,” Artificial Intelligence, vol. 247, pp. 95–118, 6 2017

    work page 2017

  7. [7]

    Symbolic representation of what robots are taught in one demonstration,

    A. M. Zanchettin, “Symbolic representation of what robots are taught in one demonstration,” Robotics and Autonomous Systems , vol. 166, p. 104452, 2023

    work page 2023

  8. [8]

    Exploiting information theory for intuitive robot programming of manual activities,

    E. Merlo, M. Lagomarsino, E. Lamon, and A. Ajoudani, “Exploiting information theory for intuitive robot programming of manual activities,” IEEE Trans. on Robotics , pp. 1–17, 2025

    work page 2025

Show all 32 references
  1. [9]

    Automated Generation of Robotic Planning Domains from Observations,

    M. Diehl, C. Paxton, and K. Ramirez-Amaro, “Automated Generation of Robotic Planning Domains from Observations,” IEEE Intl. Conf. on Intelligent Robots and Systems , pp. 6732–6738, 2021

    2021

  2. [10]

    Gaussian-process-based robot learning from demonstration,

    M. Arduengo, A. Colom ´e, J. Lobo-Prat, L. Sentis, and C. Torras, “Gaussian-process-based robot learning from demonstration,” Journal of Ambient Intelligence and Humanized Computing , pp. 1–14, 2023

    2023

  3. [11]

    Vision-based human activity recognition: a survey,

    D. R. Beddiar, B. Nini, M. Sabokrou, and A. Hadid, “Vision-based human activity recognition: a survey,”Multimedia Tools and Applications, vol. 79, pp. 30 509–30 555, 2020

    2020

  4. [12]

    Vision- based human action recognition: An overview and real world challenges,

    I. Jegham, A. B. Khalifa, I. Alouani, and M. A. Mahjoub, “Vision- based human action recognition: An overview and real world challenges,” Forensic Science Intl.: Digital Investigation , p. 200901, 2020

    2020

  5. [13]

    Robot programming by non-experts: Intuitiveness and robustness of one-shot robot programming,

    E. M. Orendt, M. Fichtner, and D. Henrich, “Robot programming by non-experts: Intuitiveness and robustness of one-shot robot programming,” in 2016 25th IEEE Intl. Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE, 2016, pp. 192–199

    2016

  6. [14]

    Language-conditioned imitation learning for robot manipulation tasks,

    S. Stepputtis, J. Campbell, M. Phielipp, S. Lee, C. Baral, and H. Ben Amor, “Language-conditioned imitation learning for robot manipulation tasks,” Advances in Neural Information Processing Systems , vol. 33, pp. 13 139–13 150, 2020

    2020

  7. [15]

    Using both demonstrations and lan- guage instructions to efficiently learn robotic tasks,

    A. Yu and R. J. Mooney, “Using both demonstrations and lan- guage instructions to efficiently learn robotic tasks,” arXiv preprint arXiv:2210.04476, 2022

    2022 arXiv

  8. [16]

    Concept2robot: Learning manipulation concepts from instructions and human demonstra- tions,

    L. Shao, T. Migimatsu, Q. Zhang, K. Yang, and J. Bohg, “Concept2robot: Learning manipulation concepts from instructions and human demonstra- tions,” The Intl. Journal of Robotics Research , vol. 40, pp. 1419–1434, 2021

    2021

  9. [17]

    What matters in language conditioned robotic imitation learning over unstructured data,

    O. Mees, L. Hermann, and W. Burgard, “What matters in language conditioned robotic imitation learning over unstructured data,” IEEE Robotics and Automation Letters , vol. 7, pp. 11 205–11 212, 2022

    2022

  10. [18]

    Bc-z: Zero-shot task generalization with robotic imitation learning,

    E. Jang, A. Irpan, M. Khansari, D. Kappler, F. Ebert, C. Lynch, S. Levine, and C. Finn, “Bc-z: Zero-shot task generalization with robotic imitation learning,” in Conf. on Robot Learning , 2022, pp. 991–1002

    2022

  11. [19]

    Zero-shot task adaptation using natural language,

    P. Goyal, R. J. Mooney, and S. Niekum, “Zero-shot task adaptation using natural language,” arXiv preprint arXiv:2106.02972 , 2021

    2021 arXiv

  12. [20]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

    2023 arXiv

  13. [21]

    Demo2code: From summarizing demonstrations to synthesizing code via extended chain-of-thought,

    Y . Wang, G. Gonzalez-Pumariega, Y . Sharma, and S. Choudhury, “Demo2code: From summarizing demonstrations to synthesizing code via extended chain-of-thought,” Advances in Neural Information Processing Systems, vol. 36, pp. 14 848–14 956, 2023

    2023

  14. [22]

    Teaching robots with show and tell: Using foundation models to synthesize robot policies from language and visual demonstration,

    M. Murray, A. Gupta, and M. Cakmak, “Teaching robots with show and tell: Using foundation models to synthesize robot policies from language and visual demonstration,” in Annual Conf. on Robot Learning , 2024

    2024

  15. [23]

    Gpt-4v (ision) for robotics: Multimodal task planning from human demonstration,

    N. Wake, A. Kanehira, K. Sasabuchi, J. Takamatsu, and K. Ikeuchi, “Gpt-4v (ision) for robotics: Multimodal task planning from human demonstration,” IEEE Robotics and Automation Letters , 2024

    2024

  16. [24]

    Large language models for robotics: Opportunities, challenges, and perspectives,

    J. Wang, Z. Wu, Y . Li, H. Jiang, P. Shu, E. Shi, H. Hu, C. Ma, Y . Liu, X. Wang et al. , “Large language models for robotics: Opportunities, challenges, and perspectives,” arXiv preprint arXiv:2401.04334 , 2024

    2024 arXiv

  17. [25]

    Correcting robot plans with natural language feedback,

    P. Sharma, B. Sundaralingam, V . Blukis, C. Paxton, T. Hermans, A. Torralba, J. Andreas, and D. Fox, “Correcting robot plans with natural language feedback,” arXiv preprint arXiv:2204.05186 , 2022

    2022 arXiv

  18. [26]

    Latte: Language trajectory transformer,

    A. Bucker, L. Figueredo, S. Haddadin, A. Kapoor, S. Ma, S. Vemprala, and R. Bonatti, “Latte: Language trajectory transformer,” in 2023 IEEE Intl. Conf. on Robotics and Automation (ICRA) . IEEE, 2023, pp. 7287– 7294

    2023

  19. [27]

    Yell at your robot: Improving on-the-fly from language corrections,

    L. X. Shi, Z. Hu, T. Z. Zhao, A. Sharma, K. Pertsch, J. Luo, S. Levine, and C. Finn, “Yell at your robot: Improving on-the-fly from language corrections,” arXiv preprint arXiv:2403.12910 , 2024

    2024 arXiv

  20. [28]

    Language to rewards for robotic skill synthesis,

    W. Yu, N. Gileadi, C. Fu, S. Kirmani, K.-H. Lee, M. G. Arenas, H.-T. L. Chiang, T. Erez, L. Hasenclever, J. Humplik et al., “Language to rewards for robotic skill synthesis,” arXiv preprint arXiv:2306.08647 , 2023

    2023 arXiv

  21. [29]

    No, to the right: Online language corrections for robotic manipulation via shared autonomy,

    Y . Cui, S. Karamcheti, R. Palleti, N. Shivakumar, P. Liang, and D. Sadigh, “No, to the right: Online language corrections for robotic manipulation via shared autonomy,” in Proceedings of the 2023 ACM/IEEE Intl. Conf. on Human-Robot Interaction , 2023, pp. 93–101

    2023

  22. [30]

    General object foundation model for images and videos at scale,

    J. Wu, Y . Jiang, Q. Liu, Z. Yuan, X. Bai, and S. Bai, “General object foundation model for images and videos at scale,” in Proceedings of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition , 2024, pp. 3783–3795

    2024

  23. [31]

    Gpt-4o system card,

    OpenAI, “Gpt-4o system card,” 2024. [Online]. Available: https: //arxiv.org/abs/2410.21276

    2024 arXiv

  24. [32]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in Intl. Conf. on machine learning . PMLR, 2023, pp. 28 492–28 518. MERLO et al.: A HUMAN-IN-THE-LOOP APPROACH TO ROBOT ACTION REPLANNING THR...

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.