Pith. sign in

REVIEW 5 major objections 5 minor 37 references

OVITA: Open-Vocabulary Interpretable Trajectory Adaptations

T0 review · 5 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read OVITA claims to be the first interpretable framework that adapts robot trajectories from open-vocabulary natural-language commands in a zero-shot way.

desk verdict A useful and well-motivated system, but the headline success rate doesn't isolate the zero-shot code-generation claim because feedback and code explanations are part of the scored protocol. read the letter →

arxiv 2508.17260 v1 pith:77H62PMR submitted 2025-08-24 cs.RO

Anurag Maurya , Tashmoy Ghosh , Anh Nguyen , Ravi Prakash This is my paper
classification cs.RO
keywords trajectoryadaptationopen-vocabularyinstructionslargelanguagemodelscode-as-policiesquadraticprogramminghuman-robotinteractionzero-shotgeneralizationinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a robot trajectory—whether from a planner or a human demonstration—can be adapted by an ordinary user's free-form language command, without any task-specific training. It claims a generic LLM, prompted with a fixed template and two examples, can turn that command into executable Python code that edits waypoints, while a quadratic-programming module smooths the result and enforces workspace, obstacle, and velocity constraints. A second LLM explains the code so non-experts can check and iterate, and a feedback loop corrects misunderstandings. If these claims hold, language becomes a practical, inspectable interface for low-level robot motion, not just high-level task planning.

What carries the argument

The central mechanism is 'code as an adaptation policy': the LLM writes Python that operates on trajectory data (positions and speeds), rather than predicting waypoints directly; this code is parsed, executed, and then passed through a quadratic-programming module (minimizing weighted deviation from the LLM reference while enforcing linearized workspace, obstacle, and velocity constraints) to yield a smooth, physically feasible trajectory.

What would settle it

Take an instruction class outside the two fixed examples—say, a novel spatial relation like 'trace a figure-eight around the cup and the box'—run OVITA without feedback, and independently rate whether the generated code's trajectory matches the instruction. If the executable-code rate or the semantic-correctness score drops toward chance on such held-out instructions, the zero-shot generalization claim is refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that trajectory adaptation can be formulated as LLM-generated code acting as an adaptation policy: given an initial trajectory, object positions, and a user instruction, a generic LLM produces both a high-level plan and Python code in one pass, and that code directly transforms the waypoint sequence. A convex QP then refines the LLM output for smoothness and constraint satisfaction, and an LLM-based explainer makes the policy legible to non-experts. The paper reports an 81.4% average task-success rate across 690 user evaluations, with success defined as a semantic-correctness score of 4 or 5, and states that OVITA outperforms both LaTTe and ExTraCT on the LaTTe s

Load-bearing premise

The framework assumes that a pretrained LLM, given only a task-agnostic template with two response-format examples plus object positions and a trajectory, will write semantically correct Python for arbitrary user instructions in new environments and on new robot platforms.

Editorial extensions

If this is right

  • If the zero-shot code-policy claim holds, trajectory adaptation no longer needs per-task datasets, fine-tuning, or handcrafted deformation features.
  • Any existing waypoint-based planner or demonstration-recording system could expose a natural-language editing interface by plugging in this pipeline.
  • Non-expert users could inspect what the robot will do, because the adaptation policy is readable code plus a generated natural-language explanation.
  • The feedback loop turns LLM hallucinations and misinterpretations into correctable errors rather than silent failures.
  • Language-driven trajectory augmentation could enrich synthetic training data for downstream imitation-learning methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The code-as-policy design may transfer to other sequential decision problems—such as path editing in CAD, animation, or surgical planning—where a user wants to modify a structured path and needs the edit to be explainable and auditable.
  • Editorial inference: The paper's success metric (user ratings of 4 or 5) is subjective; a sharper falsifier would check whether generated code is semantically correct against a labeled instruction-trajectory pair with an unambiguous ground truth.
  • Editorial inference: Because the QP only linearizes constraints around the LLM reference, the framework may handle small local edits well but could produce infeasible or unsafe trajectories for large deformations or cluttered scenes; testing that boundary is a natural next step.
  • Editorial inference: The fixed two-shot template suggests that the approach's generality hinges on the LLM's code-writing ability; as LLMs improve, the same pipeline could handle more abstract or multi-modal instructions without redesign.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes OVITA, a framework that uses a pre-trained LLM to translate natural-language instructions into executable Python code, which acts as an adaptation policy over waypoint trajectories. A second LLM explains the generated code to non-expert users, and a feedback loop allows iterative refinement. A QP-based constraint satisfaction module smooths and safety-constrains the adapted trajectory. The authors claim state-of-the-art performance against LaTTe and ExTraCT, reporting an 81.4% average success rate from 690 user-study evaluations across simulated and real-world heterogeneous robots (KUKA IIWA, Jackal, CrazyFlie), with ablations across LLMs, feedback, executability, interpretability, examples, and temperature.

Significance. If the central claims are validated, OVITA would be a useful contribution: it is training-free for new instructions, interpretable via generated code and explanations, and it demonstrates cross-platform real-robot feasibility. The paper's strengths include the formal QP formulation, multi-LLM evaluation, interactive feedback design, and extensive real-world demonstrations. The use of user studies with a two-stage protocol to reduce self-confirmation bias is thoughtful. However, the load-bearing evidence for these claims is weakened by evaluation confounds: the reported success rate includes human feedback and code explanations, the baselines are not described with the same level of experimental control, and no confidence intervals or effect sizes are reported. The claim that OVITA is the 'first interpretable framework for trajectory adaptation with open-vocabulary instruction support' is plausible but should be carefully positioned relative to prior work.

major comments (5)
  1. [§IV-B, §IV-C] The 81.4% success rate is not attributable to the language-to-code policy alone. In the two-stage evaluation, the second-stage evaluator sees not only the adapted trajectory but also the feedback history and the LLM-generated code explanation. The paper's own ablation (Fig. 8D) shows that feedback significantly improves success, yet a no-feedback success rate is not reported. Thus the headline number conflates code quality with (a) the evaluator's ability to infer intent from explanations and (b) the first-stage participant's ability to repair failures. To support the zero-shot, explanation-free claim, please provide a no-feedback, no-explanation evaluation. Also specify whether LaTTe and ExTraCT were evaluated with the same visualizations and no-feedback/no-explanation conditions; if they were, report those details.
  2. [§IV-C, Fig. 8C, §V] The 92.9% executable-code rate is a syntactic measure, not a semantic correctness measure. The limitations section candidly admits 'logical errors, though present, are readily corrected through feedback,' which means that a non-negligible fraction of code executions may implement the wrong spatial operation. Since the central claim is that a generic LLM can generate code that implements precise numerical, multi-step, and open-ended edits, the paper must report semantic correctness in a feedback-free setting. Without this, the 81.4% figure does not directly support the code-policy claim.
  3. [§IV-C, Fig. 7] The baseline superiority claim ('Our method outperforms both LaTTe and ExTraCT on both the LaTTe subset (p<0.01) and extended subset (p<0.005)') rests on Wilcoxon signed-rank tests, but the paired structure, sample size per condition, and whether multiple comparisons were controlled are not given. The 81.4% average success rate is reported without confidence intervals. Please report exact test statistics, effect sizes, and CIs, and clarify how many evaluations contributed to each p-value. The use of a curated 230-trajectory dataset also needs a clear description of how participants were assigned across conditions to avoid hidden dependencies.
  4. [§III-B, Eq. (5), §V] Obstacle-avoidance and spherical workspace constraints are linearized via first-order Taylor expansion around the reference trajectory xref. If the LLM-modified reference trajectory already violates a nonlinear constraint, the linearization at that point may not provide a valid separating hyperplane; the final QP solution could still violate the true nonlinear constraint. The paper states the approximation is 'accurate and well-suited' but provides no bound or verification. Given that the method claims to produce 'safe' and 'physically-feasible' trajectories, and the limitations section notes that QP solvers can fail under restrictive constraints, please provide empirical evidence that the linearized constraints are satisfied on the test set, e.g., by reporting the maximum violation of the true nonlinear constraints after solving.
  5. [§III-B, 'QP Formulation'] The QP hyperparameters λdev and λsmooth are selected by Optuna 'by minimizing the deviation cost.' It is unclear whether this tuning is performed on the evaluation set or a separate validation split. If the hyperparameters are tuned on the test data, the reported success rates are optimistic. Please clarify the tuning protocol, the tuning objective (deviation cost alone, or also user study scores?), and whether the same hyperparameters are used across all dataset subsets and LLMs.
minor comments (5)
  1. [§III-A] The text says 'prompts are provided in the prompts.py file of the codebase,' but no code repository or data availability link is given in the manuscript. Please include a URL in the final version.
  2. [Abstract / Fig. 10] There are typographical artifacts in the preprint, e.g., 'KUKA IIW A' in the abstract and 'IIW A' in the author affiliation line. Also, the label 'Speed Speed Speed' in Fig. 9 is duplicated; please clean the figure captions.
  3. [§III-A] The phrase 'monologue-based interactions' in the related work section is unclear. If it refers to the feedback context selector (original vs. current), please explain this term or replace it with a clearer phrase.
  4. [§IV-B] The user study description does not specify whether the 30 participants were the same across datasets and LLMs, or whether participants in stage 2 were also participants in stage 1 for other samples. Please clarify the participant assignment and counterbalancing to rule out learning effects.
  5. [§IV-F] The 'Impact of example exclusion' ablation is under-specified: it reports that success rates decline 'only slightly,' but no numbers are given. Please provide the actual success rates, and state whether the two fixed examples were task-agnostic across all conditions.

Circularity Check

0 steps flagged · score 1.0 of 10

Empirical system paper; no prediction is equivalent to its input by construction; minor evaluation confounds only.

full rationale

This is an empirical system paper rather than a derivation, so the classical circularity patterns do not apply. The central claim — that a generic LLM, given a fixed prompt template, object positions, and a trajectory, generates executable Python implementing the user's instruction — is tested by a two-stage user study; success is a human rating, not a quantity computed from the model's own outputs. The QP hyperparameters λdev and λsmooth are selected by Optuna on the deviation cost (Section III-B), not fitted to the success labels, so the reported 81.4% average success rate is not a re-statement of the tuning objective. The prompt examples are explicitly 'kept the same across all tasks' (Section III-A), so there is no per-task fitting. The one self-citation ([10], R. Prakash co-author) appears only in background about learning from demonstrations and is not load-bearing. The limitations section honestly concedes that logical errors are 'readily corrected through feedback' and that performance depends on precise user input; this is a validity caveat, because the headline number includes feedback and evaluators saw code explanations (Section IV-B), but it does not make the output equal to the input by construction. No equation defines a prediction in terms of its own fitted value. Hence no specific circular step can be quoted; the score reflects only minor evaluation self-referentiality, not circular derivation.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

No new particles, forces, dimensions, or physical entities are introduced. The paper's components are existing LLMs, generated Python code, and a QP optimizer. The free parameters are standard optimization and safety hyperparameters whose fitted values are not reported.

free parameters (3)
  • lambda_dev and lambda_smooth = not reported (tuned by Optuna TPE over 1e-3 to 1e1, 50 trials)
    QP objective weights in Eq. 1; chosen to minimize deviation cost. They affect how closely the final trajectory follows the LLM-modified reference and how smooth it is, so they are part of the pipeline.
  • safety margin delta = not reported
    Used in workspace, obstacle, and velocity constraints (Eq. 3-6); values must be set by the user or environment and affect QP feasibility.
  • v_max = robot-specific, not reported
    Velocity constraint in Eq. 6; required to make adapted trajectories executable on the real platforms.
assumptions (6)
  • domain assumption Pre-trained LLMs can generate executable Python code that correctly interprets free-form natural language trajectory instructions from a fixed prompt with only two examples.
    The whole method rests on zero-shot code generation; the paper reports 92.9% executable code rate, so a non-negligible failure fraction remains and feedback is needed.
  • domain assumption A Cartesian waypoint trajectory (x,y,z,v) is a sufficient representation for adapting motions of a manipulator, ground robot, and drone after low-level IK and control.
    Used throughout Section III-A and in the real-world experiments; this skips detailed dynamics modeling for some platforms.
  • domain assumption Object positions and dimensions from LangSAM masks, depth data, and camera intrinsics are accurate enough for QP obstacle constraints.
    Used in Section IV-E for real-world experiments; no error analysis of the perception pipeline is provided.
  • domain assumption The convex QP with first-order Taylor linearization of spherical constraints is a valid local approximation because the LLM-modified path stays close to the final trajectory.
    Section III-B states the reference is the LLM-modified trajectory, making linear constraint approximations accurate; if the modification is large, the linearization can be inaccurate.
  • domain assumption Human Likert ratings (4 or 5 equals success) measure semantic correctness of trajectory adaptations.
    Evaluation design in Section IV-B; no objective metric is used, and the paper argues RMSE is unsuitable for open-vocabulary instructions.
  • ad hoc to paper The two fixed prompt examples do not bias the LLM toward specific trajectory outputs.
    Section III-A says the examples are kept the same across all tasks and demonstrate the response format for a high-level plan, not the code policy; this is a design choice, not a proven property.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OVITA: Open-Vocabulary Interpretable Trajectory Adaptations." pith.science (2026). https://pith.science/paper/77H62PMR

@misc{pith2026250817260,
  author       = {Pith},
  title        = {Pith review of: OVITA: Open-Vocabulary Interpretable Trajectory Adaptations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/77H62PMR}},
  note         = {Machine review of arXiv:2508.17260}
}
read the original abstract

Adapting trajectories to dynamic situations and user preferences is crucial for robot operation in unstructured environments with non-expert users. Natural language enables users to express these adjustments in an interactive manner. We introduce OVITA, an interpretable, open-vocabulary, language-driven framework designed for adapting robot trajectories in dynamic and novel situations based on human instructions. OVITA leverages multiple pre-trained Large Language Models (LLMs) to integrate user commands into trajectories generated by motion planners or those learned through demonstrations. OVITA employs code as an adaptation policy generated by an LLM, enabling users to adjust individual waypoints, thus providing flexible control. Another LLM, which acts as a code explainer, removes the need for expert users, enabling intuitive interactions. The efficacy and significance of the proposed OVITA framework is demonstrated through extensive simulations and real-world environments with diverse tasks involving spatiotemporal variations on heterogeneous robotic platforms such as a KUKA IIWA robot manipulator, Clearpath Jackal ground robot, and CrazyFlie drone.

Figures

Figures reproduced from arXiv: 2508.17260 by the authors.

Figure 1
Figure 1. Traditional planning algorithms often overlook user [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. OVITA Block Diagram: A) Prompt grounding from instruction and environment description. B) LLM generates HLP and code, followed by execution and constraint satisfac￾tion. C) Visualization with code explanation. D) User feedback for refinement. user’s instruction, scene descriptions, and object positions are then structured within the prompt template, resulting in a physically grounded prompt. prompts are provided in … view at source ↗
Figure 3
Figure 3. Code as an adaptation policy: the instruction is "Move [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Visual illustration of our approach to mitigate LLM hal [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Results on open-vocabulary instructions: The original trajectory is represented in blue, while the modified trajectory using our approach is shown in red. 4) (Optional) Equality Constraints. To enforce fixed start and/or goal configurations: Ax = b : x0 = x ref 0 , xT …
Figure 6
Figure 6. Figure 6: OVITA, LaTTe, and ExTracT Trajectory Comparison [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Performance Comparison between OVITA, LaTTe and [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Evaluation of OVITA: (A) across various LLMs, (B) dataset subsets, (C) executable code generation rates, (D) feedback [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Simulation results with a robotic arm and drone along [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 28 canonical work pages

  1. [1]

    A survey on large language model based autonomous agents,

    L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y . Lin,et al., “A survey on large language model based autonomous agents,” Frontiers of Computer Science , vol. 18, no. 6, p. 186345, 2024

    work page 2024

  2. [2]

    Llm-planner: Few-shot grounded planning for embodied agents with large language models,

    C. H. Song, J. Wu, C. Washington, B. M. Sadler, W.-L. Chao, and Y . Su, “Llm-planner: Few-shot grounded planning for embodied agents with large language models,” in Proceedings of the IEEE/CVF international conference on computer vision , 2023, pp. 2998–3009

    work page 2023

  3. [3]

    Progprompt: Generating situated robot task plans using large language models,

    I. Singh, V . Blukis, A. Mousavian, A. Goyal, D. Xu, J. Tremblay, D. Fox, J. Thomason, and A. Garg, “Progprompt: Generating situated robot task plans using large language models,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2023, pp. 11 523–11 530

    work page 2023

  4. [4]

    V oxposer: Composable 3d value maps for robotic manipulation with language models,

    W. Huang, C. Wang, R. Zhang, Y . Li, J. Wu, and L. Fei-Fei, “V oxposer: Composable 3d value maps for robotic manipulation with language models,” in Conference on Robot Learning. PMLR, 2023, pp. 540–562

    2023

  5. [5]

    Language to rewards for robotic skill synthesis,

    W. Yu, N. Gileadi, C. Fu, S. Kirmani, K.-H. Lee, M. G. Arenas, H.- T. L. Chiang, T. Erez, L. Hasenclever, J. Humplik, et al., “Language to rewards for robotic skill synthesis,” in Conference on Robot Learning . PMLR, 2023, pp. 374–404

    work page 2023

  6. [6]

    Language models as zero-shot trajectory generators,

    T. Kwon, N. Di Palo, and E. Johns, “Language models as zero-shot trajectory generators,” IEEE Robotics and Automation Letters , 2024

    work page 2024

  7. [7]

    Sampling-based algorithms for optimal motion planning,

    S. Karaman and E. Frazzoli, “Sampling-based algorithms for optimal motion planning,” The international journal of robotics research, vol. 30, no. 7, pp. 846–894, 2011

    2011

  8. [8]

    Rawlings, D

    J. Rawlings, D. Mayne, and M. Diehl, Model Predictive Control: Theory, Computation, and Design . Nob Hill Publishing, 2017

    work page 2017

Show all 37 references
  1. [9]

    Recent advances in robot learning from demonstration,

    H. Ravichandar, A. S. Polydoros, S. Chernova, and A. Billard, “Recent advances in robot learning from demonstration,” Annual review of control, robotics, and autonomous systems , vol. 3, no. 1, pp. 297–330, 2020

    2020

  2. [10]

    Interactive imitation learning in robotics: A survey,

    C. Celemin, R. Pérez-Dattari, E. Chisari, G. Franzese, L. de Souza Rosa, R. Prakash, Z. Ajanovi ´c, M. Ferraz, A. Valada, J. Kober, et al. , “Interactive imitation learning in robotics: A survey,” Foundations and Trends® in Robotics, vol. 10, no. 1-2, pp. 1–197, 2022

    2022

  3. [11]

    Explainable agents and robots: Results from a systematic literature review,

    S. Anjomshoae, A. Najjar, D. Calvaresi, and K. Främling, “Explainable agents and robots: Results from a systematic literature review,” in 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2019), Montreal, Canada, May 13–17, 2019 , 2019, pp. 1078– 1088

    2019

  4. [12]

    An algorithmic perspective on imitation learning,

    T. Osa, J. Pajarinen, G. Neumann, J. A. Bagnell, P. Abbeel, J. Peters, et al. , “An algorithmic perspective on imitation learning,” Foundations and Trends® in Robotics , vol. 7, no. 1-2, pp. 1–179, 2018

    2018

  5. [13]

    A tutorial on task-parameterized movement learning and retrieval,

    S. Calinon, “A tutorial on task-parameterized movement learning and retrieval,” Intelligent service robotics , vol. 9, pp. 1–29, 2016

    2016

  6. [14]

    Dynamical movement primitives: learning attractor models for motor behaviors,

    A. J. Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor, and S. Schaal, “Dynamical movement primitives: learning attractor models for motor behaviors,” Neural computation, vol. 25, no. 2, pp. 328–373, 2013

    2013

  7. [15]

    Probabilis- tic movement primitives,

    A. Paraschos, C. Daniel, J. R. Peters, and G. Neumann, “Probabilis- tic movement primitives,” Advances in neural information processing systems, vol. 26, 2013

    2013

  8. [16]

    Kernelized movement primitives,

    Y . Huang, L. Rozo, J. Silvério, and D. G. Caldwell, “Kernelized movement primitives,” The International Journal of Robotics Research , vol. 38, no. 7, pp. 833–852, 2019

    2019

  9. [17]

    Learning multi-reference frame skills from demonstration with task-parameterized gaussian processes,

    M. R. Montero, G. Franzese, J. Kober, and C. Della Santina, “Learning multi-reference frame skills from demonstration with task-parameterized gaussian processes,” in 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2024, pp. 2832–2839

    2024

  10. [18]

    Palm-e: An embodied multimodal language model,

    D. Driess, F. Xia, M. S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, et al., “Palm-e: An embodied multimodal language model,” in International Conference on Machine Learning. PMLR, 2023, pp. 8469–8488

    2023

  11. [19]

    Large language models as generalizable policies for embodied tasks,

    A. Szot, M. Schwarzer, H. Agrawal, B. Mazoure, R. Metcalf, W. Talbott, N. Mackraz, R. D. Hjelm, and A. T. Toshev, “Large language models as generalizable policies for embodied tasks,” in The Twelfth International Conference on Learning Representations , 2023

    2023

  12. [20]

    Robotgpt: Robot manipulation learning from chatgpt,

    Y . Jin, D. Li, A. Yong, J. Shi, P. Hao, F. Sun, J. Zhang, and B. Fang, “Robotgpt: Robot manipulation learning from chatgpt,” IEEE Robotics and Automation Letters , vol. 9, no. 3, pp. 2543–2550, 2024

    2024

  13. [21]

    Do as i can, not as i say: Grounding language in robotic affordances,

    M. Ahn, A. Brohan, N. Brown, Y . Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman, et al. , “Do as i can, not as i say: Grounding language in robotic affordances,” arXiv preprint arXiv:2204.01691, 2022

    2022 arXiv

  14. [22]

    Code as policies: Language model programs for embodied control,

    J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng, “Code as policies: Language model programs for embodied control,” in 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 9493–9500

    2023

  15. [23]

    Inner monologue: Embod- ied reasoning through planning with language models,

    W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y . Chebotar,et al., “Inner monologue: Embod- ied reasoning through planning with language models,” in Conference on Robot Learning . PMLR, 2023, pp. 1769–1782

    2023

  16. [24]

    Robots that use language,

    S. Tellex, N. Gopalan, H. Kress-Gazit, and C. Matuszek, “Robots that use language,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 3, no. 1, pp. 25–55, 2020

    2020

  17. [25]

    Multimodal estimation and communication of latent semantic knowledge for robust execution of robot instructions,

    J. Arkin, D. Park, S. Roy, M. R. Walter, N. Roy, T. M. Howard, and R. Paul, “Multimodal estimation and communication of latent semantic knowledge for robust execution of robot instructions,” The International Journal of Robotics Research, vol. 39, no. 10-11, pp. 1279–1304, 2020

    2020

  18. [26]

    No, to the right: Online language corrections for robotic ma- nipulation via shared autonomy,

    Y . Cui, S. Karamcheti, R. Palleti, N. Shivakumar, P. Liang, and D. Sadigh, “No, to the right: Online language corrections for robotic ma- nipulation via shared autonomy,” in Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction , 2023, pp. 93– 101

    2023

  19. [27]

    Bert: a review of applications in natural language processing and understanding,

    M. V . Koroteev, “Bert: a review of applications in natural language processing and understanding,” arXiv preprint arXiv:2103.11943, 2021

    2021 arXiv

  20. [28]

    Correcting robot plans with natural language feedback,

    P. Sharma, B. Sundaralingam, V . Blukis, C. Paxton, T. Hermans, A. Torralba, J. Andreas, and D. Fox, “Correcting robot plans with natural language feedback,” arXiv preprint arXiv:2204.05186 , 2022

    2022 arXiv

  21. [29]

    Reshaping robot trajectories using natural language commands: A study of multi-modal data alignment using transformers,

    A. Bucker, L. Figueredo, S. Haddadinl, A. Kapoor, S. Ma, and R. Bon- atti, “Reshaping robot trajectories using natural language commands: A study of multi-modal data alignment using transformers,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IRO...

    2022

  22. [30]

    Latte: Language trajectory transformer,

    A. Bucker, L. Figueredo, S. Haddadin, A. Kapoor, S. Ma, S. Vemprala, and R. Bonatti, “Latte: Language trajectory transformer,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2023, pp. 7287–7294

    2023

  23. [31]

    Extract– explainable trajectory corrections from language inputs using textual description of features,

    J. Yow, N. P. Garg, M. Ramanathan, W. T. Ang, et al. , “Extract– explainable trajectory corrections from language inputs using textual description of features,” arXiv preprint arXiv:2401.03701 , 2024

    2024 arXiv

  24. [32]

    Optuna: A next- generation hyperparameter optimization framework,

    T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A next- generation hyperparameter optimization framework,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , 2019

    2019

  25. [33]

    Gpt-4o system card,

    A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, et al., “Gpt-4o system card,” arXiv preprint arXiv:2410.21276 , 2024

    2024 arXiv

  26. [34]

    Claude haiku [large language model],

    Anthropic, “Claude haiku [large language model],” 2024. [Online]. Available: https://www.anthropic.com

    2024

  27. [35]

    Gemini: a family of highly capable multimodal models,

    G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican,et al., “Gemini: a family of highly capable multimodal models,”arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  28. [36]

    Lowry, Concepts and Applications of Inferential Statistics , 2014

    R. Lowry, Concepts and Applications of Inferential Statistics , 2014. [Online]. Available: http://vassarstats.net/textbook/

    2014

  29. [37]

    Langsam: Language segment-anything,

    L. Medeiros, “Langsam: Language segment-anything,” https://github. com/luca-medeiros/lang-segment-anything, accessed: 2023-10-01

    2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.