RoboNav-Arm: Agentic AI-Driven Navigation and Obstacle Avoidance for Robotic Manipulator in Cluttered Environments

Aachal Sharma; Narendra Kumar Dhar

RoboNav-Arm: Agentic AI-Driven Navigation and Obstacle Avoidance for Robotic Manipulator in Cluttered Environments

T0 review · 3 major / 0 minor · reviewed 2026-07-14 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An agentic AI system lets a robot arm see, remember, pick a planner, and keep clear of clutter so it can finish tasks without collisions in unknown workspaces.

desk verdict Solid systems integration of LLM orchestration + OSUM-to-MoveIt + memory for static clutter; success rates are real but the agentic credit is unisolated and the 'dynamic' claim is unsupported. read the letter →

arxiv 2607.09716 v1 pith:7NXXFEQ3 submitted 2026-06-25 cs.RO

Aachal Sharma , Narendra Kumar Dhar This is my paper · ORCID

classification cs.RO

keywords roboticmanipulationobstacleavoidanceagenticAImotionplanningsceneunderstandingsafety-awaretrajectoryclutteredenvironmentsMoveIt

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Robotic arms still struggle when the workspace is cluttered and not known in advance; classical planners need a full map up front, while pure learning methods are sample-hungry and hard to keep safe. This paper claims that a single decision-making agent can close the loop by calling three tools on demand: a perception module that turns RGB-D images into object boxes, positions and workspace risk labels, a memory store of past successful plans, and a trajectory module that chooses among RRT-family planners and then inflates obstacles for extra clearance. The agent watches execution, re-invokes perception or memory when clearance shrinks or a plan fails, and keeps the robot moving only on trajectories that stay outside the inflated obstacles. In simulation the approach reaches 100 percent collision-free success in empty or exterior-obstacle scenes and still 83–93 percent in sparse-to-dense clutter, showing that high-level language-model reasoning plus geometric safety checks can produce usable, adaptive manipulation without a pre-built map.

What carries the argument

The decision-making agent (an LLM that receives a short semantic report of obstacle count, minimum distance, goal pose and prior failures) that selectively invokes the Object-Level Scene Understanding Module, Contextual Memory and Retrieval Module, and Trajectory Planning and Execution Module, then enforces safety by Minkowski-inflating every obstacle geometry before MoveIt collision checking and by aborting any trajectory whose clearance falls below a threshold.

What would settle it

Run the same four Gazebo scenarios with a non-LLM baseline that always uses a single fixed planner and no memory; if the agentic version shows no statistically higher collision-free success or lower replan count, the central claim that the LLM orchestration is necessary fails.

Watch

Extended reading notes

Core claim

An end-to-end agentic framework that couples real-time object-level scene understanding, memory-guided adaptation, adaptive selection among RRTConnect, RRT* and BiTRRT, and safety-inflated trajectory validation produces collision-free manipulator motion in unknown cluttered workspaces, with measured success rates of 100 percent, 100 percent, 93.3 percent and 83.3 percent across four Gazebo scenarios of rising obstacle density.

Load-bearing premise

That a language model given only a brief semantic summary plus optional past-plan retrieval can reliably choose the right planner, speed and replan trigger without formal completeness or safety proofs beyond geometric inflation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a competent engineering paper that wires known pieces—YOLO-World, SAM, ground-support centroid localization, MoveIt collision injection, RRT-family planners, ChromaDB memory, and an LLM coordinator—into a closed loop and reports honest numbers on a Kinova Gen3 in Gazebo. The measured collision-free rates (100/100/93.3/83.3 % across four static density levels, 30 trials each) and the LLM comparison (Llama lighter than DeepSeek/Qwen) are useful data points for anyone building agentic pick-and-place stacks. The OSUM’s ground-support surface centroid and LLM-inferred compact collision shapes are a clean practical detail that makes the MoveIt scene match the workspace well (Fig. 4). Table I is fair about partial overlap with concurrent LLM-TAMP work.

What is actually new is the orchestration pattern itself: short semantic report → optional memory retrieval → planner/velocity/safety choice → continuous clearance monitoring and replan. That loop is clearly described and the degradation with clutter density looks coherent. The math is just Minkowski inflation and min-distance checks; nothing load-bearing is broken.

Soft spots are real but proportionate. The abstract and conclusion say “dynamic scenarios” while every scene in Fig. 3 is static and future work lists moving obstacles—that language should be fixed. There is no non-agentic baseline (fixed RRTConnect + fixed inflation + MoveIt monitoring alone), so we cannot yet credit the LLM decisions for the success rates; Table II already shows different LLMs change replan counts and times substantially. Free parameters (σ_i, δ, ε, κ, LLM temperature) are never tabulated. These are fixable gaps for a systems paper, not fatal ones.

Who it is for: people already shipping MoveIt + sampling planners who want a concrete agentic wrapper and empirical numbers under increasing static clutter. It does not open a new theory class or guarantee safety beyond geometric checks. I would still send it to referees; the architecture is clear, the static results are reproducible in principle, and the missing controls are standard revision requests rather than desk-reject reasons. Engage if you are building similar stacks; skip if you need formal guarantees or dynamic-obstacle results.

Referee Report

3 major / 0 minor

Summary. The paper presents RoboNav-Arm, an agentic framework for collision-aware robotic manipulation in cluttered unknown workspaces. A decision-making LLM agent orchestrates three modules: Object-Level Scene Understanding (OSUM: YOLO-World + SAM + depth projection, ground-support centroid localization, LLM-inferred compact collision shapes, and a semantic workspace report), Contextual Memory and Retrieval (CMRM: ChromaDB of past environment–trajectory–safety–recovery tuples), and Trajectory Planning and Execution (TPEM: adaptive selection among RRTConnect/RRT*/BiTRRT, safety inflation of obstacles, MoveIt collision checking, and continuous clearance monitoring with replan triggers). The system is evaluated in Gazebo Classic on a Kinova Gen3 with four static workspace layouts of increasing density (empty, obstacles outside reach, sparse clutter, dense clutter). With Llama-3.1-70B selected after a three-LLM comparison, collision-free success rates of 100%/100%/93.3%/83.3% are reported over 30 trials per scenario (Tables III–IV), with corresponding increases in planning/execution time and replan count.

Significance. If the measured success rates and closed-loop behavior hold under stronger controls, the work offers a practical, modular integration of open-vocabulary perception, memory-guided high-level decisions, and safety-inflated sampling-based planning for manipulators—addressing a real gap between high-level LLM planners and low-level geometric safety. Strengths include a clear modular architecture (Fig. 2), explicit safety inflation and clearance monitoring, multi-LLM comparison (Table II), and quantitative multi-trial statistics with standard deviations. The contribution is primarily systems-level rather than theoretical; its value for the community depends on whether the agentic layer demonstrably improves over non-agentic geometric baselines and whether claims of dynamic robustness are aligned with the static evaluation.

major comments (3)

Abstract and §V claim evaluation “demonstrating robustness in dynamic scenarios,” yet §III and Fig. 3 define four static obstacle layouts; future work explicitly lists “handling dynamic moving obstacles.” This overstatement is load-bearing for the paper’s positioning against classical planners that assume static prior maps. Either restrict claims to static unknown clutter or add at least one moving-obstacle experiment.
Tables III–IV attribute 100/100/93.3/83.3 % collision-free success to the agentic loop (LLM planner/velocity/replan decisions + optional CMRM). No non-agentic baseline (fixed RRTConnect or BiTRRT + fixed inflation + continuous MoveIt monitoring alone) or ablation (agent without memory; agent without adaptive planner selection) is reported. Without these controls it is impossible to isolate the contribution of the LLM coordinator versus standard geometric planning, which is central to the claim in §I and Table I.
§II.A–II.2 introduce free safety parameters σ_i = f(g_i, λ_i), clearance threshold δ, ground-support tolerance ε, and memory similarity κ, none of which are tabulated or sensitivity-tested. Table II already shows large LLM-dependent shifts in planning time and replan count. Reproducibility and the claimed robustness require reporting the concrete values used for the main results and a brief sensitivity check.

Circularity Check

0 steps flagged · score 1.0 of 10

Empirical systems paper with measured success rates; no derivation reduces to its inputs by construction or self-citation.

full rationale

RoboNav-Arm is an engineering/systems paper whose central claims are empirical collision-free success rates (Tables III–IV: 100/100/93.3/83.3 %) measured in Gazebo under four static obstacle layouts. The agentic loop (LLM decision over obstacle_count/min_distance/previous_failures, optional ChromaDB retrieval of prior runs, adaptive choice among RRTConnect/RRT*/BiTRRT, safety inflation g′_i = g_i ⊕ B(σ_i), clearance check ϕ < δ) conditions behavior but does not algebraically define or force the reported metrics. Definitions (D=⟨G,S,A⟩, E_t={O_t,W_t}, o_i=(l_i,Δ_i,g_i,p_i) with g_i=LLM(l_i,Δ_i), p_i from ground-support centroid, trajectory validation ∀Q_k R(Q_k)∩g′_i=∅) are standard modular constructions, not self-referential. Memory M stores prior successful trajectories for similarity retrieval; this is intentional adaptation, not a tautology that makes success automatic. No parameters are fitted to a data subset and then re-predicted; no uniqueness theorem or ansatz is imported via self-citation; citations (YOLO-World, SAM, RRT variants, external LLM-robotics works) are independent. Minor residual (score 1) is only that the agent can consult its own prior runs, which is ordinary closed-loop engineering and does not render the evaluation circular. Success rates remain externally measured outcomes.

Assumptions & free parameters 4 free parameters · 4 assumptions · 3 invented entities

The central claim rests on standard robotics tooling plus several unvalidated modeling choices (safety inflation, ground-support localization, LLM reliability) and free numeric thresholds that are never fitted publicly or ablated. No new physical entities; the ‘invented’ items are named software modules whose value is only shown inside this paper’s sim.

free parameters (4)

safety inflation radius / buffer σ_i (or d_safe) and clearance threshold δ
Controls when trajectories are accepted or aborted; values are not reported numerically or ablated, yet directly determine reported collision-free rates.
ground-support height tolerance ε
Defines Bi = {p ∈ Pi | z(p) ≤ z_min + ε}; choice of ε affects estimated object positions pi injected into MoveIt.
memory similarity function κ and retrieval policy
Determines which past trajectories/strategies are reused; embedding model and threshold unspecified.
LLM choice and temperature / decoding settings
Table II shows large performance gaps across DeepSeek/Qwen/Llama; final claims use Llama without reporting decoding hyperparameters.

assumptions (4)

domain assumption Open-vocabulary detections (YOLO-World) plus SAM masks plus depth yield collision geometries accurate enough for MoveIt safety checking after LLM shape abstraction to boxes/cylinders.
Invoked throughout §II.1 and Fig. 4; no quantitative pose/size error reported.
ad hoc to paper Object position is best estimated as the centroid of the lowest ground-support patch Bi rather than the full point-cloud centroid.
Stated in §II.1 as providing ‘physically stable’ localization; no comparison experiment.
domain assumption An LLM given a short structured report can select among RRTConnect/RRT*/BiTRRT and safety strategies well enough to improve robustness over fixed planning.
Core of Decision-Making Prompt and §II.A; only supported by internal LLM comparison, not against non-LLM control.
domain assumption Static Gazebo Classic scenes with fixed eye-to-hand RGB-D are a valid test of the claimed unstructured / dynamic obstacle-avoidance problem.
Evaluation §III–IV; abstract claims dynamic robustness while scenarios S1–S4 are static.

invented entities (3)

Object-Level Scene Understanding Module (OSUM) with LLM-inferred compact collision shapes
purpose: Turn RGB-D into MoveIt collision objects plus a semantic workspace report for the agent.
Named system component; independent evidence limited to qualitative Fig. 4 alignment in this paper.
Contextual Memory and Retrieval Module (CMRM) over ChromaDB of past (E, τ, c, r) tuples
purpose: Reuse prior successful plans and recovery strategies under similar geometry.
Standard vector memory pattern rebranded; no external validation of the memory schema.
Agentic decision state D = ⟨G, S, A⟩ with A = {a_env, a_mem, a_plan}
purpose: Formalize LLM tool orchestration for perception, memory, and planning.
Lightweight notation for a standard tool-calling loop; not an independently testable physical entity.

how reviews work

0 comments

Cite this review

Pith. "Pith review of RoboNav-Arm: Agentic AI-Driven Navigation and Obstacle Avoidance for Robotic Manipulator in Cluttered Environments." pith.science (2026). https://pith.science/paper/7NXXFEQ3

@misc{pith2026260709716,
  author       = {Pith},
  title        = {Pith review of: RoboNav-Arm: Agentic AI-Driven Navigation and Obstacle Avoidance for Robotic Manipulator in Cluttered Environments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7NXXFEQ3}},
  note         = {Machine review of arXiv:2607.09716}
}

read the original abstract

Robotic manipulators operating in unstructured environments face significant challenges in safely executing goal-directed tasks due to dynamic and unforeseen obstacles, while traditional methods rely on prior knowledge or fixed perception pipelines, limiting adaptability. We propose a framework for safe task execution with effective obstacle avoidance. The environment module performs real-time obstacle detection, 3D localization, and ground surface geometry estimation. It then generates a structured semantic report that includes obstacle positions, object geometry and shape, and whether obstacles lie inside, outside, or within critical interaction zones. A central coordination module manages the overall system by handling tool invocation (e.g., memory and MoveIt collision scene updates), facilitating communication between modules, and continuously monitoring task progress until completion. Furthermore, a planning module selects an appropriate motion planning algorithm, such as RRTConnect, RRT*, or BiTRRT, based on the current environment configuration and goal requirements. The trajectory generated by the planner is further analyzed and refined to ensure safe and collision-free task execution. The proposed approach is evaluated in Gazebo Classic , demonstrating robustness in dynamic scenarios.

Figures

Figures reproduced from arXiv: 2607.09716 by the authors.

**Figure 1.** An overview of RoboNav-Arm To deal with these limitations, learning-based approaches such as reinforcement learning (RL) and imitation learning have been studied. RL-based methods allow robots to learn through interaction, which is useful for complex manipulation tasks, but they often struggle with low sample efficiency and the challenge of defining suitable reward functions [4], [5]. On the other hand, imitation l… view at source ↗

**Figure 1.** The system is built around a decision-making agent [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** The primary challenge lies in adaptively coordinating [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

Figures from the paper (4 more)

**Figure 2.** Figure 2: The overall architecture of RoboNav-Arm, which consists of different tools used by the decision-making agent. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png]

**Figure 3.** Figure 3: Experimental setup and evaluation scenarios [PITH_FULL_IMAGE:figures/full_fig_p005_3.png]

**Figure 4.** Figure 4: Visualization of workspace configurations and cor [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: Robot trajectory with obstacle configurations for four [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 4 linked inside Pith

[1]

Adaptive manipulability- based path planning strategy for industrial robot manipulators,

H. Shen, W.-F. Xie, J. Tang, and T. Zhou, “Adaptive manipulability- based path planning strategy for industrial robot manipulators,” IEEE/ASME Transactions on Mechatronics, vol. 28, no. 3, pp. 1742– 1753, 2023

2023
[2]

Trajectory optimization and obstacle avoidance of autonomous robot using robust and efficient rapidly exploring random tree,

N. U. Islam, K. Gul, F. Faiz, S. S. Ullah, and I. Syed, “Trajectory optimization and obstacle avoidance of autonomous robot using robust and efficient rapidly exploring random tree,”PLoS ONE, vol. 19, no. 10, p. e0311179, 2024

2024
[3]

Path planning algorithm for manipulators in complex scenes based on improved rrt*,

X. Zhang, P. Wang, Y . Guo, Q. Han, and K. Zhang, “Path planning algorithm for manipulators in complex scenes based on improved rrt*,” Sensors, vol. 25, no. 2, p. 328, 2025

2025
[4]

Reinforcement learning- based robotic arm path planning in complex scenarios,

Z. Meng, X. Zhou, H. Chen, and J. Mao, “Reinforcement learning- based robotic arm path planning in complex scenarios,” inProceedings of the 2nd International Conference on Computer , Internet of Things and Smart City (CIoTSC). ACM, 2025, pp. 167–172

2025
[5]

World4rl: Diffusion world models for policy refinement with reinforcement learning for robotic manipulation,

Z. Jiang, K. Liu, Y . Qin, S. Tian, Y . Zheng, M. Zhou, C. Yu, H. Li, and D. Zhao, “World4rl: Diffusion world models for policy refinement with reinforcement learning for robotic manipulation,”arXiv preprint arXiv:2509.19080, 2025

arXiv 2025
[6]

A novel robust imitation learning framework for complex skills with limited demonstrations,

W. Wang, C. Zeng, H. Zhan, and C. Yang, “A novel robust imitation learning framework for complex skills with limited demonstrations,” IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 3947–3959, 2025

2025
[7]

Llm-planner: Few-shot grounded planning for embodied agents with large language models,

C. Song, B. M. Sadler, J. Wu, W.-L. Chao, C. Washington, and Y . Su, “Llm-planner: Few-shot grounded planning for embodied agents with large language models,” inProceedings of the IEEE International Conference on Computer Vision (ICCV), 2023, pp. 2986–2997

2023
[8]

Empow- ering large language models on robotic manipulation with affordance prompting,

G. Cheng, C. Zhang, W. Cai, L. Zhao, C. Sun, and J. Bian, “Empow- ering large language models on robotic manipulation with affordance prompting,”arXiv preprint arXiv:2404.11027, 2024

arXiv 2024

Show all 18 references

[9]

Vision-language model predictive control for manipulation planning and trajectory generation,

J. Chen, W. Zhao, Z. Meng, D. Mao, R. Song, W. Pan, and W. Zhang, “Vision-language model predictive control for manipulation planning and trajectory generation,”arXiv preprint arXiv:2504.05225, 2025. [Online]. Available: https://arxiv.org/abs/2504.05225

2025 arXiv
[10]

Smart-llm: Smart multi- agent robot task planning using large language models,

S. Kannan, V . Venkatesh, and B.-C. Min, “Smart-llm: Smart multi- agent robot task planning using large language models,” inProceed- ings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024, pp. 12 140–12 147

2024
[11]

V oxposer: Composable 3d value maps for robotic manipulation with language models,

W. Huang, C. Wang, R. Zhang, Y . Li, J. Wu, and L. Fei-Fei, “V oxposer: Composable 3d value maps for robotic manipulation with language models,”arXiv preprint arXiv:2307.05973, 2023

2023 arXiv
[12]

Language models as zero-shot trajectory generators,

T. Kwon, N. D. Palo, and E. Johns, “Language models as zero-shot trajectory generators,”IEEE Robotics and Automation Letters, vol. 9, no. 7, pp. 6728–6735, 2024

2024
[13]

An end-to-end gpt-4o informed pipeline for manipulation task planning with sim2real validation,

S. Guo, T. Tian, J. Zhou, Z. Liu, L. Zhou, R. Zhao, Z. Huang, C. Yuan, and M. H. Ang, “An end-to-end gpt-4o informed pipeline for manipulation task planning with sim2real validation,” in2025 10th International Conference on Control and Robotics Engineering (ICCRE), 2025, pp. 99–104

2025
[14]

LLM3: Large language model-based task and motion planning with motion failure reasoning,

S. Wang, M. Han, Z. Jiao, Z. Zhang, Y . N. Wu, S.-C. Zhu, and H. Liu, “LLM3: Large language model-based task and motion planning with motion failure reasoning,” inMulti-modal F oundation Model meets Embodied AI Workshop @ ICML2024, 2024. [Online]. Available: https://openreview...

2024
[15]

Evolutionary computation applied to the control of a robotic manipulator with obstacle avoidance,

L. P. Vinaud Neto, N. C. Rodrigues, B. R. T. Santos, and E. Do Valle Sim˜oes, “Evolutionary computation applied to the control of a robotic manipulator with obstacle avoidance,” in2025 IEEE Congress on Evolutionary Computation (CEC), 2025, pp. 1–4

2025
[16]

Non-holonomic mobile manipulator nmpc for occlusion avoidance based on elliptic cone fov representation,

R. Por ´ee, M. Mujica, N. Tricot, and V . Cadenat, “Non-holonomic mobile manipulator nmpc for occlusion avoidance based on elliptic cone fov representation,” in2025 European Conference on Mobile Robots (ECMR), 2025, pp. 1–7

2025
[17]

Yolo-world: Real-time open-vocabulary object detection,

T. Cheng, L. Song, Y . Ge, W. Liu, and X. Wang, “Yolo-world: Real-time open-vocabulary object detection,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 16 901–16 911

2024
[18]

Segment anything,

A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Doll ´ar, and R. Gir- shick, “Segment anything,”arXiv preprint arXiv:2304.02643, 2023

2023 arXiv

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.

[1] [1]

Adaptive manipulability- based path planning strategy for industrial robot manipulators,

H. Shen, W.-F. Xie, J. Tang, and T. Zhou, “Adaptive manipulability- based path planning strategy for industrial robot manipulators,” IEEE/ASME Transactions on Mechatronics, vol. 28, no. 3, pp. 1742– 1753, 2023

2023

[2] [2]

Trajectory optimization and obstacle avoidance of autonomous robot using robust and efficient rapidly exploring random tree,

N. U. Islam, K. Gul, F. Faiz, S. S. Ullah, and I. Syed, “Trajectory optimization and obstacle avoidance of autonomous robot using robust and efficient rapidly exploring random tree,”PLoS ONE, vol. 19, no. 10, p. e0311179, 2024

2024

[3] [3]

Path planning algorithm for manipulators in complex scenes based on improved rrt*,

X. Zhang, P. Wang, Y . Guo, Q. Han, and K. Zhang, “Path planning algorithm for manipulators in complex scenes based on improved rrt*,” Sensors, vol. 25, no. 2, p. 328, 2025

2025

[4] [4]

Reinforcement learning- based robotic arm path planning in complex scenarios,

Z. Meng, X. Zhou, H. Chen, and J. Mao, “Reinforcement learning- based robotic arm path planning in complex scenarios,” inProceedings of the 2nd International Conference on Computer , Internet of Things and Smart City (CIoTSC). ACM, 2025, pp. 167–172

2025

[5] [5]

World4rl: Diffusion world models for policy refinement with reinforcement learning for robotic manipulation,

Z. Jiang, K. Liu, Y . Qin, S. Tian, Y . Zheng, M. Zhou, C. Yu, H. Li, and D. Zhao, “World4rl: Diffusion world models for policy refinement with reinforcement learning for robotic manipulation,”arXiv preprint arXiv:2509.19080, 2025

arXiv 2025

[6] [6]

A novel robust imitation learning framework for complex skills with limited demonstrations,

W. Wang, C. Zeng, H. Zhan, and C. Yang, “A novel robust imitation learning framework for complex skills with limited demonstrations,” IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 3947–3959, 2025

2025

[7] [7]

Llm-planner: Few-shot grounded planning for embodied agents with large language models,

C. Song, B. M. Sadler, J. Wu, W.-L. Chao, C. Washington, and Y . Su, “Llm-planner: Few-shot grounded planning for embodied agents with large language models,” inProceedings of the IEEE International Conference on Computer Vision (ICCV), 2023, pp. 2986–2997

2023

[8] [8]

Empow- ering large language models on robotic manipulation with affordance prompting,

G. Cheng, C. Zhang, W. Cai, L. Zhao, C. Sun, and J. Bian, “Empow- ering large language models on robotic manipulation with affordance prompting,”arXiv preprint arXiv:2404.11027, 2024

arXiv 2024