Pith. sign in

REVIEW 4 major objections 6 minor 43 references

Fusing indoor IoT sensors with robot fleets lets LLM planners cut exploration cost while holding high task success under partial observability.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 21:37 UTC pith:P5LWLGH5

load-bearing objection Solid systems paper: indoor R2X benchmark + ablations show IoT mainly buys efficiency once R2R restores success; gains hinge on clean semantic IoT labels. the 4 major comments →

arxiv 2603.20182 v4 pith:P5LWLGH5 submitted 2026-03-20 cs.RO cs.MA

IndoorR2X: Indoor Robot-to-Everything Coordination with LLM-Driven Planning

Fan Yang , Soumya Teotia , Shaunak A. Mehta , Prajit KrisshnaKumar , Quanting Xie , Jun Liu , Yueqi Song , Wenkai Li
show 3 more authors
Atsunori Moteki Kanji Uchino Yonatan Bisk
This is my paper
classification cs.RO cs.MA
keywords multi-robot coordinationRobot-to-Everythingpartial observabilityIoT sensingLLM planningsemantic world modelindoor service robotsAI2-THOR
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Indoor multi-robot teams fail or waste effort when each robot only sees its own field of view and visited rooms. The paper argues that existing building IoT—static cameras and device status feeds—can close that gap if their observations are fused with robot reports into one shared semantic world model. IndoorR2X is a benchmark and coordination framework that does exactly that: a hub maintains a global state, an LLM planner emits parallel dependency-graph plans over it, and an orchestrator executes and replans. Across 85 multi-room scenes, robot-to-robot sharing already lifts success far above isolated robots, while full robot-to-everything further shortens paths, action steps, and planner token use without lowering success. A physical Stretch-robot deployment shows the same pattern: cameras reveal out-of-sight targets so robots skip search. The practical claim is that smart-building sensors turn multi-robot coordination from exhaustive exploration into guided routing.

Core claim

Under strictly partial observability, fusing static IoT observations with onboard robot perception into a shared global semantic state lets an LLM planner maintain high multi-robot task success while reducing action steps, path length, and planning tokens relative to robot-to-robot sharing alone; inter-robot communication is necessary for feasibility, and IoT supplies the efficiency gain.

What carries the argument

The coordination hub’s global semantic state W_t, updated by a fusion function over robot and IoT observations, over which an LLM emits a parallelizable dependency-graph plan that an orchestrator dispatches and replans online.

Load-bearing premise

The system treats fused IoT semantic reports—especially camera-derived object locations and device on/off states—as accurate enough that the planner can act on them as trustworthy priors rather than noisy claims that need verification.

What would settle it

In the same 85-scene suite with three robots, inject realistic rates of incorrect IoT semantic status (wrong device state or object location) matching the paper’s corruption ablation: if success falls sharply while path length stays low, the efficiency claim depends on clean IoT semantics; if success and efficiency both hold under verified fusion, the claim stands.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Homes and offices with CCTV and appliance status feeds can run multi-robot household tasks with less redundant search than pure robot teams.
  • LLM multi-robot planners should be evaluated under enforced partial observability plus configurable external sensors, not oracle global state.
  • Missing IoT detections mainly add travel; false semantic status can cause unrecoverable plan errors, so verification layers become a design requirement.
  • Team size can grow while success stays high, but aggregate path and coordination overhead still rise, so fleet sizing remains an efficiency trade-off.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same hub-plus-LLM pattern could apply to warehouses or hospitals that already instrument rooms with fixed cameras and equipment telemetry.
  • Because token cost drops with better priors, IoT fusion may make stronger closed-source planners cheaper per episode than pure robot dialogue.
  • A natural next test is closed-loop verification: robots re-check IoT claims before irreversible actions, trading a small path cost for higher success under corruption.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. IndoorR2X introduces a benchmark and centralized coordination framework for LLM-driven multi-robot planning under realistic partial observability, fusing onboard robot observations with static indoor IoT/CCTV signals into a shared global semantic state W_t. The hub runs an online sense–plan–act loop (Algorithm 1): R2X fusion (Eq. 5), LLM planning into a dependency DAG (Eq. 6), parallel dispatch, and event-driven replanning. On 85 AI2-THOR-based scenes with three robots and GPT-4.1, R2X matches R2R at 92% success while reducing action steps (108 vs 116), path length (88 m vs 99 m), and LLM tokens (42,438 vs 47,875), outperforming isolated robots (66%) and adapted SMART-LLM/EMOS baselines (88%). Ablations cover LLM scale, IoT latency, team size, and omission vs semantic corruption; a qualitative Stretch + webcam deployment is also reported.

Significance. If the efficiency and reliability claims hold under realistic IoT noise, the work is a useful step for indoor multi-robot systems: it formalizes an R2X channel that existing indoor LLM multi-robot benchmarks largely omit, provides a configurable partial-observability testbed, and shows that infrastructure priors can convert search into routing without sacrificing success. Strengths include an explicit world-model state (Eqs. 1–5), a checkable coordination loop (Algorithm 1), controlled IR/R2R/R2X ablations, and an honest omission-vs-corruption analysis (Fig. 4) that identifies semantic misinformation as the fragile failure mode. The real Stretch demo, though qualitative, supports transfer of the sensing concept beyond pure simulation.

major comments (4)
  1. [Table II, Sec. III–IV, Eqs. (5)–(6), Fig. 4] Table II / Sec. III–IV / Eqs. (5)–(6): The headline efficiency claim (R2X matches R2R success while cutting steps, path, and tokens) treats VLM-derived CCTV logs and device status as actionable priors in W_t. Fig. 4 (right) shows incorrect semantic status linearly reduces success, so the main-protocol gains rest on IoT semantics being mostly correct, not merely present. The manuscript does not report VLM (Qwen-VL) accuracy, false-positive/false-negative rates, or any noise model for the CCTV event logs used in Table II. Without that characterization—or a main-result condition with measured VLM error—the efficiency delta is hard to interpret as a pure R2X benefit rather than a clean-prior benefit. Please quantify IoT/VLM semantic error under the default protocol and restate the claim accordingly.
  2. [Tables II–IV, Figs. 3–4, Sec. V] Tables II–IV and Figs. 3–4 report point estimates only (success, steps, path, tokens) with no number of episodes per cell, variance, confidence intervals, or significance tests. For a systems claim that R2X improves efficiency over R2R by ~7–11% while holding success fixed, and that latency/team-size trends are meaningful, statistical reporting is load-bearing for reproducibility and for judging whether the R2X–R2R gap is stable. Please add n, dispersion, and a brief statistical protocol for the primary comparisons.
  3. [Table II, Sec. V.C] Table II claims superiority over SMART-LLM and EMOS “adapted to our setting” (unknown target positions, partial observability). The adaptation procedure, observation interface, replan policy, and whether those systems received the same world-model schema or only R2R maps are not specified in enough detail to verify fairness. Because the 4% success and large path/token gaps vs. baselines are used to position IndoorR2X, the adaptation protocol needs to be explicit (or the comparison limited to the paper’s own IR/R2R/R2X ablations).
  4. [Sec. V.I, Contributions] Sec. V.I and the contributions list “real-world validation,” but the Stretch experiment is a single qualitative scenario with no success/path/time comparison against R2R or IR, and no report of webcam/VLM error. This does not yet support the contribution wording. Either add quantitative real-world metrics under the same protocols, or narrow the claim to a feasibility demonstration.
minor comments (6)
  1. [Abstract, Sec. II] Abstract and introduction call IndoorR2X “the first” indoor R2X LLM multi-robot benchmark; Table I supports novelty relative to listed families, but a short related-work caveat on other robot–IoT fusion systems (even if non-LLM) would reduce overclaim risk.
  2. [Sec. IV.A, Eq. (1)] Eq. (1) defines W_t = (S^R_t, S^O_t, S^A_t) but area/room state S^A_t is never expanded analogously to Eqs. (2)–(3); a brief definition would help implementers.
  3. [Fig. 3, Sec. V.F] Fig. 3 caption says success is stable up to N=5, but the text also says >90% up to five robots; please align axis labels, exact N values, and whether metrics are totals or per-robot.
  4. [Sec. III] CCTV coverage is “approximately 50%” (Sec. III) but is listed as a free parameter; a sensitivity sweep on coverage fraction would strengthen the infrastructure claim and is currently missing.
  5. [Throughout] Minor consistency: arXiv header shows v3 / 31 Mar 2026 while affiliations and project URL are fine; ensure camera-ready version freezes versioned links and model names (GPT-4.1, Gemma-3-27b, Llama-3.1-8b).
  6. [Fig. 5] Fig. 5 is dense; a short timeline or numbered step list matching the DAG would make the qualitative rollout easier to follow.

Circularity Check

0 steps flagged

No circular derivation: empirical systems paper measures IR/R2R/R2X outcomes under defined protocols; claims are not forced by construction or self-citation chains.

full rationale

IndoorR2X is a benchmark-and-framework paper. Its load-bearing claims (Table II: R2X matches R2R at 92% success while cutting action steps, path length, and LLM tokens vs IR and adapted SMART-LLM/EMOS) are experimental outcomes under fixed protocols (partial observability, ~50% CCTV coverage, VLM event logs fused via Eq. 5, LLM plans via Eq. 6), not algebraic identities or fitted parameters renamed as predictions. The world model W_t = (S^R_t, S^O_t, S^A_t) and fusion F are design choices evaluated against ablations (IR/R2R/R2X, LLM scale, latency, team size, omission vs corruption); they do not define success rate or path length in terms of themselves. Citations to V2X, SMART-LLM, EMOS, AI2-THOR, etc. supply background and baselines, not uniqueness theorems or ansätze that force the headline result. Defining a new benchmark and reporting metrics on it is standard systems practice, not circularity under the stated criteria. No self-definitional loop, fitted-input-as-prediction, load-bearing self-citation chain, or renaming of a known result was found. (IoT semantic trustworthiness is a validity/robustness concern, not circularity.)

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 3 invented entities

The central efficiency claim rests on a simulation-and-systems stack: AI2-THOR physics and object semantics, a strict partial-observability policy, ~50% random CCTV coverage, VLM textification of video, a hand-designed global semantic state and fusion rules, and an LLM planner whose outputs are assumed executable after light validation. Free parameters are mostly experimental knobs (coverage fraction, team size, latency, corruption rates), not curve-fit constants. Invented entities are engineering constructs (benchmark, hub state, R2X channel), not new physical particles; independent evidence is partial via the real-robot demo and public simulators.

free parameters (5)
  • CCTV area coverage fraction = ~50%
    Benchmark randomly configures static cameras to cover approximately 50% of house area (Sec. III); this hand-chosen coverage level directly affects how much exploration R2X can skip.
  • Default robot team size N = 3 (main results)
    Main tables use three robots; scalability sweep N=2..6 (Fig. 3). Team size is an experimental control that changes coordination overhead.
  • IoT latency t_delay = 0 / 5 / 10 steps
    Artificial delay steps {0,5,10} injected before fusion (Table IV); chosen to probe staleness, not derived.
  • MAX_FAILS replan budget
    Algorithm 1 terminates after MAX_FAILS; value is a system hyperparameter affecting success vs early abort.
  • IoT omission/corruption rates in robustness ablations = 0–100% (swept)
    Fig. 4 sweeps missing detections and incorrect semantic status; rates are experimenter-set stress tests.
axioms (6)
  • domain assumption Each robot’s knowledge is limited to its current field of view and previously visited areas (strict partial observability).
    Stated as the key differentiator of the benchmark (Sec. III); without it, IoT value collapses because planners could already be omniscient.
  • domain assumption AI2-THOR / ArchitecTHOR / RoboTHOR dynamics, object properties, and action outcomes are an adequate proxy for indoor multi-robot household tasks.
    Entire evaluation (85 scenes, 850 tasks) runs in these engines (Sec. III, V).
  • domain assumption VLM (e.g., Qwen-VL) conversion of CCTV video into time-stamped text event logs preserves planning-relevant semantics.
    Sec. III and real-world section route camera feeds through VLM-to-text before hub fusion.
  • domain assumption A single centralized LLM can produce valid multi-agent dependency-graph plans from serialized task+world-state prompts, with orchestrator matching handling assignment.
    Core of Sec. IV planning (Eq. 6) and Algorithm 1; Table III shows this axiom fails for small models.
  • ad hoc to paper World-model transition F correctly updates nested receptacle/inventory topology after Pick/Put and merges multi-source observations.
    Defined in Sec. IV-A (Eqs. 1–5); correctness of later plans depends on this bookkeeping.
  • ad hoc to paper Adapted SMART-LLM and EMOS baselines under unknown target positions are fair representatives of prior R2R LLM coordination.
    Table II comparison depends on these adaptations (Sec. V-C); details of adaptation fidelity are limited in the text.
invented entities (3)
  • IndoorR2X benchmark (85 scenes, configurable IoT layouts, task suites, partial-observability protocol) independent evidence
    purpose: Provide a controlled indoor testbed for multi-robot LLM planning with Robot-to-Everything sensing.
    Authors construct the suite on top of existing simulators; novelty is the R2X evaluation protocol, not a new physical law.
  • Coordination hub global semantic state W_t = (S^R_t, S^O_t, S^A_t) with R2X fusion F no independent evidence
    purpose: Unify robot and IoT observations into a shared belief for LLM planning and replanning.
    Engineering representation introduced in Sec. IV-A; evaluated only inside this framework.
  • Indoor R2X information channel (static IoT/CCTV + device status as “X”) independent evidence
    purpose: Name and operationalize infrastructure sensing as a first-class complement to R2R under partial observability.
    Conceptual transfer from outdoor V2X; independent evidence is partial via real cameras in the Stretch demo and cited IoT literature.

pith-pipeline@v1.1.0-grok45 · 17870 in / 4403 out tokens · 49775 ms · 2026-07-13T21:37:35.553971+00:00 · methodology

0 comments
read the original abstract

Although robot-to-robot (R2R) communication improves indoor scene understanding beyond what a single robot can achieve, R2R alone cannot overcome partial observability without substantial exploration overhead or scaling team size. In contrast, many indoor environments already include low-cost Internet of Things (IoT) sensors (e.g., cameras) that provide persistent, building-wide context beyond onboard perception. We therefore introduce IndoorR2X, a benchmark and simulation framework for Large Language Model (LLM)-driven multi-robot task planning with Robot-to-Everything (R2X) perception and communication in indoor environments. IndoorR2X integrates observations from mobile robots and static IoT devices to construct a global semantic state that supports scalable scene understanding, reduces redundant exploration, and enables high-level coordination through LLM-based planning. IndoorR2X provides configurable simulation environments, sensor layouts, robot teams, and task suites to systematically evaluate semantic-level coordination strategies. Extensive experiments across diverse settings demonstrate that IoT-augmented world modeling improves multi-robot efficiency and reliability, and we highlight key insights and failure modes for advancing LLM-based collaboration between robot teams and indoor IoT sensors. Project page: https://fandulu.github.io/IndoorR2X_project_page/.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 11 linked inside Pith

  1. [1]

    Housekeep: Tidying virtual households using commonsense reasoning,

    Y . Kant, A. Ramachandran, S. Yenamandra, I. Gilitschenski, D. Batra, A. Szot, and H. Agrawal, “Housekeep: Tidying virtual households using commonsense reasoning,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 355–373

    2022

  2. [2]

    Smart-llm: Smart multi-agent robot task planning using large language models,

    S. S. Kannan, V . L. Venkatesh, and B.-C. Min, “Smart-llm: Smart multi-agent robot task planning using large language models,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024, pp. 12 140–12 147

    2024

  3. [3]

    Lamma-p: Gen- eralizable multi-agent long-horizon task allocation and planning with lm-driven pddl planner,

    X. Zhang, H. Qin, F. Wang, Y . Dong, and J. Li, “Lamma-p: Gen- eralizable multi-agent long-horizon task allocation and planning with lm-driven pddl planner,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 10 221–10 221

    2025

  4. [4]

    Online multi-robot coordination and cooperation with task precedence relationships,

    W. Gosrich, S. Agarwal, K. Garg, S. Mayya, M. Malencia, M. Yim, and V . Kumar, “Online multi-robot coordination and cooperation with task precedence relationships,”IEEE Transactions on robotics, 2025

    2025

  5. [5]

    An integrated semantic framework for designing context- aware internet of robotic things systems,

    L. Sabri, S. Bouznad, S. Rama Fiorini, A. Chibani, E. Prestes, and Y . Amirat, “An integrated semantic framework for designing context- aware internet of robotic things systems,”Integrated Computer-Aided Engineering, vol. 25, no. 2, pp. 137–156, 2018

    2018

  6. [6]

    Robot-enabled support of daily activities in smart home environments,

    G. Wilson, C. Pereyda, N. Raghunath, G. De La Cruz, S. Goel, S. Nesaei, B. Minor, M. Schmitter-Edgecombe, M. E. Taylor, and D. J. Cook, “Robot-enabled support of daily activities in smart home environments,”Cognitive systems research, vol. 54, pp. 258–272, 2019

    2019

  7. [7]

    Internet of robotic things in smart domains: Applications and challenges,

    L. Romeo, A. Petitti, R. Marani, and A. Milella, “Internet of robotic things in smart domains: Applications and challenges,”Sensors, vol. 20, no. 12, p. 3355, 2020

    2020

  8. [8]

    Sensor data fusion for optimal robotic navigation using regression based on an iot system,

    V . V . Aroulanandam, P. Sherubha, K. Lalitha, J. Hymavathi, R. Thia- garajan,et al., “Sensor data fusion for optimal robotic navigation using regression based on an iot system,”Measurement: Sensors, vol. 24, p. 100598, 2022

    2022

  9. [9]

    V2x- sim: Multi-agent collaborative perception dataset and benchmark for autonomous driving,

    Y . Li, D. Ma, Z. An, Z. Wang, Y . Zhong, S. Chen, and C. Feng, “V2x- sim: Multi-agent collaborative perception dataset and benchmark for autonomous driving,”IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 10 914–10 921, 2022

    2022

  10. [10]

    V2x-real: a largs-scale dataset for vehicle-to-everything cooperative perception,

    H. Xiang, Z. Zheng, X. Xia, R. Xu, L. Gao, Z. Zhou, X. Han, X. Ji, M. Li, Z. Meng,et al., “V2x-real: a largs-scale dataset for vehicle-to-everything cooperative perception,” inEuropean Conference on Computer Vision. Springer, 2024, pp. 455–470

    2024

  11. [11]

    V2x-llm: Enhancing v2x integration and understanding in connected vehicle corridors,

    K. Wu, P. Li, Y . Zhou, R. Gan, J. You, Y . Cheng, J. Zhu, S. T. Parker, B. Ran, D. A. Noyce,et al., “V2x-llm: Enhancing v2x integration and understanding in connected vehicle corridors,”arXiv preprint arXiv:2503.02239, 2025

    Pith/arXiv arXiv 2025

  12. [12]

    Accidentgpt: Accident analysis and prevention from v2x environmental perception with multi-modal large model,

    L. Wang, Y . Ren, H. Jiang, P. Cai, D. Fu, T. Wang, Z. Cui, H. Yu, X. Wang, H. Zhou,et al., “Accidentgpt: Accident analysis and prevention from v2x environmental perception with multi-modal large model,”arXiv preprint arXiv:2312.13156, 2023

    Pith/arXiv arXiv 2023

  13. [13]

    Twostep: Multi-agent task planning using classical planners and large language models,

    D. Bai, I. Singh, D. Traum, and J. Thomason, “Twostep: Multi-agent task planning using classical planners and large language models,” arXiv preprint arXiv:2403.17246, 2024

    Pith/arXiv arXiv 2024

  14. [14]

    Heterogeneous embodied multi-agent collaboration,

    X. Liu, D. Guo, X. Zhang, and H. Liu, “Heterogeneous embodied multi-agent collaboration,”IEEE Robotics and Automation Letters, vol. 9, no. 6, pp. 5377–5384, 2024

    2024

  15. [15]

    EMOS: Embodiment-aware heterogeneous multi-robot operating system with LLM agents,

    J. Chen, C. Yu, X. Zhou, T. Xu, Y . Mu, M. Hu, W. Shao, Y . Wang, G. Li, and L. Shao, “EMOS: Embodiment-aware heterogeneous multi-robot operating system with LLM agents,” inThe Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://openreview.net/forum?id=Ey8KcabBpB

    2025

  16. [16]

    Vehicle-to-everything (v2x) in the autonomous vehicles domain–a technical review of communica- tion, sensor, and ai technologies for road user safety,

    S. A. Yusuf, A. Khan, and R. Souissi, “Vehicle-to-everything (v2x) in the autonomous vehicles domain–a technical review of communica- tion, sensor, and ai technologies for road user safety,”Transportation Research Interdisciplinary Perspectives, vol. 23, p. 100980, 2024

    2024

  17. [17]

    A dynamic priority-based batch verification scheme for v2x communication in vehicular networks*,

    Y . Yang, H. Yu, X. Fu, Y . Ren, Y . Zhao, and Y . Shi, “A dynamic priority-based batch verification scheme for v2x communication in vehicular networks*,” in2025 IEEE Intelligent Vehicles Symposium (IV), 2025, pp. 781–786

    2025

  18. [18]

    Integrating llms with its: Recent advances, potentials, challenges, and future directions,

    D. Mahmud, H. Hajmohamed, S. Almentheri, S. Alqaydi, L. Aldha- heri, R. A. Khalil, and N. Saeed, “Integrating llms with its: Recent advances, potentials, challenges, and future directions,”IEEE Trans- actions on Intelligent Transportation Systems, 2025

    2025

  19. [19]

    Umbrella collab- orative robotics testbed and iot platform,

    T. Farnham, S. Jones, A. Aijaz, Y . Jin, I. Mavromatis, U. Raza, A. Portelli, A. Stanoev, and M. Sooriyabandara, “Umbrella collab- orative robotics testbed and iot platform,” in2021 IEEE 18th An- nual Consumer Communications & Networking Conference (CCNC). IEEE, 2021, pp. 1–7

    2021

  20. [20]

    Enhancing robots navigation in internet of things indoor systems,

    Y . Tashtoush, I. Haj-Mahmoud, O. Darwish, M. Maabreh, B. Als- inglawi, M. Elkhodr, and N. Alsaedi, “Enhancing robots navigation in internet of things indoor systems,”Computers, vol. 10, no. 11, p. 153, 2021

    2021

  21. [21]

    Is secure communication in the r2i (robot- to-infrastructure) model possible? identification of threats,

    K. Krzykowska-Piotrowska, E. Dudek, M. Siergiejczyk, A. Rosi ´nski, and W. Wawrzy ´nski, “Is secure communication in the r2i (robot- to-infrastructure) model possible? identification of threats,”Energies, vol. 14, no. 15, p. 4702, 2021

    2021

  22. [22]

    Large language models for multi- robot systems: A survey,

    P. Li, Z. An, S. Abrar, and L. Zhou, “Large language models for multi- robot systems: A survey,”arXiv preprint arXiv:2502.03814, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    Llm- mars: Large language model for behavior tree generation and nlp- enhanced dialogue in multi-agent robot systems,

    A. Lykov, M. Dronova, N. Naglov, M. Litvinov, S. Satsevich, A. Bazhenov, V . Berman, A. Shcherbak, and D. Tsetserukou, “Llm- mars: Large language model for behavior tree generation and nlp- enhanced dialogue in multi-agent robot systems,”arXiv preprint arXiv:2312.09348, 2023

    Pith/arXiv arXiv 2023

  24. [24]

    Roco: Dialectic multi-robot col- laboration with large language models,

    Z. Mandi, S. Jain, and S. Song, “Roco: Dialectic multi-robot col- laboration with large language models,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 286–299

    2024

  25. [25]

    Coherent: Collaboration of heterogeneous multi-robot system with large language models,

    K. Liu, Z. Tang, D. Wang, Z. Wang, X. Li, and B. Zhao, “Coherent: Collaboration of heterogeneous multi-robot system with large language models,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 10 208–10 214

    2025

  26. [26]

    Collabot: Vision-language guided simultaneous collaborative manipulation,

    K. Song, S. Ma, G. Chen, N. Jin, G. Zhao, M. Ding, Z. Xiong, and J. Pan, “Collabot: Vision-language guided simultaneous collaborative manipulation,”arXiv preprint arXiv:2508.03526, 2025

    Pith/arXiv arXiv 2025

  27. [27]

    Dexter-llm: Dynamic and explainable coordination of multi-robot systems in unknown envi- ronments via large language models,

    Y . Zhu, J. Chen, X. Zhang, M. Guo, and Z. Li, “Dexter-llm: Dynamic and explainable coordination of multi-robot systems in unknown envi- ronments via large language models,” in2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2025, pp. 10 182–10 189

    2025

  28. [28]

    Vader: Visual affordance detection and error recovery for multi robot human collaboration,

    M. Ahn, M. G. Arenas, M. Bennice, N. Brown, C. Chan, B. David, A. Francis, G. Gonzalez, R. Hessmer, T. Jackson,et al., “Vader: Visual affordance detection and error recovery for multi robot human collaboration,”arXiv preprint arXiv:2405.16021, 2024

    Pith/arXiv arXiv 2024

  29. [29]

    Elhplan: Effi- cient long-horizon task planning for multi-agent collaboration,

    S. Ling, Y . Wang, C. Fan, T. L. Lam, and J. Hu, “Elhplan: Effi- cient long-horizon task planning for multi-agent collaboration,”arXiv preprint arXiv:2509.24230, 2025

    arXiv 2025

  30. [30]

    Long-horizon planning for multi-agent robots in partially observable environments,

    S. Nayak, A. Morrison Orozco, M. Have, J. Zhang, V . Thirumalai, D. Chen, A. Kapoor, E. Robinson, K. Gopalakrishnan, J. Harrison, et al., “Long-horizon planning for multi-agent robots in partially observable environments,”Advances in Neural Information Processing Systems, vol. 37, pp. 67 929–67 967, 2024

    2024

  31. [31]

    Planning and acting in partially observable stochastic domains,

    L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, “Planning and acting in partially observable stochastic domains,”Artificial Intelli- gence, vol. 101, no. 1–2, pp. 99–134, 1998

    1998

  32. [32]

    Coor- dinated multi-robot exploration,

    W. Burgard, M. Moors, C. Stachniss, and F. E. Schneider, “Coor- dinated multi-robot exploration,” inIEEE Transactions on Robotics, vol. 21, no. 3. IEEE, 2005, pp. 376–386

    2005

  33. [33]

    Frontier-based exploration using multiple robots,

    B. Yamauchi, “Frontier-based exploration using multiple robots,” in Proceedings of the Second International Conference on Autonomous Agents, 1998, pp. 47–53

    1998

  34. [34]

    Composi- tional coordination for multi-robot teams with large language models,

    Z. Huang, G. Shi, Y . Wu, V . Kumar, and G. S. Sukhatme, “Composi- tional coordination for multi-robot teams with large language models,” arXiv preprint arXiv:2507.16068, 2025

    arXiv 2025

  35. [35]

    Dynamic task adaptation for multi-robot manufacturing systems with large language models,

    J. Lim and I. Kovalenko, “Dynamic task adaptation for multi-robot manufacturing systems with large language models,”arXiv preprint arXiv:2505.22804, 2025

    Pith/arXiv arXiv 2025

  36. [36]

    AI2-THOR: An Interactive 3D Environment for Visual AI,

    E. Kolve, R. Mottaghi, W. Han, E. VanderBilt, L. Weihs, A. Herrasti, D. Gordon, Y . Zhu, A. Gupta, and A. Farhadi, “AI2-THOR: An Interactive 3D Environment for Visual AI,”arXiv, 2017

    2017

  37. [37]

    ProcTHOR: Large-Scale Embodied AI Using Procedural Generation,

    M. Deitke, E. VanderBilt, A. Herrasti, L. Weihs, J. Salvador, K. Ehsani, W. Han, E. Kolve, A. Farhadi, A. Kembhavi, and R. Mottaghi, “ProcTHOR: Large-Scale Embodied AI Using Procedural Generation,” inNeurIPS, 2022, outstanding Paper Award

    2022

  38. [38]

    Robothor: An open simulation-to-real embodied ai platform,

    M. Deitke, W. Han, A. Herrasti, A. Kembhavi, E. Kolve, R. Mottaghi, J. Salvador, D. Schwenk, E. VanderBilt, M. Wallingford,et al., “Robothor: An open simulation-to-real embodied ai platform,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 3164–3174. 8

    2020

  39. [39]

    Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond,

    J. Bai, S. Bai, S. Yang, S. Wang, S. Tan, P. Wang, J. Lin, C. Zhou, and J. Zhou, “Qwen-vl: A versatile vision-language model for un- derstanding, localization, text reading, and beyond,”arXiv preprint arXiv:2308.12966, 2023

    Pith/arXiv arXiv 2023

  40. [40]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat,et al., “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  41. [41]

    Gemma 3 technical report,

    G. Team, A. Kamath, J. Ferret, S. Pathak, N. Vieillard, R. Merhej, S. Perrin, T. Matejovicova, A. Ramé, M. Rivière,et al., “Gemma 3 technical report,”arXiv preprint arXiv:2503.19786, 2025

    Pith/arXiv arXiv 2025

  42. [42]

    The llama 3 herd of models,

    A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan,et al., “The llama 3 herd of models,”arXiv e-prints, pp. arXiv–2407, 2024

    2024

  43. [43]

    The design of stretch: A compact, lightweight mobile manipulator for indoor human environments,

    C. C. Kemp, A. Edsinger, H. M. Clever, and B. Matulevich, “The design of stretch: A compact, lightweight mobile manipulator for indoor human environments,” in2022 International Conference on Robotics and Automation (ICRA). IEEE, 2022, pp. 3150–3157. 9

    2022