Pith. sign in

REVIEW 3 major objections 4 minor 45 references

This paper claims that an LLM robotic planner becomes more reliable under ambiguity when it iteratively queries a scene-graph representation instead of consuming a flat observation, letting it ask targeted clarification questions rather tha

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 17:53 UTC pith:73HLX73P

load-bearing objection The framework has a real core, but the evaluation metrics are too weak to support the headline claims, and Table II directly contradicts the abstract's consistent 10% CQR improvement. the 3 major comments →

arxiv 2603.18271 v3 pith:73HLX73P submitted 2026-03-18 cs.RO

SG-CoT: An Ambiguity-Aware Robotic Planning Framework using Scene Graph Representations

Akshat Rana , Peeyush Agarwal , K.P.S. Rana , Amarjit Malhotra This is my paper
classification cs.RO
keywords scene graphslarge language modelsrobot task planningambiguity resolutionclarification questionspartial observabilitychain-of-thought reasoningsimulated tabletop environments
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that an LLM-based robot planner becomes more reliable in ambiguous and partially observed environments when, instead of processing a single flat observation, it iteratively queries a structured scene-graph representation of the scene. The core idea is that grounding each reasoning step in a small, task-relevant slice of the graph lets the model detect when an instruction has multiple valid interpretations—duplicate objects, missing objects, vague wording, or out-of-view objects—and ask a targeted clarifying question rather than guess. The authors report consistent gains over three prior methods: at least 10% higher correct-question rate, and success-rate increases of 4% in single-agent and 15% in multi-agent environments. If these numbers hold, selective scene-graph retrieval is a practical way to make LLM planners safer in realistic settings.

Core claim

The paper claims that ambiguity in robot task planning is best treated as a grounding problem rather than an uncertainty-estimation problem. SG-CoT builds a scene graph—objects as nodes with attributes, edges as semantic/spatial relations—from a single observation, then lets the LLM iteratively call retrieve_node and retrieve_edge to inspect only task-relevant parts of the graph. The loop of reasoning, retrieval, and reflection continues until the model gives an action sequence or asks a clarification question. In simulation, it outperforms three prior baselines: at least 10% higher correct-question rate, and 4% (single-agent) / 15% (multi-agent) higher success rate. The claimed mechanism is

What carries the argument

The central mechanism is the scene graph, G=(V,E), where nodes are detected objects with attribute sets and edges are open-vocabulary spatial or semantic relations, together with the two retrieval functions retrieve_node and retrieve_edge. Instead of feeding the entire scene description to the LLM, SG-CoT allows the model to query the graph iteratively, so each reasoning step is checked against a small, relevant slice of the environment. This selective grounding—the 'chain-of-thought' over the graph—is what the authors credit with preventing hallucination and enabling the model to locate the source of ambiguity and phrase a clarifying question.

Load-bearing premise

The load-bearing premise is that, in ambiguous trials, success is scored as 1 whenever the robot asks any clarification question, and question correctness is judged by whether the LLM's self-reported tag matches the perturbation type; if success instead required the question to be genuinely informative and to lead to task completion after the user answers, the reported margins over baselines might shrink.

What would settle it

In the paper's own 400-trial single-agent and multi-agent setups, replace the ambiguous-trial success criterion with a stricter one: the robot must ask a question that a human judge deems informative, and the task must succeed after the user answers. If SG-CoT's success-rate and correct-question-rate lead over the best prior baselines collapses under this criterion, the central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • LLM planners should query structured environment representations iteratively rather than consume complete scene descriptions in one pass.
  • Asking for clarification becomes a first-class planning action: a robot can identify the missing piece of information and ask for exactly that, reducing risky guessing.
  • The framework extends to multi-robot cooperation under partial observability: an agent that cannot see a target can ask the other robot for its local scene-graph information.
  • The formal classification of ambiguity into multiplicity, absence, underspecification, and observation-level types gives a vocabulary for benchmarking future ambiguity-aware planners.
  • Performance scales with LLM capacity (the larger model outperforms the smaller one), indicating the method benefits from stronger models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A stricter success criterion—requiring that the clarification question be genuinely informative (e.g., judged by a human, or followed by successful task completion once answered)—could determine whether the reported gains reflect true ambiguity resolution or a bias toward asking questions.
  • The same scene-graph grounding mechanism may transfer to other LLM-agent settings where hallucination comes from missing or false environmental context, such as domestic assistants with partial sensor coverage.
  • Because the VLM builds the scene graph in a single step, injecting controlled edge errors into the graph would quantify how much of the planner's benefit survives imperfect perception; the paper identifies this as a limitation but does not measure the sensitivity.
  • A testable extension is to measure retrieval steps and token cost as a function of scene size and ambiguity count, since the paper claims latency scales with instruction complexity rather than object count.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes SG-CoT, a two-stage framework for LLM-based robotic planning under ambiguity. In the first stage, a vision-language model builds a scene graph of observed objects, attributes, and relations. In the second stage, the LLM iteratively queries this scene graph through retrieval functions before committing to an action sequence or asking a clarification question. The framework is evaluated on simulated single-agent tabletop tasks with three injected ambiguity types (multiplicity, absence, underspecification) and on partially observable multi-agent tasks. The central claims are that SG-CoT consistently outperforms CLARA, ProgPrompt, and Inner Monologue, with a minimum 10% improvement in question accuracy (CQR) and a minimum 4% (single-agent) and 15% (multi-agent) improvement in success rate (SR).

Significance. If the claims were supported, the work would be a useful contribution: it extends ambiguity handling beyond instruction underspecification to environment-induced ambiguities, uses scene graphs as a structured grounding mechanism for LLM planners, and proposes a clarification-question evaluation metric. The scene-graph retrieval loop is a sensible way to ground iterative reasoning. However, the reported evaluation metrics do not measure what the claims require. The success metric for ambiguous trials counts any clarification request as success, and the question-accuracy metric checks only a self-reported tag against the injected perturbation type. Moreover, the abstract's 'consistently outperforms' and 'minimum 10% improvement' statements are directly contradicted by the paper's own Table II for the multiplicity condition. The paper does not provide code or data, and the quantitative evidence, as presented, is not sufficient to establish the claimed effectiveness.

major comments (3)
  1. [§IV-A] The success metric for ambiguous trials is defined as 'SR is 1 when the robot asks for clarification, and 0 otherwise.' This rewards any clarification request regardless of relevance, informativeness, or answerability. A vacuous question such as 'What?' would receive full credit. The later statement that 'the goal condition is considered satisfied if the correct clarification question is generated' introduces a different, stricter criterion that is not operationalized and conflicts with the metric definition. Consequently, the reported SR improvements (4% single-agent, 15% multi-agent) do not establish that SG-CoT asks useful questions; they may only reflect a higher propensity to ask.
  2. [§IV-A] CQR is computed by comparing a tag that the LLM itself outputs with the ground-truth perturbation type. This checks whether the model's self-reported category matches the injected ambiguity class; it does not evaluate the content of the question or whether the question would actually resolve the ambiguity. A system that tags correctly while asking an uninformative question would score full CQR. CQR is therefore partly a self-consistency measure rather than an externally grounded measure of clarification quality. An outcome-based evaluation (e.g., task success after the user answers the question) or a human-judgment study is needed to support the paper's central claim about question quality.
  3. [Table II, §IV-B, Abstract] The abstract states that 'SG-CoT consistently outperforms prior methods, with a minimum of 10% improvement in question accuracy' and that SG-CoT 'consistently outperforms' baselines. Table II contradicts this in the multiplicity condition: SG-CoT CQR is 0.52 vs. InnerMono's 0.53 under Qwen, and 0.51 vs. 0.62 under Gemini. The 'minimum 10% improvement' holds only for the overall row, not per ambiguity type, and it is not even an improvement in multiplicity. This is an internal inconsistency between the paper's headline claim and its own reported results, and it undermines the claim of consistent superiority.
minor comments (4)
  1. [Table I] The underspecified instruction template contains a malformed token '[V AGUE]' (likely intended to be '[VAGUE]' or a placeholder). Please fix.
  2. [§III-C] The retrieval function signatures in the examples (e.g., retrieve_node(type='bowl', color='red')) are not formally specified in the methodology. The prompt schema lists 'nodes' and 'edges' but not the function-call format. Clarify the exact API the LLM is expected to emit.
  3. [§IV-B] No variance, confidence intervals, or significance tests are reported for any metric. Given that each condition uses 100 trials and several differences are only a few percentage points, it is difficult to assess whether the reported gains are statistically meaningful.
  4. [§V] The limitations paragraph appropriately acknowledges the absence of real-world experiments and the risk of VLM hallucination in scene-graph construction. These limitations are consistent with the concerns raised above and should be addressed in a revision.

Circularity Check

0 steps flagged

No structural circularity: SG-CoT is an empirical pipeline whose reported gains are not derived from its own definitions or fitted parameters.

full rationale

The paper proposes a two-stage system (scene graph generation + iterative LLM retrieval) and evaluates it against external baselines. There is no derivation chain in which an output is defined as an input: no parameter is fitted to a subset and then reported as a prediction; no uniqueness theorem from the authors' prior work is invoked; no ansatz is smuggled in via self-citation. The only potentially self-referential element is the CQR metric (Section IV-A), which scores the LLM's self-reported ambiguity tag against the ground-truth perturbation type; this is a weak external-validity property, but it is a measurement protocol, not a logical reduction, and it does not by construction force SG-CoT's advantage. The SR definition for ambiguous trials (any clarification question counts as success, Section IV-A) is similarly a lenient evaluation choice, not a circular derivation. The paper's own limitations passage (Section V) concedes VLM hallucination risk and simulation-only evaluation, which are correctness concerns, not circularity. The Table II inconsistency with the abstract's 'minimum of 10%' claim is an internal quantitative inconsistency, not a circularity. Therefore no circular step meets the quoted-reduction threshold, and the appropriate score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The framework introduces no physical or mathematical entities; the ambiguity-tag taxonomy is a labeling scheme rather than an independent postulate. The load-bearing assumptions are about perception accuracy, the exhaustiveness of the taxonomy, and the validity of the evaluation metrics.

free parameters (4)
  • Scene-graph schema vocabulary
    The set of attribute types (color, size, is_broken, is_open) and edge types (on_top_of, inside_of) in the initial prompt P0 is hand-chosen; it bounds what retrieval functions can find and therefore what ambiguities are detectable.
  • max_turns
    Termination limit in Algorithm 1; not reported in experiments, affects outcomes when the model is forced to conclude after too few or too many reasoning steps.
  • vague-word dictionary
    Curated set of vague terms used to construct underspecification trials; not released, directly shapes the difficulty of that category.
  • Ground-truth ambiguity tags
    The three/four taxonomy tags are defined by the authors and used both to construct perturbations and to score CQR; they are not externally benchmarked.
axioms (5)
  • domain assumption VLM-generated scene graph is an accurate-enough proxy for the environment.
    Section III-B constructs G from a VLM; the Conclusion admits VLMs 'can hallucinate ... and introduce incorrect edges', so correctness of the graph is assumed rather than guaranteed.
  • domain assumption LLM tool calls are correctly parsed and executed.
    Algorithm 1 assumes ft is intercepted and Rt returned without error; no error handling for malformed calls is described.
  • ad hoc to paper The defined ambiguity taxonomy exhausts the ambiguities a planner must handle.
    Section III-A defines absence, multiplicity, underspecification, and observation-level ambiguities; experiments only test these categories, so generalization to other ambiguity types is unassessed.
  • domain assumption Asking a clarification question is sufficient for task success in ambiguous trials.
    Section IV-B: 'once the appropriate clarification is obtained from the user, the ambiguity is resolved and the instruction can be successfully executed' — this justifies scoring SR=1 for any ask, but in practice users may not resolve it.
  • domain assumption A self-reported ambiguity tag is a valid proxy for a correct clarifying question.
    Section IV-A defines CQR as tag-matching; it does not evaluate whether the question text would actually disambiguate the situation.

pith-pipeline@v1.3.0-alltime-deepseek · 11576 in / 14975 out tokens · 142913 ms · 2026-08-02T17:53:41.777404+00:00 · methodology

0 comments
read the original abstract

Ambiguity poses a major challenge to large language models (LLMs) used as robotic planners. In this letter, we present Scene Graph-Chain-of-Thought (SG-CoT), a two-stage framework where LLMs iteratively query a scene graph representation of the environment to detect and clarify ambiguities. First, a structured scene graph representation of the environment is constructed from input observations, capturing objects, their attributes, and relationships with other objects. Second, the LLM is equipped with retrieval functions to query portions of the scene graph that are relevant to the provided instruction. This grounds the reasoning process of the LLM in the observation, increasing the reliability of robotic planners under ambiguous situations. SG-CoT also allows the LLM to identify the source of ambiguity and pose a relevant disambiguation question to the user or another robot. Extensive experimentation demonstrates that SG-CoT consistently outperforms prior methods, with a minimum of 10% improvement in question accuracy and a minimum success rate increase of 4% in single-agent and 15% in multi-agent environments, validating its effectiveness for more generalizable robot planning.

Figures

Figures reproduced from arXiv: 2603.18271 by Akshat Rana, Amarjit Malhotra, K.P.S. Rana, Peeyush Agarwal.

Figure 1
Figure 1. Figure 1: The figure shows possible configurations of the defined ambiguity types. For each category, the image of the left is the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: SG-CoT. The robot receives an instruction from the user and an observation from the environment. The observation is [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 1 canonical work pages

  1. [1]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control,

    A. Brohan, N. Brown, J. Carbajalet al., “Rt-2: Vision-language-action models transfer web knowledge to robotic control,” inarXiv preprint arXiv:2307.15818, 2023

    Pith/arXiv arXiv 2023

  2. [2]

    Navgpt: Explicit reasoning in vision- and-language navigation with large language models,

    G. Zhou, Y . Hong, and Q. Wu, “Navgpt: Explicit reasoning in vision- and-language navigation with large language models,”arXiv preprint arXiv:2305.16986, 2023

    Pith/arXiv arXiv 2023

  3. [3]

    Mapgpt: Map-guided prompting with adaptive path planning for vision-and-language navigation,

    J. Chen, B. Lin, R. Xuet al., “Mapgpt: Map-guided prompting with adaptive path planning for vision-and-language navigation,” inProceed- ings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024

    2024

  4. [4]

    Do as i can and not as i say: Grounding language in robotic affordances,

    M. Ahn, A. Brohan, N. Brownet al., “Do as i can and not as i say: Grounding language in robotic affordances,” inarXiv preprint arXiv:2204.01691, 2022

    Pith/arXiv arXiv 2022

  5. [5]

    Inner monologue: Embodied rea- soning through planning with language models,

    W. Huang, F. Xia, T. Xiaoet al., “Inner monologue: Embodied rea- soning through planning with language models,” inarXiv preprint arXiv:2207.05608, 2022

    Pith/arXiv arXiv 2022

  6. [6]

    Robots that ask for help: Un- certainty alignment for large language model planners,

    A. Z. Ren, A. Dixit, A. Bodrovaet al., “Robots that ask for help: Un- certainty alignment for large language model planners,” inProceedings of the Conference on Robot Learning (CoRL), 2023

    2023

  7. [7]

    AmbiK: Dataset of ambiguous tasks in kitchen environment,

    A. Ivanova, B. Eva, Z. V olovikovaet al., “AmbiK: Dataset of ambiguous tasks in kitchen environment,” inProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), W. Che, J. Nabende, E. Shutovaet al., Eds. Vienna, Austria: Association for Computational Linguistics, Jul. 2025, pp. 33 216–33 241. [Online...

    2025

  8. [8]

    Aligning language models to explicitly handle ambiguity,

    H. J. Kim, Y . Kim, C. Parket al., “Aligning language models to explicitly handle ambiguity,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Y . Al-Onaizan, M. Bansal, and Y .-N. Chen, Eds. Miami, Florida, USA: Association for Computational Linguistics, Nov. 2024, pp. 1989–2007. [Online]. Available: https://aclan...

    2024

  9. [9]

    Clara: Classifying and disambiguating user commands for reliable interactive robotic agents,

    J. Park, S. Lim, J. Leeet al., “Clara: Classifying and disambiguating user commands for reliable interactive robotic agents,”IEEE Robotics and Automation Letters, vol. 9, no. 2, pp. 1059–1066, 2024

    2024

  10. [10]

    Open-vocabulary detection via vision and language knowledge distillation,

    X. Gu, T.-Y . Lin, W. Kuoet al., “Open-vocabulary detection via vision and language knowledge distillation,”arXiv preprint arXiv:2104.13921, 2021

    Pith/arXiv arXiv 2021

  11. [11]

    Blip: Bootstrapping language-image pre- training for unified vision-language understanding and generation,

    J. Li, D. Li, C. Xionget al., “Blip: Bootstrapping language-image pre- training for unified vision-language understanding and generation,” in ICML, 2022

    2022

  12. [12]

    Lemma: Learning language-conditioned multi-robot manipulation,

    R. Gong, X. Gao, Q. Gaoet al., “Lemma: Learning language-conditioned multi-robot manipulation,”IEEE Robotics and Automation Letters, vol. 8, no. 10, pp. 6835–6842, 2023

    2023

  13. [13]

    Combined task and motion planning through an extensible planner-independent interface layer,

    S. Srivastava, E. Fang, L. Rianoet al., “Combined task and motion planning through an extensible planner-independent interface layer,” in2014 IEEE International Conference on Robotics and Automation (ICRA), 2014, pp. 639–646

    2014

  14. [14]

    Progprompt: Generating situated robot task plans using large language models,

    I. Singh, V . Blukis, A. Mousavianet al., “Progprompt: Generating situated robot task plans using large language models,” in2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 11 523–11 530

    2023

  15. [15]

    Code as policies: Language model programs for embodied control,

    J. Liang, W. Huang, F. Xiaet al., “Code as policies: Language model programs for embodied control,” in2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 9493–9500

    2023

  16. [16]

    Llm-planner: Few-shot grounded planning for embodied agents with large language models,

    C. H. Song, J. Wu, C. Washingtonet al., “Llm-planner: Few-shot grounded planning for embodied agents with large language models,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2023

    2023

  17. [17]

    Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,

    W. Huang, P. Abbeel, D. Pathaket al., “Language models as zero-shot planners: Extracting actionable knowledge for embodied agents,”arXiv preprint arXiv:2201.07207, 2022

    Pith/arXiv arXiv 2022

  18. [18]

    React: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yuet al., “React: Synergizing reasoning and acting in language models,”arXiv preprint arXiv:2210.03629, 2022

    Pith/arXiv arXiv 2022

  19. [19]

    Text2reaction : Enabling reactive task planning using large language models,

    Z. Yang, L. Ning, H. Wanget al., “Text2reaction : Enabling reactive task planning using large language models,”IEEE Robotics and Automation Letters, vol. 9, no. 5, pp. 4003–4010, 2024

    2024

  20. [20]

    Grounding llms for robot task planning using closed-loop state feedback,

    V . Bhat, A. U. Kaypak, P. Krishnamurthyet al., “Grounding llms for robot task planning using closed-loop state feedback,” 2024. [Online]. Available: https://arxiv.org/abs/2402.08546

    arXiv 2024

  21. [21]

    Clam: Selective clarification for ambiguous questions with generative language models,

    L. Kuhn, Y . Gal, and S. Farquhar, “Clam: Selective clarification for ambiguous questions with generative language models,” 2023. [Online]. Available: https://arxiv.org/abs/2212.07769

    Pith/arXiv arXiv 2023

  22. [22]

    Eclair: enhanced clarification for interactive responses in an enterprise ai assistant,

    J. Murzaku, Z. Liu, V . Muppalaet al., “Eclair: enhanced clarification for interactive responses in an enterprise ai assistant,” inProceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth Symposium on Educational Advances in Artificial Intel...

  23. [23]

    Clarify when necessary: Resolving ambiguity through interaction with lms,

    M. J. Q. Zhang and E. Choi, “Clarify when necessary: Resolving ambiguity through interaction with lms,” 2023. [Online]. Available: https://arxiv.org/abs/2311.09469

    Pith/arXiv arXiv 2023

  24. [24]

    Ambiguity in llms is a concept missing problem,

    Z. Hu, C. Wang, Y . Shuet al., “Ambiguity in llms is a concept missing problem,” 2025. [Online]. Available: https://arxiv.org/abs/2505.11679

    arXiv 2025

  25. [25]

    A comprehensive survey of scene graphs: Generation and application,

    X. Chang, P. Ren, P. Xuet al., “A comprehensive survey of scene graphs: Generation and application,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 1, pp. 1–26, 2023

    2023

  26. [26]

    Image retrieval using scene graphs,

    J. Johnson, R. Krishna, M. Starket al., “Image retrieval using scene graphs,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015

    2015

  27. [27]

    Auto-encoding and distilling scene graphs for image captioning,

    X. Yang, H. Zhang, and J. Cai, “Auto-encoding and distilling scene graphs for image captioning,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 5, pp. 2313–2327, 2022

    2022

  28. [28]

    In defense of scene graphs for image captioning,

    K. Nguyen, S. Tripathi, B. Duet al., “In defense of scene graphs for image captioning,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2021, pp. 1407–1416

    2021

  29. [29]

    Scene graph reasoning for visual question answering,

    M. Hildebrandt, H. Li, R. Koneret al., “Scene graph reasoning for visual question answering,” 2020. [Online]. Available: https: //arxiv.org/abs/2007.01072

    Pith/arXiv arXiv 2020

  30. [30]

    Image generation from scene graphs,

    J. Johnson, A. Gupta, and L. Fei-Fei, “Image generation from scene graphs,” in2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 1219–1228

    2018

  31. [31]

    Self-supervised learning of scene-graph representations for robotic sequential manipulation planning,

    S. Nguyen, O. Oguz, V . Hartmannet al., “Self-supervised learning of scene-graph representations for robotic sequential manipulation planning,” inProceedings of the 2020 Conference on Robot Learning, ser. Proceedings of Machine Learning Research, J. Kober, F. Ramos, and C. Tomlin, Eds., vol. 155. PMLR, 16–18 Nov 2021, pp. 2104–2119. [Online]. Available: h...

    2020

  32. [32]

    Hierarchical planning for long-horizon manipulation with geometric and symbolic scene graphs,

    Y . Zhu, J. Tremblay, S. Birchfieldet al., “Hierarchical planning for long-horizon manipulation with geometric and symbolic scene graphs,” in2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE Press, 2021, p. 6541–6548. [Online]. Available: https://doi.org/10.1109/ICRA48506.2021.9561548

    arXiv 2021

  33. [33]

    Hierarchical Open-V ocabulary 3D Scene Graphs for Language-Grounded Robot Navigation,

    A. Werby, C. Huang, M. B ¨uchneret al., “Hierarchical Open-V ocabulary 3D Scene Graphs for Language-Grounded Robot Navigation,” inPro- ceedings of Robotics: Science and Systems, Delft, Netherlands, July 2024

    2024

  34. [34]

    Bird’s-eye-view scene graph for vision-language navigation,

    R. Liu, X. Wang, W. Wanget al., “Bird’s-eye-view scene graph for vision-language navigation,” inICCV, 2023, pp. 10 968–10 980

    2023

  35. [35]

    Sg-bot: Object rearrangement via coarse-to-fine robotic imagination on scene graphs,

    G. Zhai, X. Cai, D. Huanget al., “Sg-bot: Object rearrangement via coarse-to-fine robotic imagination on scene graphs,”arXiv preprint arXiv:2309.12188

    Pith/arXiv arXiv

  36. [36]

    Roboexp: Action-conditioned scene graph via interactive exploration for robotic manipulation,

    H. Jiang, B. Huang, R. Wuet al., “Roboexp: Action-conditioned scene graph via interactive exploration for robotic manipulation,”CoRL, 2024

    2024

  37. [37]

    Verigraph: Scene graphs for execution verifiable robot planning,

    D. Ekpo, M. Levy, S. Suriet al., “Verigraph: Scene graphs for execution verifiable robot planning,” 2024. [Online]. Available: https://arxiv.org/abs/2411.10446

    Pith/arXiv arXiv 2024

  38. [38]

    Sayplan: Grounding large language models using 3d scene graphs for scalable task planning,

    K. Rana, J. Haviland, S. Garget al., “Sayplan: Grounding large language models using 3d scene graphs for scalable task planning,” in 7th Annual Conference on Robot Learning, 2023. [Online]. Available: https://openreview.net/forum?id=wMpOMO0Ss7a

    2023

  39. [39]

    Sg-nav: Online 3d scene graph prompting for llm-based zero-shot object navigation,

    H. Yin, X. Xu, Z. Wuet al., “Sg-nav: Online 3d scene graph prompting for llm-based zero-shot object navigation,”arXiv preprint arXiv:2410.08189, 2024

    Pith/arXiv arXiv 2024

  40. [40]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection,

    S. Liu, Z. Zeng, T. Renet al., “Grounding dino: Marrying dino with grounded pre-training for open-set object detection,”arXiv preprint arXiv:2303.05499, 2023

    Pith/arXiv arXiv 2023

  41. [41]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmanset al., “Chain-of-thought prompting elicits reasoning in large language models,” inProceedings of the 36th International Conference on Neural Information Processing Systems, ser. NIPS ’22. Red Hook, NY , USA: Curran Associates Inc., 2022

    2022

  42. [42]

    The art of SOCRATIC QUESTIONING: Recursive thinking with large language models,

    J. Qi, Z. Xu, Y . Shenet al., “The art of SOCRATIC QUESTIONING: Recursive thinking with large language models,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 4177–4199. [Online]. Available: https://aclanth...

    2023

  43. [43]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,

    G. Comanici, E. Bieber, M. Schaekermannet al., “Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,” 2025. [Online]. Available: https://arxiv.org/abs/2507.06261

    Pith/arXiv arXiv 2025

  44. [44]

    Qwen3 technical report,

    A. Yang, A. Li, B. Yanget al., “Qwen3 technical report,” 2025. [Online]. Available: https://arxiv.org/abs/2505.09388

    Pith/arXiv arXiv 2025

  45. [2025]

    Available: https://doi.org/10.1609/aaai.v39i28.35363

    [Online]. Available: https://doi.org/10.1609/aaai.v39i28.35363

    work page doi:10.1609/aaai.v39i28.35363