Pith. sign in

REVIEW 3 major objections 5 minor 84 references

Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A vision-language loop can replace rewards and demonstrations as the driver of robotic exploration.

desk verdict The closed-loop architecture is worth a look, but the paper's headline 4.1–7.8x entropy claim is contradicted by its own Table 1, and the authors need to fix that before the results can be trusted. read the letter →

arxiv 2505.07815 v3 pith:X4WFJAYB submitted 2025-05-12 cs.RO cs.CVcs.LG

Seungjae Lee , Daniel Ekpo , Haowen Liu , Furong Huang , Abhinav Shrivastava , Jia-Bin Huang This is my paper · ORCID
classification cs.ROcs.CVcs.LG
keywords explorationvision-languagemodelsscenegraphsroboticmanipulationreward-freedatacollectionbehaviorcloningworldagenticsystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents IVE, an autonomous exploration system in which a vision-language model reads an RGB-D image into a semantic scene graph, imagines a novel but plausible next configuration, verifies the proposed plan against recent interaction history, and then executes it through a library of pick-and-place skill primitives. The central claim is that this imagine-verify-execute loop generates physically grounded and semantically diverse interaction data without external rewards, goals, or demonstrations. In tabletop experiments, IVE visits states with 4.1 to 7.8 times higher state-visitation entropy than intrinsic-reward RL baselines and reaches 82 to 122 percent of the scene diversity achieved by human experts. Downstream policies and world models trained on its data match or exceed those trained on human-collected demonstrations, so the method is a candidate replacement for expensive human data collection in open-ended manipulation settings.

What carries the argument

The load-bearing object is the scene graph, a small typed graph whose nodes are object names and whose edges are relations such as Stacked on and Near. A scene-describer VLM builds this graph from a single observation; an explorer VLM imagines a desired future graph and a skill sequence; a verifier VLM simulates the outcome, checks stability and feasibility, and returns feedback; and a memory module stores every graph encountered, retrieving structurally similar ones by edit-based graph distance to guide novelty. Action tools translate each skill into concrete pick-and-place commands with RGB-D perception, so the imagined transition becomes a real interaction whose outcome is recorded back into memory. The closed loop of imagination, verification, execution, and memory is what converts semantic imagination into physically grounded experience.

What would settle it

The decisive check is to measure both perceptual and verification quality directly: annotate a held-out set of scenes with ground-truth object relations and compute precision and recall of the scene describer, and record how often plans the verifier accepts actually fail during execution. If accepted plans fail at high rates or the graphs systematically mislabel relations, the reported entropy and diversity gains would be artifacts of the graph representation rather than evidence of grounded exploration.

Watch

Extended reading notes

Core claim

The discovery is that semantic abstraction plus memory is enough to turn a VLM's imagination into grounded exploration data. IVE represents each observation as a typed scene graph of objects and relations; an edit-distance retrieval from memory pushes the imagined next graph toward configurations that have not been seen before; and a verifier over the same VLM class rejects plans that are unstable or physically infeasible before any action is executed. The reported consequence is a 4.1 to 7.8 times increase in state entropy over intrinsic-reward RL baselines, 82 to 122 percent of expert-human scene diversity, and behavior-cloning and world-model performance on par with or better than training on human demonstrations. The paper attributes this to planning over graphs instead of pixels: pixel-level novelty chasing is replaced by semantic novelty that is checked against physical feasibility and memory.

Load-bearing premise

The load-bearing premise is that the VLM's scene graphs and feasibility judgments are accurate enough that the imagined transition matches what actually happens when the robot executes it; the paper does not report separate accuracy numbers for the scene describer or the verifier, so if either silently fails, the recorded experience is imagined diversity rather than grounded interaction.

Editorial extensions

If this is right

  • The method removes the need for hand-designed rewards or goal distributions in tabletop exploration, because the VLM's imagination and the verifier's rejection loop supply direction.
  • Data collected by IVE is directly usable for imitation-style policy learning: a visuomotor diffusion policy trained on it matches or exceeds policies trained on human demonstrations.
  • World models trained on IVE data predict subsequent frames nearly as well as models trained on human data, indicating the collected transitions reflect actual physical dynamics.
  • Removing memory or replacing the learned explorer with a rule-based one lowers the number of unique scenes discovered, so memory-guided novelty is a measurable source of the diversity gain.
  • Because the pipeline is automated, interaction data can be scaled up without human supervision or manual teleoperation, provided the action-tool library covers the behaviors of interest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the evaluation quantizes state into scene graphs, the reported entropy gain may undercount fine-grained continuous diversity; comparing against pose-based or pixel-based state metrics would test whether the advantage survives at the level of exact object configurations.
  • The verifier is the main grounding bottleneck: with a stronger or more conservative verifier, the same explorer could attempt more ambitious rearrangements, while a weaker verifier would decouple imagined novelty from executed reality.
  • The same architecture should transfer beyond tabletops if the fixed action-tool library were replaced by learned or more general skills; longer-horizon behaviors would then exercise memory and verification in settings the authors did not test.
  • A testable prediction is that exploration quality tracks the underlying VLM's perceptual accuracy: with identical memory and action tools, more accurate scene descriptions should produce more novel graphs and fewer rejected plans.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. IVE is a VLM-driven exploration framework that abstracts RGB-D observations into scene graphs, uses an Explorer to imagine novel scene configurations, a Verifier to filter plans for physical feasibility, a memory of past scene graphs to encourage novelty, and action tools to execute skills. The paper evaluates exploration diversity via unique scene graph counts and state entropy, compares against RND, RE3, and human explorers, and tests downstream behavior cloning and world-model learning in VIMA Bench and on a real UR5e tabletop. The central claimed result is a 4.1 to 7.8x increase in state entropy over RL baselines and 82% to 122% of human scene diversity, with downstream policies that roughly match or exceed those trained on human demonstrations.

Significance. The paper addresses a timely problem—autonomous reward-free data collection for robotic manipulation—and the pipeline is clearly specified, including full prompts and action-tool implementation details that are valuable for reproducibility. If the quantitative claims were validated, the system would be a practical contribution to using VLMs for structured exploration. However, the headline entropy claim is not supported by the paper's own Table 1, and the accuracy of the Scene Describer and Verifier is unmeasured, so the strength of the central contribution is currently unclear. The downstream BC and world-model experiments, the ablation analysis, and the real-world implementation remain useful assets even if the stated entropy factors are corrected.

major comments (3)
  1. [Abstract / Section 1 / Section 4.2 / Table 1] The stated 4.1 to 7.8x increase in state entropy over RL baselines is contradicted by the only entropy values given in Table 1. In the 5-object VIMA setting, IVE's entropy is 2.283 versus 1.907 for SAC+RND and 1.754 for SAC+RE3, corresponding to ratios of about 1.20x and 1.30x; in the 4-object setting, IVE's entropy is 1.528 versus 1.907 and 1.959, corresponding to ratios of about 0.80x and 0.78x, i.e., lower entropy than either RL baseline. The repeated 1.907 value for SAC+RND in both environments also suggests a copy or metric-definition error. If the Entropy column is not the same state-entropy quantity defined in Appendix A, the paper must define it; if it is the same quantity, the abstract, Section 1, and Section 4.2 must be corrected to report the actual ratios, and the evaluation horizon used for the claimed factors in Figure 5 must be stated so that the 4.1 to 7.8x numbers are reproducible.
  2. [Section 3.1 / Section 3.3 / Appendix F] The manuscript reports no quantitative evaluation of either the Scene Describer or the Verifier, despite both being load-bearing for the claim that IVE produces physically grounded exploration. No precision or recall for scene-graph relation extraction versus ground truth is reported, even though VIMA provides ground-truth object positions for exactly this check; no acceptance/rejection accuracy for the Verifier is reported, and there is no analysis of whether rejected plans would actually have failed in execution. Because the same VLM class (GPT-4o) generates and evaluates plans, an unmeasured systematic bias in the Verifier could make the reported exploration reflect imagined rather than executed diversity. I ask the authors to report scene-graph accuracy in simulation, verifier decision accuracy against actual execution outcomes, and the plan rejection rate during exploration.
  3. [Section 4.2 / Appendix A] The primary exploration metric is aligned with the system's own optimization target by construction: IVE's Explorer is explicitly tasked with discovering novel scene graphs, and the top-line evaluation counts unique scene graphs and computes their visitation entropy. Although Appendix A states that evaluation scene graphs are produced with a separate quantization pipeline, the metric is still scene-graph diversity, so the comparison partly measures how well IVE optimizes the evaluation criterion. To support the broader claim of diverse state-space exploration, the paper should add at least one independent diversity measure (e.g., object-pose coverage, action-state coverage, or pixel-level state entropy) or else empirically show that scene-graph diversity is predictive of the reported downstream gains beyond this alignment.
minor comments (5)
  1. [Figure 5] The caption is incomplete: the final clause 'and 4.2 for baseline details' should read 'and Section 4.2 for baseline details,' and the closing parenthesis should be supplied.
  2. [Section 3.4, Eq. (1)] The retrieval threshold tau is introduced without a specified value or sensitivity analysis; please state how tau was chosen and whether the main results are stable across a reasonable range of thresholds.
  3. [Appendix A] The entropy formula should specify the base of the logarithm and the exact state set over which p(s) is normalized; the current summation domain is implicit.
  4. [Section 4.4] The phrase 'up to +58% in task success' is ambiguous between absolute percentage-point gains and relative gains; please clarify using the underlying rates in Table 1.
  5. [Section 4.3] The reported '22% and 27% drop in unique scenes discovered' for the memory and explorer ablations should state whether these are relative or absolute drops and should report the corresponding raw counts for each ablation.

Circularity Check

1 steps flagged · score 4.0 of 10

Headline scene-graph diversity metric is the same novelty objective the Explorer is prompted to optimize; downstream BC/WM results are independent.

  1. self definitional [Section 3.2 (Explorer), Appendix A (Evaluation Metrics), Appendix F.2 (Explorer prompt)]
    "To promote novelty and avoid redundancy, the Explorer compares ˆGt+1 with retrieved scene graphs from memory, encouraging transitions that are diverse and previously unseen. [Explorer prompt:] generate a sequence of actions that discover a new scene configuration - one that has not been seen before. ... Unique Scene Graphs: The number of distinct scene graphs encountered during exploration. A higher count reflects greater semantic diversity."

    IVE's exploration objective is defined as producing novel scene graphs: the Explorer is prompted to imagine a future scene graph not present in memory and to avoid previously seen configurations. The headline evaluation then defines the state space as a discrete set of scene graphs and measures exploration by the number and entropy of distinct scene graphs visited. Consequently, the central diversity claims (4.1-7.8x entropy gain over RL, 82-122% of human scene diversity) largely re-measure the system's own prompting target. Beating pixel-novelty RL baselines on this metric is partly guaranteed by construction rather than an emergent external property.

full rationale

The paper's derivation chain is mostly empirical rather than deductive, and the only notable circularity is the alignment between the Explorer's objective and the headline metric. The Explorer explicitly uses memory of past scene graphs to propose configurations 'not been seen before,' while Appendix A scores exploration as the number and entropy of distinct scene graphs; hence the system is being graded on the same symbolic quantity it is instructed to diversify. That is a self-definitional element, though not a fitted-parameter or equation-level reduction. The paper takes some precaution by using a separate quantization/evaluation pipeline and by comparing to humans and RL baselines that do not share the scene-graph objective. The self-citation [50] (VeriGraph) is not load-bearing: it only motivates scene graphs as an intermediate representation, alongside non-self citations. The limitations section acknowledges manual action tools and detection failures, but does not report verifier accuracy; that is a grounding/correctness risk, not circularity. The abstract's 4.1-7.8x factor is not supported by the Entropy column of Table 1 if that column is state entropy, but the discrepancy is a reporting/consistency concern rather than a definitional reduction. Overall: the central exploration-diversity claim is partially circular because it measures the objective the system was built to maximize, yet the downstream BC and WM evaluations provide independent content, warranting a score of 4.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

IVE does not introduce new physical objects, forces, or conserved quantities. Its components are software modules, and the scene graph is a data structure rather than an invented entity. No falsifiable handle outside the method itself is offered.

free parameters (3)
  • memory retrieval threshold tau = not reported
    Equation (1) selects past scene graphs within edit distance tau; tau controls how much history the Explorer and Verifier see. No sensitivity analysis is given, and retrieval directly shapes novelty decisions.
  • placement offsets = 0.08 m relational offsets, 0.01 m drop offset
    Appendix B.2: hand-chosen offsets for STACKED_ON and IN_FRONT_OF/BEHIND/LEFT/RIGHT placement. These affect whether executed arrangements actually realize the imagined graph.
  • action sequence length limit = NUM_STEPS_HERE, not stated
    Explorer prompt caps the number of actions per proposal. It bounds how complex imagined transitions can be and is chosen by the authors rather than derived.
assumptions (4)
  • domain assumption The scene graph with object nodes and Stacked On / Near edges is a sufficient abstraction of task-relevant state.
    Used throughout Section 3 and Appendix A to define both exploration goals and evaluation metrics. If graphs omit relations relevant to downstream tasks, measured diversity may not reflect useful experience.
  • domain assumption GPT-4o physical commonsense is accurate enough for the Verifier to predict stability and feasibility from language history.
    Section 3.3 and prompt F.3: the Verifier returns YES or NO without executing. Any systematic error propagates into invalid or unsafe exploration, and this accuracy is never measured.
  • domain assumption The Action Tools library can realize the skill sequences produced by the Explorer.
    Section 3.5 and Appendix B: execution depends on AnyGrasp, LangSAM, and hand-coded placements. Failures in perception or control are not modeled in the exploration loop.
  • domain assumption VimaBench simulation dynamics are a faithful proxy for real tabletop physics.
    Most RL baseline comparisons and ablations are in VimaBench. If simulated physics differ from real physics, the grounded-verification advantage may not transfer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Models." pith.science (2026). https://pith.science/paper/X4WFJAYB

@misc{pith2026250507815,
  author       = {Pith},
  title        = {Pith review of: Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X4WFJAYB}},
  note         = {Machine review of arXiv:2505.07815}
}
read the original abstract

Exploration is essential for general-purpose robotic learning, especially in open-ended environments where dense rewards, explicit goals, or task-specific supervision are scarce. Vision-language models (VLMs), with their semantic reasoning over objects, spatial relations, and potential outcomes, present a compelling foundation for generating high-level exploratory behaviors. However, their outputs are often ungrounded, making it difficult to determine whether imagined transitions are physically feasible or informative. To bridge the gap between imagination and execution, we present IVE (Imagine, Verify, Execute), an agentic exploration framework inspired by human curiosity. Human exploration is often driven by the desire to discover novel scene configurations and to deepen understanding of the environment. Similarly, IVE leverages VLMs to abstract RGB-D observations into semantic scene graphs, imagine novel scenes, predict their physical plausibility, and generate executable skill sequences through action tools. We evaluate IVE in both simulated and real-world tabletop environments. The results show that IVE enables more diverse and meaningful exploration than RL baselines, as evidenced by a 4.1 to 7.8x increase in the entropy of visited states. Moreover, the collected experience supports downstream learning, producing policies that closely match or exceed the performance of those trained on human-collected demonstrations.

Figures

Figures reproduced from arXiv: 2505.07815 by the authors.

Figure 1
Figure 1. Comparison of human, RL, and IVE exploration strategies. (a) [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Autonomous scene exploration with IVE. IVE enables autonomous exploration of the scene with diverse objects, e.g., a tangram (top) or common objects (bottom). state-space coverage [11, 12]. While effective in simulation or low-dimensional tasks, RL methods often struggle in real-world robotic settings where environments are high-dimensional, semantically rich, and subject to physical constraints and safety risks. In… view at source ↗
Figure 3
Figure 3. Overview of IVE. Given an observation ot , the Scene Describer constructs a semantic scene graph Gt . The Explorer leverages this representation, along with the current observation and retrieved past scene graphs, to generate (“imagine”) a candidate future scene graph Gˆ t+1 and a sequence of skills µ 1∶N t . The Verifier evaluates the feasibility of these imagined transitions using recent interaction history. If ve… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Example of transforming skill to action in a real-world environment. To bridge the high-level skill sequence µ 1∶N t generated by the Explorer with real-world execution, we employ an Action Tools module that translates each skill into a corre￾sponding low-level action …
Figure 5
Figure 5. Figure 5: Exploration capability evaluation across simulated and real-world environments. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Ablation study of IVE. (Top) Illustration of each variant, highlighting removed modules in gray. (Bottom) Exploration performance is measured by the number of unique scene graphs, entropy, empowerment, and information gain (see Appendix A for metric details). We evalua…
Figure 7
Figure 7. Figure 7: The Region-Based Placement Tool overlays the workspace with a labeled grid, allowing the VLM to reference specific spatial locations when issuing placement commands. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Exploring with Embodied Agents: This figure compares the exploration capabilities of our method, IVE, powered by different Vision-Language Models (VLMs) rather than GPT-4o. Notably, IVE, regardless of the VLM used, matches or surpasses the human expert in generating un…
Figure 9
Figure 9. Figure 9: To evaluate the performance of the behavior cloning policy, we train Diffusion Policy [63] on each dataset and evaluate it on goal-conditioned tasks, where the initial observation is fixed and the agent is tested with six different goals. 16 [PITH_FULL_IMAGE:figures/f…
Figure 10
Figure 10. Figure 10: Qualitative examples of World Model (WM) predictions using datasets collected by different exploration methods. Red rectangles highlight regions with notable prediction errors. F Prompts for IVE F.1 Scene Describer The Scene Describer takes an RGB image and produces a…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

84 extracted references · 74 canonical work pages

  1. [1]

    A. Ten, P. Kaushik, P.-Y . Oudeyer, and J. Gottlieb. Humans monitor learning progress in curiosity-driven exploration.Nature communications, 2021

    2021

  2. [2]

    Modirshanechi, K

    A. Modirshanechi, K. Kondrakiewicz, W. Gerstner, and S. Haesler. Curiosity-driven exploration: foundations in neuroscience and computational modeling.Trends in Neurosciences, 2023

    2023

  3. [3]

    Lidayan, Y

    A. Lidayan, Y . Du, E. Kosoy, M. Rufova, P. Abbeel, and A. Gopnik. Intrinsically-motivated humans and agents in open-world exploration.arXiv preprint arXiv:2503.23631, 2025

    arXiv 2025

  4. [4]

    Aubret, L

    A. Aubret, L. Matignon, and S. Hassas. An information-theoretic perspective on intrinsic motivation in reinforcement learning: A survey.Entropy, 25(2), 2023

    2023

  5. [5]

    S. Lee, D. Cho, J. Park, and H. J. Kim. Cqm: Curriculum reinforcement learning with a quantized world model.Advances in Neural Information Processing Systems, 2023

    2023

  6. [6]

    Sukhija, S

    B. Sukhija, S. Coros, A. Krause, P. Abbeel, and C. Sferrazza. MaxinfoRL: Boosting exploration in reinforcement learning through information gain maximization. InInternational Conference on Learning Representations, 2025

    work page 2025

  7. [7]

    Anthony, K

    G.-C. Anthony, K. Marino, and R. Fergus. Efficient exploration and discriminative world model learning with an object-centric abstraction. InInternational Conference on Learning Representations, 2024

    work page 2024

  8. [8]

    Pathak, P

    D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell. Curiosity-driven exploration by self- supervised prediction. InInternational Conference on Machine Learning, 2017

    work page 2017

Show all 84 references
  1. [9]

    Mendonca, O

    R. Mendonca, O. Rybkin, K. Daniilidis, D. Hafner, and D. Pathak. Discovering and achieving goals via world models.Advances in Neural Information Processing Systems, 2021

    2021

  2. [10]

    Y . Seo, L. Chen, J. Shin, H. Lee, P. Abbeel, and K. Lee. State entropy maximization with random encoders for efficient exploration. InInternational Conference on Machine Learning, volume 139, 2021

    2021

  3. [11]

    V . H. Pong, M. Dalal, S. Lin, A. Nair, S. Bahl, and S. Levine. Skew-fit: state-covering self-supervised reinforcement learning. InInternational Conference on Machine Learning, 2020. 9

    2020

  4. [12]

    E. S. Hu, R. Chang, O. Rybkin, and D. Jayaraman. Planning goals for exploration. In International Conference on Learning Representations, 2023

    2023

  5. [13]

    Huang, C

    W. Huang, C. Wang, R. Zhang, Y . Li, J. Wu, and L. Fei-Fei. V oxposer: Composable 3d value maps for robotic manipulation with language models. InConference on Robot Learning, 2023

    2023

  6. [14]

    D. Shah, B. Osi´nski, S. Levine, et al. Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action. InConference on Robot Learning, 2023

    2023

  7. [15]

    Etukuru, N

    H. Etukuru, N. Naka, Z. Hu, S. Lee, C. Paxton, S. Chintala, L. Pinto, and N. M. M. Shafiullah. Robot utility models: General policies for zero-shot deployment in new environments. InCORL Workshop, 2024

    2024

  8. [16]

    A. H. Tan, A. Fung, H. Wang, and G. Nejat. Mobile robot navigation using hand-drawn maps: A vision language model approach.arXiv preprint arXiv:2502.00114, 2025

    2025 arXiv

  9. [17]

    Kuang, H

    Y . Kuang, H. Lin, and M. Jiang. Openfmnav: Towards open-set zero-shot object navigation via vision-language foundation models. InICLR Workshop, 2024

    2024

  10. [18]

    Jiang, B

    W. Jiang, B. Lei, K. Ashton, and K. Daniilidis. Multimodal llm guided exploration and active mapping using fisher information.arXiv preprint arXiv:2410.17422, 2024

    2024 arXiv

  11. [19]

    Jiang, B

    H. Jiang, B. Huang, R. Wu, Z. Li, S. Garg, H. Nayyeri, S. Wang, and Y . Li. RoboEXP: Action- conditioned scene graph via interactive exploration for robotic manipulation. InConference on Robot Learning, 2024

    2024

  12. [20]

    Sancaktar, C

    C. Sancaktar, C. Gumbsch, A. Zadaianchuk, P. Kolev, and G. Martius. SENSEI: Semantic exploration guided by foundation models to learn versatile world models. InWorkshop on Training Agents with Foundation Models at RLC, 2024

    2024

  13. [21]

    L. Li, J. Xu, Q. Dong, C. Zheng, X. Sun, L. Kong, and Q. Liu. Can language models understand physical concepts? InConference on Empirical Methods in Natural Language Processing, 2023

    2023

  14. [22]

    Y . Hu, F. Lin, T. Zhang, L. Yi, and Y . Gao. Look before you leap: Unveiling the power of gpt-4v in robotic vision-language planning. InICRA Workshop, 2024

    2024

  15. [23]

    Elnoor, K

    M. Elnoor, K. Weerakoon, G. Seneviratne, R. Xian, T. Guan, M. K. M. Jaffar, V . Rajagopal, and D. Manocha. Robot navigation using physically grounded vision-language models in outdoor environments.CoRR, 2024

    2024

  16. [24]

    Gaven, T

    L. Gaven, T. Carta, C. Romac, C. Colas, S. Lamprier, O. Sigaud, and P.-Y . Oudeyer. Magellan: Metacognitive predictions of learning progress guide autotelic llm agents in large goal spaces. InInternational Conference on Machine Learning. Proceedings of Machine Learning Research, 2025

    2025

  17. [25]

    Colas, T

    C. Colas, T. Karch, O. Sigaud, and P.-Y . Oudeyer. Autotelic agents with intrinsically motivated goal-conditioned reinforcement learning: a short survey.Journal of Artificial Intelligence Research, 74, 2022

    2022

  18. [26]

    Burda, H

    Y . Burda, H. Edwards, A. Storkey, and O. Klimov. Exploration by random network distillation. InInternational Conference on Learning Representations, 2019

    2019

  19. [27]

    D. Kim, J. Shin, P. Abbeel, and Y . Seo. Accelerating reinforcement learning with value- conditional state entropy exploration.Advances in Neural Information Processing Systems, 2023

    2023

  20. [28]

    Martin, S

    J. Martin, S. S. Narayanan, T. Everitt, and M. Hutter. Count-based exploration in feature space for reinforcement learning. InInternational Joint Conference on Artificial Intelligence, 2017. 10

    2017

  21. [29]

    Shahidzadeh, S

    A.-H. Shahidzadeh, S. J. Yoo, P. Mantripragada, C. D. Singh, C. Fermüller, and Y . Aloimonos. Actexplore: Active tactile exploration on unknown objects. InInternational Conference on Robotics and Automation, 2024

    2024

  22. [30]

    Durugkar, S

    I. Durugkar, S. S. Hansen, S. Spencer, and V . Mnih. Wasserstein distance maximizing intrinsic control. InNeurIPS Workshop, 2021

    2021

  23. [31]

    Klissarov and M

    M. Klissarov and M. C. Machado. Deep laplacian-based options for temporally-extended exploration. InInternational Conference on Machine Learning, 2023

    2023

  24. [32]

    J. Bae, K. Park, and Y . Lee. Tldr: Unsupervised goal-conditioned rl via temporal distance-aware representations. InConference on Robot Learning, 2024

    2024

  25. [33]

    D. Cho, S. Lee, and H. J. Kim. Outcome-directed reinforcement learning by uncertainty \& temporal distance-aware curriculum goal generation. InInternational Conference on Learning Representations, 2023

    2023

  26. [34]

    Pitis, H

    S. Pitis, H. Chan, S. Zhao, B. Stadie, and J. Ba. Maximum entropy gain exploration for long horizon multi-goal reinforcement learning. InInternational Conference on Machine Learning, 2020

    2020

  27. [35]

    Yarats, R

    D. Yarats, R. Fergus, A. Lazaric, and L. Pinto. Reinforcement learning with prototypical representations. InInternational Conference on Machine Learning. Proceedings of Machine Learning Research, 2021

    2021

  28. [36]

    S. V . Mahankali, Z.-W. Hong, A. Sekhari, A. Rakhlin, and P. Agrawal. Random latent explo- ration for deep reinforcement learning. InInternational Conference on Machine Learning, 2024

    2024

  29. [37]

    M. Ahn, A. Brohan, N. Brown, Y . Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakr- ishnan, K. Hausman, et al. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022

    2022 arXiv

  30. [38]

    Z. Zhou, P. Atreya, A. Lee, H. R. Walke, O. Mees, and S. Levine. Autonomous improvement of instruction following skills via foundation models. InConference on Robot Learning, 2024

    2024

  31. [39]

    Z. Zhou, P. Atreya, Y . L. Tan, K. Pertsch, and S. Levine. Autoeval: Autonomous evaluation of generalist robot manipulation policies in the real world. InCORL Workshop, 2025

    2025

  32. [40]

    R. Shah, A. Yu, Y . Zhu, Y . Zhu, and R. Martín-Martín. Bumble: Unifying reasoning and acting with vision-language models for building-wide mobile manipulation.arXiv preprint arXiv:2410.06237, 2024

    2024 arXiv

  33. [41]

    Johnson, R

    J. Johnson, R. Krishna, M. Stark, L.-J. Li, D. Shamma, M. Bernstein, and L. Fei-Fei. Image retrieval using scene graphs. InIEEE Conference on Computer Vision and Pattern Recognition, 2015

    2015

  34. [42]

    Krishna, Y

    R. Krishna, Y . Zhu, O. Groth, J. Johnson, K. Hata, J. Kravitz, S. Chen, Y . Kalantidis, L.-J. Li, D. A. Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations.International Journal on Computer Vision, 123, 2017

    2017

  35. [43]

    J. Ji, R. Krishna, L. Fei-Fei, and J. C. Niebles. Action genome: Actions as compositions of spatio-temporal scene graphs. InIEEE Conference on Computer Vision and Pattern Recognition, 2020

    2020

  36. [44]

    Johnson, A

    J. Johnson, A. Gupta, and L. Fei-Fei. Image generation from scene graphs. InIEEE Conference on Computer Vision and Pattern Recognition, 2018

    2018

  37. [45]

    Ashual and L

    O. Ashual and L. Wolf. Specifying object attributes and relations in interactive scene generation. International Conference Computer Vision, 2019. 11

    2019

  38. [46]

    Dhamo, A

    H. Dhamo, A. Farshad, I. Laina, N. Navab, G. Hager, F. Tombari, and C. Rupprecht. Semantic image manipulation using scene graphs. InIEEE Conference on Computer Vision and Pattern Recognition, 2020

    2020

  39. [47]

    Herzig, A

    R. Herzig, A. Bar, H. Xu, G. Chechik, T. Darrell, and A. Globerson. Learning canonical representations for scene graph to image generation. InEuropean Conference on Computer Vision, 2020

    2020

  40. [48]

    K. Rana, J. Haviland, S. Garg, J. Abou-Chakra, I. Reid, and N. Suenderhauf. Sayplan: Ground- ing large language models using 3d scene graphs for scalable robot task planning. InConference on Robot Learning, 2023

    2023

  41. [49]

    Z. Ni, X. Deng, C. Tai, X. Zhu, Q. Xie, W. Huang, X. Wu, and L. Zeng. Grid: Scene-graph-based instruction-driven robotic task planning. InInternational Conference on Intelligent Robots and Systems, 2024

    2024

  42. [50]

    D. Ekpo, M. Levy, S. Suri, C. Huynh, and A. Shrivastava. Verigraph: Scene graphs for execution verifiable robot planning.arXiv preprint arXiv:2411.10446, 2024

    2024 arXiv

  43. [51]

    Q. Gu, A. Kuwajerwala, S. Morin, K. M. Jatavallabhula, B. Sen, A. Agarwal, C. Rivera, W. Paul, K. Ellis, R. Chellappa, et al. Conceptgraphs: Open-vocabulary 3d scene graphs for perception and planning. InInternational Conference on Robotics and Automation, 2024

    2024

  44. [52]

    Ravichandran, L

    Z. Ravichandran, L. Peng, N. Hughes, J. D. Griffith, and L. Carlone. Hierarchical representations and explicit memory: Learning effective navigation policies on 3d scene graphs using graph neural networks. InInternational Conference on Robotics and Automation, 2022

    2022

  45. [53]

    G. Zhai, X. Cai, D. Huang, Y . Di, F. Manhardt, F. Tombari, N. Navab, and B. Busam. Sg-bot: Object rearrangement via coarse-to-fine robotic imagination on scene graphs. InInternational Conference on Robotics and Automation, 2024

    2024

  46. [54]

    Amiri, K

    S. Amiri, K. Chandan, and S. Zhang. Reasoning with scene graphs for robot planning under partial observability.IEEE Robotics and Automation Letters, 2022

    2022

  47. [55]

    Jiang, A

    Y . Jiang, A. Gupta, Z. Zhang, G. Wang, Y . Dou, Y . Chen, L. Fei-Fei, A. Anandkumar, Y . Zhu, and L. Fan. Vima: General robot manipulation with multimodal prompts. InInternational Conference on Machine Learning, 2023

    2023

  48. [56]

    Gpt-4o: Openai’s new multimodal model

    OpenAI. Gpt-4o: Openai’s new multimodal model. https://openai.com/index/gpt-4o,

  49. [57]

    H.-S. Fang, C. Wang, H. Fang, M. Gou, J. Liu, H. Yan, W. Liu, Y . Xie, and C. Lu. Anygrasp: Robust and efficient grasp perception in spatial and temporal domains.IEEE Transactions on Robotics, 2023

    2023

  50. [58]

    Medeiros

    L. Medeiros. Lang segment anything. https://github.com/luca-medeiros/ lang-segment-anything, 2023

    2023

  51. [59]

    P. Liu, Z. Guo, M. Warke, S. Chintala, C. Paxton, N. M. M. Shafiullah, and L. Pinto. Dynamem: Online dynamic spatio-semantic memory for open world mobile manipulation. InCORL Workshop, 2024

    2024

  52. [60]

    P. Liu, Y . Orru, J. Vakil, C. Paxton, N. M. M. Shafiullah, and L. Pinto. Ok-robot: What really matters in integrating open-knowledge models for robotics. InICRA Workshop, 2024

    2024

  53. [61]

    Haarnoja, A

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. InInternational Conference on Machine Learning, 2018. 12

    2018

  54. [62]

    G. Zhou, H. Pan, Y . LeCun, and L. Pinto. Dino-wm: World models on pre-trained visual features enable zero-shot planning.arXiv preprint arXiv:2411.04983, 2024

    2024 arXiv

  55. [63]

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, 2023. 13 A Evaluation Metrics for Exploration Capability To quantitatively eva...

    2023

  56. [65]

    To the left of,

    Relation-Based Placement Tools:Execute relational actions that position objects with respect to others (e.g., “To the left of,” “Stacked on”) as shown in Figure 4

  57. [66]

    Region-Based Placement Tools:Place objects at specific locations on a predefined 2D layout (e.g., grid cells)

  58. [67]

    Arranger Tool:Manages workspace cleanliness by moving unreferenced or obstructive objects to free, uncovered regions, enabling subsequent actions. Modularity.The design of the Action Tools module is deliberately modular and extensible—new primitives or skills can be incorporat...

  59. [68]

    Begin with one object

  60. [69]

    Add one new object at a time , to your partial scene graph

  61. [70]

    - ** Use only ** the Allowed Re lat io ns in the scene graph

    For each newly added object : - D et er min e its spatial relation ( s ) to the objects already in the scene graph . - ** Use only ** the Allowed Re lat io ns in the scene graph . - Do not assign more than one relation for the same object pair ‘( new_object , e x i s t i n g _...

  62. [71]

    - ** List all verified re la ti on s ** between pairs of objects , using the Allowed Re la ti on s in the scene graph

    ** Once all objects ** have been i n t r o d u c e d and verified , compile a ** complete scene graph **: - ** List all nodes ** ( the objects in the final scene ) . - ** List all verified re la ti on s ** between pairs of objects , using the Allowed Re la ti on s in the scene graph

  63. [72]

    Global Object Names

    ** Use only ** objects from the " Global Object Names ."

  64. [73]

    --- ## Scene Graph R e p r e s e n t a t i o n - Nodes : Objects present in the scene

    Even if there ’ s missing nodes or edges in a final scene graph ( because at least one object is missing ) , you must still provide a complete ** scratch pad ** and ** scene graph ** with existing r ela ti on s . --- ## Scene Graph R e p r e s e n t a t i o n - Nodes : Objects...

  65. [74]

    Order Matters : Plan your actions so that p r e c o n d i t i o n s are s ati sf ie d before you move an object

  66. [75]

    Scene B o u n d a r i e s : If an object is near the scene boundary , avoid pushing it further toward the edge or placing new objects in a risky position

  67. [76]

    - In c lut te re d areas , an attempt to move one object may cause u n i n t e n d e d c o l l i s i o n s or shifts in n e i g h b o r i n g objects

    M a n i p u l a t i o n ( Suction ) C o n s t r a i n t s : - The suction can only reliably pick the topmost exposed surface . - In c lut te re d areas , an attempt to move one object may cause u n i n t e n d e d c o l l i s i o n s or shifts in n e i g h b o r i n g objects ...

  68. [77]

    Desired Scene Graph should use re la ti on s among < S C E N E G R A P H _ R E L A T I O N S > only , same as other Scene Graphs

    Note : The list of allowed r el at io ns in Action Types and the r ela ti on s used in Scene Graph R e p r e s e n t a t i o n ([ Stacked On , Near ]) may differ . Desired Scene Graph should use re la ti on s among < S C E N E G R A P H _ R E L A T I O N S > only , same as oth...

  69. [78]

    ‘ move ( obj_a , RELATION , obj_b ) ’ e . g . , ‘ move ( white cup , Stacked On , red plate ) ’ - Moves one object to a position relative to another . - Allowed RELATION list : ‘[ In Front Of , Behind , To The Left Of , To The Right Of , Stacked On ] ’

  70. [79]

    A1 " ,

    ‘ move ( obj_a , GRID_ID ) ’ e . g . , ‘ move ( blue ball , B3 ) ’ - Moves an object to a grid location on the image . ( ‘[" A1 " , " B3 " , ... , " E10 "] ’) 3 , ‘ arrange ( obj_a ) ’ e . g . , ‘ arrange ( red block ) ’ - Pick up the objects and organize them in a clear area ...

  71. [80]

    ** Simulate ** the effect of the action sequence from the current scene

  72. [81]

    ** Predict ** the r es ult in g scene graph

  73. [82]

    ** Compare ** the p re dic te d graph with the desired one

  74. [83]

    ** Evaluate physical f e a s i b i l i t y and e xec ut io n s ta bi li ty **

  75. [84]

    move obj_A before c o n t i n u i n g

    ** Provide a judgment **: - Valid and feasible 21 - Invalid ( with reason ) - Valid but risky ( suggest a targeted fix ) - Too unstable to proceed ( re co mme nd d ecl ut te r mode ) --- < ACTION_TYPES > --- ## T r a n s i t i o n History A sequence of a l t e r n a t i n g sc...

  76. [2024]

    Accessed: 2024-04-30

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.