Pith. sign in

REVIEW 2 major objections 5 minor 62 references

A generative 3D world engine that turns language into physics-valid robot training environments, lifting real-robot success from 21.7% to 75.0%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 15:44 UTC pith:5TU4Y7WW

load-bearing objection Solid systems integration for sim-ready embodied worlds; static QA is credible, but the headline RL/sim-to-real numbers are mostly imported from companion papers and should not be treated as fully self-contained here. the 2 major comments →

arxiv 2607.07459 v2 pith:5TU4Y7WW submitted 2026-07-08 cs.RO cs.CV

EmbodiedGen V2: An Agentic, Simulation-Ready 3D World Engine for Embodied AI

Xinjie Wang , Liu Liu , Taojun Ding , Andrew Choi , Chaodong Huang , Mengao Zhao , Ziang Li , Jackson Jiang
show 4 more authors
Chunlei Yu Shengxiang Liu Wei Xu Zhizhong Su
This is my paper
classification cs.RO cs.CV
keywords embodied AIsim-ready 3D assetstask-driven world generationaffordance annotationcross-simulator exportvision-language-action policiessim-to-real transfernatural-language scene editing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Robots that learn in simulation need more than pretty 3D models: they need complete task environments with correct scale, collision geometry, physical properties, graspable parts, and layouts that obey the task. EmbodiedGen V2 claims that one shared sim-ready representation can carry all of that from generation through editing and across simulators, so that open-ended language can become executable worlds without hand assembly. The system produces assets, part-level interaction labels, task-conditioned layouts, multi-room navigable houses, and stateful natural-language edits under the same contract. Static checks report high acceptance and collision reliability; more importantly, policies trained only in the generated worlds improve dramatically in simulation and transfer to real robots. If the claim holds, the bottleneck for closed-loop robot learning shifts from scene authoring to curriculum design over automatically generated environments.

Core claim

EmbodiedGen V2 establishes that a unified sim-ready world representation—coupling metric geometry, physical validity, interaction affordances, task semantics, and standardized simulator interfaces—can turn generative 3D pipelines into scalable infrastructure for embodied policy training, evaluation, and deployment, with generated environments alone raising simulation success from 9.7% to 79.8% and real-robot success from 21.7% to 75.0%.

What carries the argument

The unified sim-ready representation: a shared object-and-scene contract (textured and collision geometry, physical parameters, part affordances, typed Scene Graph, 6-DoF poses, and cross-simulator export) that generation, affordance labeling, placement, large-scale scene solving, and stateful Vibe Coding all write into and read from.

Load-bearing premise

The system assumes that language and vision models can recover scale, mass, friction, part semantics, and task scene graphs accurately enough that the generated worlds still supply a trustworthy learning signal for policies.

What would settle it

Train the same online RL pipeline on matched real-world task distributions using only EmbodiedGen V2 scenes versus carefully measured real or hand-built scenes; if the generated-only policies systematically lag in real-robot success or show large residual dynamics failures, the central infrastructure claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Open-ended task language can become batch-loadable training and evaluation worlds without manual scene assembly.
  • The same assets and layouts can be reused across major physics simulators without per-engine rework.
  • Stateful natural-language editing can keep a physics-valid world while users refine instances and relations.
  • Scaling the number of generated scenes can shrink the gap between in-distribution and out-of-distribution policy success.
  • Sim-to-real VLA fine-tuning can draw its environment curriculum primarily from generative worlds rather than fixed hand-built suites.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If VLM-inferred physics and affordances are only approximately right, the system may still work by providing diversity and domain randomization rather than ground-truth dynamics fidelity.
  • The same contract could later support automated curriculum generation: agents propose harder tasks, the engine instantiates them, and policies train on the resulting distribution.
  • Multi-room topology plus addressable furniture suggests a natural path from tabletop manipulation curricula to long-horizon mobile manipulation without a second world format.
  • Persistent editable world state makes generated environments a debugging substrate for failed policies, not only a data source.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. EmbodiedGen V2 presents a generative 3D world engine that converts open-ended text/image inputs into executable, policy-ready simulation environments for embodied AI. Building on V1, it unifies sim-ready asset generation (pluggable TRELLIS/SAM3D/Hunyuan3D with mesh repair, CoACD collision proxies, VLM physical-property recovery, and multi-format export), part-level affordance autolabeling, task-driven Scene Graph worlds, multi-room navigable scenes, and stateful natural-language Vibe Coding edits. Static evaluations report 96.5% human acceptance and 98.6% collision success on assets (Table 2), 50% end-to-end affordance pass (Table 3), and 83.3% direct usability of 150 task-driven worlds (Table 4). Downstream closed-loop claims (Table 5, §3.4) summarize companion studies [6,7] in which online RL in generated scenes raises simulation success 9.7%→79.8% and real-robot success 21.7%→75.0%.

Significance. If the systems claims hold, this is a substantial contribution to embodied AI infrastructure: it targets the practical bottleneck of assembling sim-ready assets into task-conditioned, editable, cross-simulator worlds rather than isolated visual 3D generation. Strengths include a modular generate–verify–retry asset stack with concrete ablations (Tables 2–4), standardized URDF/MJCF/USD export demonstrated across six simulators (Fig. 6), released code and a 4K+ affordance-annotated asset collection, and explicit failure analysis (Fig. 11). The work is timely for scalable VLA/RL training. The closed-loop policy results, while impressive, are largely imported from concurrent co-authored studies and are only partially isolated inside this manuscript; the primary self-contained contribution is the generation stack and its static executability metrics.

major comments (2)
  1. [§3.4, Table 5] §3.4 and Table 5: The abstract and contribution list present online RL gains (9.7%→79.8% sim; 21.7%→75.0% real) as primary evidence that EmbodiedGen V2 is scalable policy-ready infrastructure. Those numbers are summarized from companion papers [6,7] rather than reproduced here with ablations that isolate the world engine. This manuscript does not report matched training under an identical RL recipe on non-EmbodiedGen or hand-built scenes of comparable diversity, nor sensitivity of learned policies to VLM-recovered mass/friction/scale (§2.2) or LLM Scene-Graph errors (§2.4). Either add such controls (or clear negative results) or reframe the closed-loop claims as external validation of usability rather than causal proof of the engine’s contribution.
  2. [§2.2–2.4, Tables 2–4] §2.2–2.4 and Tables 2–4: The weakest load-bearing assumption is that VLM/LLM-inferred physical metadata, part affordances, and task Scene Graphs are accurate enough to supply unbiased closed-loop learning signal. Static QA is strong (96.5% acceptance, 98.6% collision, 83.3% world usability), but affordance end-to-end pass is only 50% (Table 3), and there is no quantitative audit of recovered mass/friction/scale against real objects or of Scene-Graph relation errors against human layouts. Without such checks, high static scores can coexist with systematic physics/semantics bias that confounds attribution of the policy gains. A modest calibration study or error analysis on these inferred quantities would substantially strengthen the central claim.
minor comments (5)
  1. [Tables 2–4] Tables 2–4 report point estimates without confidence intervals or inter-annotator agreement for human acceptance / world usability; adding these would make the static claims more robust.
  2. [§2.5] §2.5 large-scale multi-room generation is described in detail but lacks a quantitative evaluation table comparable to Tables 2–4 (e.g., navigability, topology correctness, furniture addressability rates).
  3. [§2.6] §2.6 Vibe Coding is illustrated qualitatively (Fig. 8) but has no success/failure rates for multi-turn edits or physics-validation retries; even a small user study or automated edit-success metric would help.
  4. [§3.1–3.2] Clarify free parameters that affect reported metrics: aesthetic quality-gate thresholds, grasp slip thresholds (5 cm / 30°), adaptive lift-height for collision success, and scene complexity tiers.
  5. [Throughout] Minor presentation: occasional spacing issues (e.g., “T ask-driven”, “F unctional”) and ensure all companion arXiv numbers [6,7] are consistently cited when closed-loop numbers appear in the abstract and introduction.

Circularity Check

1 steps flagged

Static pipeline metrics are independently measured; closed-loop policy gains are load-bearing self-citations to concurrent co-authored work.

specific steps
  1. self citation load bearing [Abstract; §3.4 Downstream Closed-Loop Validation; Table 5]
    "Online reinforcement learning with generated environments further improves simulation success from 9.7% to 79.8%, and transfers to real robots with task success increasing from 21.7% to 75.0%. ... Choi et al. [6] use EmbodiedGen V2-generated interactive environments for online reinforcement learning (RL) of robot vision-language-action (VLA) policies... Table 5 summarizes the closed-loop results reported by the downstream study. ... Choi and Xu [7] further use EmbodiedGen V2-generated scenes to train sim-to-real VLA policies for cube stacking... raising real-world cube-stacking success from 43"

    The paper's central claim that EmbodiedGen V2 constitutes scalable infrastructure for training/deploying embodied policies rests quantitatively on these gains. The numbers and experimental settings are not re-derived or fully ablated here; they are imported from concurrent co-authored arXiv preprints ([6] Choi/Wang/Su/Xu; [7] Choi/Xu) that themselves use EmbodiedGen V2 scenes. Without independent matched controls or sensitivity analysis inside this manuscript, the policy-transfer half of the strongest claim reduces to self-citation rather than self-contained evidence.

full rationale

This is a systems paper whose primary technical content (unified sim-ready representation, asset pipeline with quality gates/CoACD/export, affordance autolabeling, task-driven Scene Graphs + BFS placement, large-scale rooms, Vibe Coding harness) is self-contained and evaluated with held-out assets/worlds under explicit protocols (Tables 2–4, human inspection, SAPIEN grasp trials). Those numbers (96.5% acceptance, 98.6% collision, 83.3% world usability) are not definitional or fitted-then-predicted. The only circularity is moderate self-citation load-bearing for the strongest claim that generated environments enable closed-loop VLA/RL improvement and sim-to-real transfer: the headline 9.7%→79.8% sim and 21.7%→75.0% real numbers (and the cube-stacking 43.1%→88.9%) are summarized from concurrent papers [6,7] whose author lists overlap heavily with the present work, without full ablations isolating the world engine inside this manuscript. That does not collapse the derivation by construction, so the score remains low; the independent static evaluations and system description still stand.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 3 invented entities

This is an engineering stack paper. Load-bearing content is mostly domain assumptions about simulators, third-party generators, and VLM reliability, plus hand-chosen QA thresholds—not free parameters fitted to a physical law. Invented entities are mostly interface contracts (sim-ready representation, role-typed Scene Graph, Vibe Coding harness), not new physical objects.

free parameters (4)
  • Aesthetic / hierarchical quality-gate thresholds
    Pipeline rejects samples below a predefined aesthetic and multi-stage quality threshold; exact cutoffs are not fully specified but directly affect reported acceptance rates.
  • Grasp validation slip thresholds (5 cm, 30°)
    Used in §3.2 to accept/reject grasp candidates; changes the affordance pass rate by construction.
  • Scene complexity tiers {Minimalist, Simple, Medium, Detail}
    Discrete control knobs for furniture density and solver cost in large-scale scene generation; chosen design parameters, not data-derived constants.
  • Adaptive lift-height threshold for collision success
    Grasp-and-lift success depends on an adaptive height proportional to bounding-box height; the proportionality rule is a free evaluation choice.
axioms (5)
  • domain assumption Mainstream physics engines (MuJoCo, Isaac, SAPIEN, Bullet, Genesis) share enough semantics that URDF/MJCF/USD conversion preserves contact behavior for training.
    Cross-simulator portability claim in §2.2 and Fig. 6 depends on this equivalence.
  • domain assumption VLMs can recover usable real-world scale, mass, friction, and part semantics from multi-view renderings.
    Physical property recovery (§2.2) and affordance annotation (§2.3) rest on this without independent metrology.
  • domain assumption CoACD convex decomposition yields collision proxies adequate for stable manipulation contacts.
    Used throughout asset and scene packaging; ablation shows modest but nonzero effect on collision success.
  • ad hoc to paper Task language can be factored into ROBOT/BACKGROUND/CONTEXT/TARGETS/DISTRACTORS with single-parent spatial relations sufficient for executable layouts.
    Scene Graph schema in §2.4 is a design choice that enables the pipeline; failures in Fig. 11 show when it breaks.
  • domain assumption Standard mesh repair, UV baking, and physics settling remove most generative-mesh defects relevant to simulation.
    Generate–verify–retry loop assumes residual defects are rare after gates.
invented entities (3)
  • Unified sim-ready world representation (asset + typed Scene Graph + poses + history) no independent evidence
    purpose: Single contract coupling geometry, physics, affordances, edit state, and simulator export.
    Central abstraction of the paper; useful interface, not an independently measured natural object.
  • Vibe Coding agent–skill–harness for stateful 3D world editing no independent evidence
    purpose: Turn natural-language dialogue into bounded, physics-validated scene deltas.
    Named systems construct built from LLM tools plus deterministic solvers; evidence is qualitative sessions, not external theory.
  • Task-role Scene Graph (background/context/targets/distractors/robot) no independent evidence
    purpose: Map open language tasks to placeable interactive worlds.
    Paper-specific factorization enabling generation; validated mainly by internal acceptance rate.

pith-pipeline@v1.1.0-grok45 · 26192 in / 3625 out tokens · 39733 ms · 2026-07-14T15:44:13.431390+00:00 · methodology

0 comments
read the original abstract

We present EmbodiedGen V2, a generative 3D world engine for building executable policy-ready environments for embodied intelligence. Sim-ready 3D asset generation has advanced rapidly, yet assembling such assets into policy-ready task environments remains largely manual, limiting scalable closed-loop learning. EmbodiedGen V2 addresses this gap through a unified sim-ready representation that connects cross-simulator assets, interaction affordances, task-driven worlds, large-scale multi-room scenes, and stateful Vibe Coding into a generative, editable, and reusable simulation pipeline. The generated environments support manipulation, navigation, mobile manipulation, cross-simulator deployment, and embodied policy training. In evaluation, the asset pipeline achieves 96.5% human acceptance and 98.6% collision success, and 83.3% of task-driven worlds are directly usable for downstream simulation without manual modification. Online reinforcement learning with generated environments further improves simulation success from 9.7% to 79.8%, and transfers to real robots with task success increasing from 21.7% to 75.0%. These results establish EmbodiedGen V2 as scalable simulation infrastructure for training, evaluating, and deploying embodied policies.

Figures

Figures reproduced from arXiv: 2607.07459 by Andrew Choi, Chaodong Huang, Chunlei Yu, Jackson Jiang, Liu Liu, Mengao Zhao, Shengxiang Liu, Taojun Ding, Wei Xu, Xinjie Wang, Zhizhong Su, Ziang Li.

Figure 1
Figure 1. Figure 1: Overview of EmbodiedGen V2. Left: natural-language task to sim-ready scene via Scene Graph and affordance-annotated assets. Middle: large-scale multi-room generation at different controllable complexity tiers. Right: Vibe Coding 3D world editing. All outputs deploy consistently across mainstream simulators. 1 arXiv:2607.07459v1 [cs.RO] 8 Jul 2026 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The sim-ready 3D asset generation pipeline. From text or image inputs, the system produces [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Twelve text-conditioned garments deployed as deformable meshes in Genesis [ [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The affordance autolabeling pipeline. From sim-ready assets, the system produces structured [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Task-driven interactive worlds generation pipeline: scene graph generation from a natural-language [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The same task-driven interactive world layout instantiated across six physics simulators (Genesis, [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Large-scale scenes generation. A task description is first distilled into a scene blueprint, then [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Three Vibe Coding 3D editing sessions (kitchen, top; living room, middle; office, bottom). From [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Floorplan canvas of the spatial-computing skill. Each pair shows a top-down rendering (left) and the corresponding symbolic floorplan with room and instance labels (right). Open-vocabulary references are grounded against this canvas, and the skill evaluates Eq. (1) on its room polygons and instance bounding boxes. 3 Experiments 3.1 Sim-Ready Pipeline Quality Evaluation We ablate each stage of the sim-ready… view at source ↗
Figure 10
Figure 10. Figure 10: Qualitative examples of task-driven interactive worlds generation. Each world is generated from [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Representative failure cases in task-driven interactive worlds generation. Top: asset-level failures [PITH_FULL_IMAGE:figures/full_fig_p017_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Visualizations from [6]. Left: parallelized RL snapshot for training general pick-and-place using EmbodiedGen V2-generated scenes. Right: sim-to-real deployment of an EmbodiedGen V2 fine-tuned VLA [PITH_FULL_IMAGE:figures/full_fig_p018_12.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

62 extracted references · 22 linked inside Pith

  1. [1]

    Embodiedgen: Towards a generative 3d world engine for embodied intelligence, 2025

    Xinjie Wang, Liu Liu, Yu Cao, Ruiqi Wu, Wenkang Qin, Dehui Wang, Wei Sui, and Zhizhong Su. Embodiedgen: Towards a generative 3d world engine for embodied intelligence, 2025. URL https://arxiv.org/abs/2506. 10600

    2025

  2. [2]

    Structured 3d latents for scalable and versatile 3d generation

    Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d generation. arXiv preprint arXiv:2412.01506 , 2024

    Pith/arXiv arXiv 2024

  3. [3]

    Sam 3d: 3dfy anything in images

    SAM 3D Team, Xingyu Chen, Fu-Jen Chu, Pierre Gleize, Kevin J Liang, Alexander Sax, Hao Tang, Weiyao Wang, Michelle Guo, Thibaut Hardin, Xiang Li, Aohan Lin, Jiawei Liu, Ziqi Ma, Anushka Sagar, Bowen Song, Xiaodong Wang, Jianing Yang, Bowen Zhang, Piotr Dollár, Georgia Gkioxari, Matt Feiszli, and Jitendra Malik. Sam 3d: 3dfy anything in images. arXiv prepr...

    Pith/arXiv arXiv 2025

  4. [4]

    Hunyuan3d 2.1: From images to high-fidelity 3d assets with production-ready pbr material, 2025

    Tencent Hunyuan3D Team. Hunyuan3d 2.1: From images to high-fidelity 3d assets with production-ready pbr material, 2025

    2025

  5. [5]

    3d-fixer: Coarse-to-fine in-place completion for 3d scenes from a single image

    Ze-Xin Yin, Liu Liu, Xinjie Wang, Wei Sui, Zhizhong Su, Jian Yang, and jin Xie. 3d-fixer: Coarse-to-fine in-place completion for 3d scenes from a single image. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2026

    2026

  6. [6]

    Scaling sim-to-real reinforcement learning for robot vlas with generative 3d worlds

    Andrew Choi, Xinjie Wang, Zhizhong Su, and Wei Xu. Scaling sim-to-real reinforcement learning for robot vlas with generative 3d worlds. arXiv preprint arXiv:2603.18532 , 2026

    arXiv 2026

  7. [7]

    Rankq: Offline-to-online reinforcement learning via self-supervised action ranking

    Andrew Choi and Wei Xu. Rankq: Offline-to-online reinforcement learning via self-supervised action ranking. arXiv preprint arXiv:2605.11151 , 2026

    Pith/arXiv arXiv 2026

  8. [8]

    Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation

    Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang, Yifei Feng, Mingxin Yang, Sheng Zhang, Xianghui Yang, Huiwen Shi, Sicong Liu, Junta Wu, Yihang Lian, Fan Yang, Ruining Tang, Zebin He, Xinzhou Wang, Jian Liu, Xuhui Zuo, Zhuo Chen, Biwen Lei, et al. Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets ge...

    Pith/arXiv arXiv 2025

  9. [9]

    Step1x-3d: Towards high-fidelity and controllable generation of textured 3d assets

    Weiyu Li, Xuanyang Zhang, Zheng Sun, Di Qi, Hao Li, Wei Cheng, Weiwei Cai, Shihao Wu, Jiarui Liu, Zihao Wang, Xiao Chen, Feipeng Tian, Jianxiong Pan, Zeming Li, Gang Yu, Xiangyu Zhang, Daxin Jiang, and Ping Tan. Step1x-3d: Towards high-fidelity and controllable generation of textured 3d assets. arXiv preprint arXiv:2505.07747, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    Hun- yuan3d 2.5: Towards high-fidelity 3d assets generation with ultimate details

    Zeqiang Lai, Yunfei Zhao, Haolin Liu, Zibo Zhao, Qingxiang Lin, Huiwen Shi, Xianghui Yang, Mingxin Yang, Shuhui Yang, Yifei Feng, Sheng Zhang, Xin Huang, Di Luo, Fan Yang, Fang Yang, Lifu Wang, Sicong Liu, Yixuan Tang, Yulin Cai, Zebin He, Tian Liu, Yuhong Liu, Jie Jiang, Linus, Jingwei Huang, and Chunchao Guo. Hun- yuan3d 2.5: Towards high-fidelity 3d as...

    Pith/arXiv arXiv 2025

  11. [11]

    Native and compact structured latents for 3d generation

    Jianfeng Xiang, Xiaoxue Chen, Sicheng Xu, Ruicheng Wang, Zelong Lv, Yu Deng, Hongyuan Zhu, Yue Dong, Hao Zhao, Nicholas Jing Yuan, and Jiaolong Yang. Native and compact structured latents for 3d generation. arXiv preprint arXiv:2512.14692 , 2025

    Pith/arXiv arXiv 2025

  12. [12]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Do- minik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. arXiv preprint arXiv:2403.03206 , ...

    Pith/arXiv arXiv doi:10.48550/arxiv.2403.03206 2024

  13. [13]

    Kolors: Effective training of diffusion model for photorealistic text-to-image synthesis

    Kolors Team. Kolors: Effective training of diffusion model for photorealistic text-to-image synthesis. arXiv preprint, 2024

    2024

  14. [14]

    Daniel Gatis. rembg. https://github.com/danielgatis/rembg, 2025. A tool to remove images background

    2025

  15. [15]

    Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.02643, 2023

    Pith/arXiv arXiv 2023

  16. [16]

    Rmbg-1.4: Background removal model

    BRIA AI. Rmbg-1.4: Background removal model. https://huggingface.co/briaai/RMBG-1.4, 2023. Accessed: 2025-05-19. 21

    2023

  17. [17]

    3d gaussian splatting for real- time radiance field rendering, 2023

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real- time radiance field rendering, 2023. URL https://arxiv.org/abs/2308.04079

    Pith/arXiv arXiv 2023

  18. [18]

    Aesthetic subsets in laion 2170337258 samples, 2025

    Christoph Schuhmann. Aesthetic subsets in laion 2170337258 samples, 2025. URL http://captions. christoph-schuhmann.de/aesthetic_viz_laion_sac+logos+ava1-l14-linearMSE-en-2.37B.html . Retrieved May 16, 2025

    2025

  19. [19]

    Approximate convex decomposition for 3d meshes with collision-aware concavity and tree search

    Xinyue Wei, Minghua Liu, Zhan Ling, and Hao Su. Approximate convex decomposition for 3d meshes with collision-aware concavity and tree search. ACM Transactions on Graphics (TOG) , 41(4):1–18, 2022

    2022

  20. [20]

    Chang, Leonidas J

    Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Leonidas J. Guibas, and Hao Su. SAPIEN: A simulated part-based interactive environment. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2020

    2020

  21. [21]

    Pybullet, a python module for physics simulation for games, robotics and machine learning

    Erwin Coumans and Yunfei Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning. http://pybullet.org, 2016–2021

    2016

  22. [22]

    Isaac gym: High performance gpu-based physics simulation for robot learning, 2021

    Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. Isaac gym: High performance gpu-based physics simulation for robot learning, 2021

    2021

  23. [23]

    Mujoco: A physics engine for model-based control

    Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems , pages 5026–5033. IEEE, 2012. doi: 10.1109/IROS.2012.6386109

    doi:10.1109/iros.2012.6386109 2012

  24. [24]

    Genesis: A generative and universal physics engine for robotics and beyond, December 2024

    Genesis Authors. Genesis: A generative and universal physics engine for robotics and beyond, December 2024. URL https://github.com/Genesis-Embodied-AI/Genesis

    2024

  25. [25]

    Isaac Sim

    NVIDIA. Isaac Sim. https://github.com/isaac-sim/IsaacSim, 2024. Version 5.1.0, Apache-2.0 License

    2024

  26. [26]

    Roboverse: Towards a unified platform, dataset and benchmark for scalable and generalizable robot learning, 2025

    Haoran Geng, Feishi Wang, Songlin Wei, Yuyang Li, Bangjun Wang, Boshi An, Charlie Tianyue Cheng, Haozhe Lou, Peihao Li, Yen-Jen Wang, Yutong Liang, Dylan Goetting, Chaoyi Xu, Haozhe Chen, Yuxi Qian, Yiran Geng, Jiageng Mao, Weikang Wan, Mingtong Zhang, Jiangran Lyu, Siheng Zhao, Jiazhao Zhang, Jialiang Zhang, Chengyang Zhao, Haoran Lu, Yufei Ding, Ran Gon...

    Pith/arXiv arXiv 2025

  27. [27]

    3D AffordanceNet: A benchmark for visual object affordance understanding

    Shengheng Deng, Xun Xu, Chaozheng Wu, Ke Chen, and Kui Jia. 3D AffordanceNet: A benchmark for visual object affordance understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021

    2021

  28. [28]

    Where2Act: From pixels to actions for articulated 3D objects

    Kaichun Mo, Leonidas J Guibas, Mustafa Mukadam, Abhinav Gupta, and Shubham Tulsiani. Where2Act: From pixels to actions for articulated 3D objects. In Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021

    2021

  29. [29]

    P3-SAM: Native 3D part segmentation

    Changfeng Ma, Yang Li, Xinhao Yan, Jiachen Xu, Yunhan Yang, Chunshi Wang, Zibo Zhao, Yanwen Guo, Zhuo Chen, and Chunchao Guo. P3-SAM: Native 3D part segmentation. arXiv preprint arXiv:2509.06784 , 2025

    arXiv 2025

  30. [30]

    gpt-5.4 model

    OpenAI. gpt-5.4 model. https://developers.openai.com/api/docs/models/gpt-5.4, 2026. Accessed: 2026- 06-24

    2026

  31. [31]

    GraspGen: A diffusion-based framework for 6-DOF grasping with on-generator training

    Adithyavairavan Murali, Balakumar Sundaralingam, Yu-Wei Chao, Wentao Yuan, Jun Yamada, Mark Carlson, Fabio Ramos, Stan Birchfield, Dieter Fox, and Clemens Eppner. GraspGen: A diffusion-based framework for 6-DOF grasping with on-generator training. arXiv preprint arXiv:2507.13097 , 2025

    Pith/arXiv arXiv 2025

  32. [32]

    Holodeck: Language guided generation of 3d embodied ai environments

    Yue Yang, Fan-Yun Sun, Luca Weihs, Eli VanderBilt, Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, Chris Callison-Burch, Mark Yatskar, Aniruddha Kembhavi, and Christopher Clark. Holodeck: Language guided generation of 3d embodied ai environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2024

  33. [33]

    Layoutgpt: Compositional visual planning and generation with large language models

    Weixi Feng, Wanrong Zhu, Tsu-Jui Fu, Varun Jampani, Arjun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. Layoutgpt: Compositional visual planning and generation with large language models. In Advances in Neural Information Processing Systems , 2023. 22

    2023

  34. [34]

    Infinigen indoors: Photorealistic indoor scenes using procedural generation

    Alexander Raistrick, Lingjie Mei, Karhan Kayan, David Yan, Yiming Zuo, Beining Han, Hongyu Wen, Meenal Parakh, Stamatis Alexandropoulos, Lahav Lipson, Zeyu Ma, and Jia Deng. Infinigen indoors: Photorealistic indoor scenes using procedural generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 21783–...

    2024

  35. [35]

    OpenAI Codex CLI

    OpenAI. OpenAI Codex CLI. https://github.com/openai/codex, 2025. Accessed: 2026-05-11

    2025

  36. [36]

    Gemini CLI

    Google. Gemini CLI. https://github.com/google-gemini/gemini-cli , 2025. Accessed: 2026-05-11

    2025

  37. [37]

    BridgeData V2: A dataset for robot learning at scale

    Homer Walke, Kevin Black, Abraham Lee, Moo Jin Kim, Max Du, Chongyi Zheng, Tony Zhao, Philippe Hansen- Estruch, Quan Vuong, Andre He, et al. BridgeData V2: A dataset for robot learning at scale. In Conference on Robot Learning (CoRL) , 2023

    2023

  38. [38]

    Evaluating real-world robot manipulation policies in simulation

    Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sieh, Sean Kirmani, Sergey Levine, Jiajun Wu, Chelsea Finn, Hao Su, Quan Vuong, and Ted Xiao. Evaluating real-world robot manipulation policies in simulation. arXiv preprint arXiv:2405.05941 , 2024

    Pith/arXiv arXiv 2024

  39. [39]

    DreamFusion: Text-to-3D using 2D diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. DreamFusion: Text-to-3D using 2D diffusion. In International Conference on Learning Representations , 2023

    2023

  40. [40]

    Zero-1-to-3: Zero-shot one image to 3D object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-1-to-3: Zero-shot one image to 3D object. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9298–9309, 2023

    2023

  41. [41]

    LRM: Large reconstruction model for single image to 3D

    Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. LRM: Large reconstruction model for single image to 3D. In International Conference on Learning Representations, 2024

    2024

  42. [42]

    Gen2Sim: Scaling up robot learning in simulation with generative models

    Pushkal Katara, Zhou Xian, and Katerina Fragkiadaki. Gen2Sim: Scaling up robot learning in simulation with generative models. In International Conference on Robotics and Automation , 2024

    2024

  43. [43]

    PhysX-3D: Physical-grounded 3D asset generation

    Ziang Cao, Zhaoxi Chen, Liang Pan, and Ziwei Liu. PhysX-3D: Physical-grounded 3D asset generation. In Advances in Neural Information Processing Systems , 2025

    2025

  44. [44]

    PhysX-Anything: Simulation-ready physical 3D assets from single image

    Ziang Cao, Fangzhou Hong, Zhaoxi Chen, Liang Pan, and Ziwei Liu. PhysX-Anything: Simulation-ready physical 3D assets from single image. arXiv preprint arXiv:2511.13648 , 2025

    arXiv 2025

  45. [45]

    PhysForge: Generating physics-grounded 3D assets for interactive virtual world

    Yunhan Yang, Chunshi Wang, Junliang Ye, Yang Li, Zanxin Chen, Zehuan Huang, Yao Mu, Zhuo Chen, Chun- chao Guo, and Xihui Liu. PhysForge: Generating physics-grounded 3D assets for interactive virtual world. arXiv preprint arXiv:2605.05163 , 2026

    Pith/arXiv arXiv 2026

  46. [46]

    PhyScene: Physically interactable 3D scene synthesis for embodied AI

    Yandan Yang, Baoxiong Jia, Peiyuan Zhi, and Siyuan Huang. PhyScene: Physically interactable 3D scene synthesis for embodied AI. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    2024

  47. [47]

    Rein3D: Reinforced 3D indoor scene generation with panoramic video diffusion models

    Dehui Wang, Congsheng Xu, Rong Wei, Yue Shi, Shoufa Chen, Dingxiang Luo, Tianshuo Yang, Xiaokang Yang, Yusen Qin, Rui Tang, and Yao Mu. Rein3D: Reinforced 3D indoor scene generation with panoramic video diffusion models. arXiv preprint arXiv:2604.10578 , 2026

    Pith/arXiv arXiv 2026

  48. [48]

    Agentic 3D scene generation with spatially contextualized VLMs

    Xinhang Liu, Yu-Wing Tai, and Chi-Keung Tang. Agentic 3D scene generation with spatially contextualized VLMs. arXiv preprint arXiv:2505.20129 , 2025

    Pith/arXiv arXiv 2025

  49. [49]

    SegViGen: Repurposing 3D generative model for part segmentation

    Lin Li, Haoran Feng, Zehuan Huang, Haohua Chen, Wenbo Nie, Shaohua Hou, Keqing Fan, Pan Hu, Sheng Wang, Buyu Li, and Lu Sheng. SegViGen: Repurposing 3D generative model for part segmentation. arXiv preprint arXiv:2603.16869 , 2026

    Pith/arXiv arXiv 2026

  50. [50]

    ManiTwin: Scaling data-generation-ready digital object dataset to 100K

    Kaixuan Wang, Tianxing Chen, Jiawei Liu, Honghao Su, Shaolong Zhu, Minxuan Wang, Zixuan Li, Yue Chen, Huan-ang Gao, Yusen Qin, Jiawei Wang, Qixuan Zhang, Lan Xu, Jingyi Yu, Yao Mu, and Ping Luo. ManiTwin: Scaling data-generation-ready digital object dataset to 100K. arXiv preprint arXiv:2603.16866 , 2026

    arXiv 2026

  51. [51]

    Chat-Edit-3D: Interactive 3D scene editing via text prompts

    Shuangkang Fang, Yufeng Wang, Yi-Hsuan Tsai, Yi Yang, Wenrui Ding, Shuchang Zhou, and Ming-Hsuan Yang. Chat-Edit-3D: Interactive 3D scene editing via text prompts. In European Conference on Computer Vision, 2024

    2024

  52. [52]

    Blender – a 3D modelling and rendering package

    Blender Online Community. Blender – a 3D modelling and rendering package. https://www.blender.org, 2024

    2024

  53. [53]

    Autodesk Maya

    Autodesk Inc. Autodesk Maya. https://www.autodesk.com/products/maya, 2024

    2024

  54. [54]

    RT-2: Vision-language-action models transfer web knowledge to robotic control

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, A vinava Dubey, Chelsea Finn, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818 , 2023. 23

    Pith/arXiv arXiv 2023

  55. [55]

    OpenVLA: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Oier Mees, Suraj Gupta, Homer Walke, Joey Hejna, Ayzaan Wahid, Quan Vuong, Adam Gleave, et al. OpenVLA: An open-source vision-language-action model. arXiv preprint arXiv:2406.09246 , 2024

    Pith/arXiv arXiv 2024

  56. [56]

    π0: A vision-language-action flow model for general robot control

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. π0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164 , 2024

    Pith/arXiv arXiv 2024

  57. [57]

    GigaBrain-0: A world model-powered vision-language-action model

    Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Haoyun Li, Jie Li, Jiagang Zhu, Lv Feng, Peng Li, et al. GigaBrain-0: A world model-powered vision-language-action model. arXiv preprint arXiv:2510.19430, 2025

    arXiv 2025

  58. [58]

    HoloBrain-0 technical report

    Xuewu Lin, Tianwei Lin, Yun Du, Hongyu Xie, Yiwei Jin, Jiawei Li, Shijie Wu, Qingze Wang, Mengdi Li, Mengao Zhao, Ziang Li, Chaodong Huang, Hongzhe Bi, Lichao Huang, and Zhizhong Su. HoloBrain-0 technical report. arXiv preprint arXiv:2602.12062 , 2026

    arXiv 2026

  59. [59]

    Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew J. Davison. RLBench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Letters , 5(2):3019–3026, 2020

    2020

  60. [60]

    ManiSkill3: GPU parallelized robotics simulation and rendering for generalizable embodied AI

    Stone Tao, Fanbo Xiang, Arth Shukla, Yuzhe Qin, Xander Hinrichsen, Xiaodi Yuan, Chen Bao, Xinsong Lin, Yulin Liu, Tse-kai Chan, et al. ManiSkill3: GPU parallelized robotics simulation and rendering for generalizable embodied AI. arXiv preprint arXiv:2410.00425 , 2024

    Pith/arXiv arXiv 2024

  61. [61]

    Domain random- ization for transferring deep neural networks from simulation to the real world

    Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Domain random- ization for transferring deep neural networks from simulation to the real world. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 23–30, 2017

    2017

  62. [62]

    Embody4D: A generalist 4D world model for embodied AI

    Peiyan Tu, Hanxin Zhu, Jingwen Sun, Shaojie Ren, Cong Wang, Jiayi Luo, Xiaoqian Cheng, and Zhibo Chen. Embody4D: A generalist 4D world model for embodied AI. arXiv preprint arXiv:2605.01799 , 2026. 24

    Pith/arXiv arXiv 2026