Pith. sign in

REVIEW 4 minor 89 references

SPEAR: A Simulator for Photorealistic Embodied AI Research

T0 review · 0 major / 4 minor · reviewed 2026-07-10 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SPEAR turns any Unreal Engine application into a programmable photorealistic simulator by exposing over 14,000 engine functions to Python and delivering 1920×1080 images into NumPy at 73 frames per second.

desk verdict Practical systems win: SPEAR really does give order-of-magnitude more UE surface and faster NumPy delivery than AirSim/CARLA/UnrealCV+, with public code and clean tables. read the letter →

arxiv 2607.06701 v1 pith:HKXYZXXM submitted 2026-07-07 cs.CV cs.AIcs.GRcs.LGcs.RO

Mike Roberts , Renhan Wang , Rushikesh Zawar , Rachith Dey-Prakash , Quentin Leboutet , Stephan R. Richter , Matthias Müller , German Ros
show 5 more authors
Rui Tang Stefan Leutenegger Yannick Hold-Geoffroy Kalyan Sunkavalli Vladlen Koltun
This is my paper · ORCID
classification cs.CVcs.AIcs.GRcs.LGcs.RO
keywords UnrealEnginephotorealisticsimulationembodiedAIsyntheticdataruntimereflectionPythonbindingsground-truthrenderingproceduralcontentgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Existing photorealistic simulators built on Unreal Engine give researchers only a few hundred hand-crafted Python calls, move high-resolution images slowly back to user code, and usually ship as large monolithic applications that are hard to drop into existing projects. SPEAR claims to remove those barriers with a modular plugin that hooks Unreal’s runtime reflection system, automatically exposing more than 14,000 unique functions and 53,000 properties as ordinary Python objects. A transaction-based programming model lets users write graphs of engine work with arbitrary data dependencies and guarantees that each graph runs deterministically inside a single Unreal frame; shared memory then streams beauty images and ground-truth modalities (intrinsic decompositions, material IDs, shading parameters) straight into NumPy arrays at 73 FPS. A sympathetic reader cares because the combination of breadth, speed, and modularity makes it practical to drive humans, cars, robots, city-scale scenes, procedural content, multi-view faces, and external physics co-simulations from ordinary Python on unmodified Unreal projects.

What carries the argument

The transaction-based programming model: begin_frame / end_frame contexts that specify arbitrary dependency graphs of Unreal work, executed deterministically on the game thread inside a single frame, together with asynchronous futures, SpFunctions for zero-copy NumPy interchange, and interprocess shared memory.

What would settle it

An independent benchmark on the same Unreal sample projects that either matches SPEAR’s 73 FPS end-to-end NumPy path with a prior plugin under identical conditions, or demonstrates that a substantial fraction of common embodied-AI tasks require reflection-invisible engine APIs that SPEAR cannot reach without additional custom C++.

Watch

Extended reading notes

Core claim

By exposing Unreal Engine’s reflection system through a thin set of 193 hand-crafted server entry points and a client that treats reflected functions and properties as native Python objects, SPEAR delivers an order-of-magnitude increase in programmable functionality (14,485 UE functions, 53,537 variables) while rendering 1920×1080 photorealistic beauty images directly into a user’s NumPy array at 73 FPS—roughly 10–20 imes faster than prior UE plugins—and supplies ground-truth modalities unavailable in any earlier UE-based simulator.

Load-bearing premise

The whole approach assumes that Unreal’s existing reflection system plus a modest set of 193 hand-crafted entry points already surface essentially every function an embodied-AI researcher needs, so no further per-project C++ wrappers are required.

Editorial extensions

If this is right

  • Researchers can control multiple agents with distinct action spaces (humans, cars, robots) inside unmodified commercial Unreal sample projects without forking the engine.
  • City-scale photorealistic environments and Unreal’s procedural content generation systems become fully scriptable from Python.
  • Synchronized multi-view capture of detailed human characters and interactive co-simulation with external physics engines become practical at interactive rates.
  • The same Python interface works for standalone packaged applications, live editor sessions, and the path tracer.
  • Natural-language scene editing becomes feasible by letting an AI coding assistant emit SPEAR transaction programs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because new BlueprintCallable engine features automatically appear through reflection, SPEAR’s surface area can grow with Unreal itself without library updates, lowering long-term maintenance for research groups.
  • The combination of speed and dense ground-truth modalities positions the system as a candidate data engine for large vision-language and world models that need photorealistic multi-view supervision at scale.
  • The ability to keep the Unreal application fully interactive while paused between transactions may change how researchers debug long-horizon embodied policies.
  • If the thin reflection-bridge pattern proves portable, similar lightweight clients could be built for other engines that expose comparable runtime metadata.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

0 major / 4 minor

Summary. SPEAR is a modular Python library and UE plugin suite that programmatically controls any Unreal Engine application by exposing UE’s runtime reflection system. It claims an order-of-magnitude increase in programmable surface (over 14K unique UE functions and 53K properties via only 193 hand-crafted server entry points), photorealistic 1920×1080 beauty-image delivery into NumPy arrays at up to 73 FPS with shared memory and asynchronous transactions, and ground-truth modalities (non-diffuse intrinsic decomposition, material IDs, PBR parameters) unavailable in prior UE simulators. A transaction model (begin_frame / end_frame contexts) lets users specify arbitrary dependency graphs of UE work that execute deterministically inside a single game-thread frame. Utility is shown through multi-agent control across Epic sample projects, PCG manipulation, MetaHuman multi-view rendering, MuJoCo co-simulation, and LLM-driven scene editing.

Significance. If the measured gains hold, SPEAR removes two long-standing bottlenecks for photorealistic embodied-AI research: the need for large custom UE forks and the high communication overhead of existing plugins. The reflection-based design, modest code footprint (Table 1), public repository, and controlled head-to-head FPS numbers (Tables 2–3) make the contribution immediately usable and falsifiable. The transaction model and SpFunction/shared-memory path are concrete engineering advances that other simulators can adopt. The work therefore has clear potential as infrastructure for training agents, generating synthetic data, and bridging vision-language models to high-fidelity virtual worlds.

minor comments (4)
  1. [Table 1] Table 1 caption and surrounding text should state the exact UE version and the precise reflection-query method used to obtain the 14 485 / 53 537 counts so that later readers can reproduce the enumeration.
  2. [§6] In §5 the phrase “over 150 megapixels per second” appears only in the conclusions; a short calculation linking the 73 FPS 1920×1080 figure to that claim would improve clarity.
  3. [Fig. 5] Figure 5’s timeline diagram would benefit from an explicit legend distinguishing “queue work”, “execute work”, and “blocked” states; the current shading is dense.
  4. [§4] A brief note on whether the same shared-memory path works under Linux / macOS (or any platform-specific caveats) would help multi-platform users.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: engineering systems paper whose claims are empirical measurements and implementation facts, not derived predictions.

full rationale

SPEAR is a systems contribution whose central claims (14K+ reflection-visible UE functions exposed via 193 hand-crafted entry points, 73 FPS 1920 imes1080 NumPy delivery with shared memory/async transactions, novel ground-truth modalities, and a transaction programming model) are established by direct implementation, runtime enumeration of UE’s reflection system, and controlled end-to-end timing against external baselines (Tables 1–3). There are no fitted parameters re-labeled as predictions, no self-definitional equations, no uniqueness theorems imported from the authors’ prior work, and no ansatz smuggled via citation. The example applications simply exercise the exposed surface; they do not close a logical loop. The paper is therefore self-contained against external benchmarks and exhibits zero circularity of the kinds enumerated.

Assumptions & free parameters 0 free parameters · 3 assumptions · 2 invented entities

As a systems paper the central claims rest on Unreal Engine’s existing reflection and component architecture plus a modest set of engineering choices (shared memory, task queues, SpFunctions). No free parameters are fitted to scientific data; the only ‘invented’ constructs are the programming abstractions needed to surface that architecture cleanly to Python.

assumptions (3)
  • domain assumption Unreal Engine’s runtime reflection system (UFUNCTION/UPROPERTY) correctly enumerates and invokes essentially all engine functionality that embodied-AI users require.
    Invoked throughout §3–4 and Table 1; the 14K-function count is obtained by iterating the reflection system.
  • domain assumption A single TCP/IP client–server pair plus inter-process shared memory can transfer high-resolution images without becoming the dominant bottleneck on contemporary hardware.
    Underpins the 73 FPS claim (Table 2) and the shared-memory design in §4.
  • ad hoc to paper Deterministic execution of arbitrary Python-specified work graphs inside one UE frame is achievable by queuing tasks on the game thread at begin/end frame boundaries.
    Core of the programming model (§3, Fig. 5); not a standard UE guarantee.
invented entities (2)
  • begin_frame / end_frame transaction model
    purpose: Lets users express complex dependency graphs of UE work that execute deterministically inside a single engine frame while remaining fully asynchronous with respect to the game thread.
    Introduced in §3; no prior UE Python interface provides this guarantee.
  • SpFunction dispatch with shared-memory NumPy arrays
    purpose: Allows large image tensors to cross the Python–UE boundary with zero extra copies while still appearing as ordinary Python callables.
    Defined in §4; essential for the measured rendering throughput.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SPEAR: A Simulator for Photorealistic Embodied AI Research." pith.science (2026). https://pith.science/paper/HKXYZXXM

@misc{pith2026260706701,
  author       = {Pith},
  title        = {Pith review of: SPEAR: A Simulator for Photorealistic Embodied AI Research},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HKXYZXXM}},
  note         = {Machine review of arXiv:2607.06701}
}
read the original abstract

Interactive simulators have become powerful tools for training embodied agents and generating synthetic visual data, but existing photorealistic simulators suffer from limited generality, programmability, and rendering speed. We address these limitations by introducing SPEAR: A Simulator for Photorealistic Embodied AI Research. At its core, SPEAR is a Python library that can connect to, and programmatically control, any Unreal Engine (UE) application via a modular plugin architecture. SPEAR exposes over 14K unique UE functions to Python, representing an order-of-magnitude increase in programmable functionality over existing UE-based simulators. Additionally, a single SPEAR instance can render 1920x1080 photorealistic beauty images directly into a user's NumPy array at 73 frames per second - an order of magnitude faster than existing UE plugins - while also providing ground truth image modalities that are not available in any existing UE-based simulator (e.g., a non-diffuse intrinsic image decomposition, material IDs, and physically based shading parameters). Finally, SPEAR introduces an expressive high-level programming model that enables users to specify complex graphs of UE work with arbitrary data dependencies among work items, and to execute these graphs deterministically within a single UE frame. We demonstrate the utility of SPEAR through a diverse collection of example applications: controlling multiple embodied agents with distinct action spaces (e.g., humans, cars, and robots) across several in-the-wild UE projects; rendering photorealistic city-scale environments; manipulating UE's procedural content generation systems; rendering synchronized multi-view images of detailed human faces; coordinating an interactive co-simulation with the MuJoCo physics simulator; and editing scenes with natural language via an AI coding assistant.

Figures

Figures reproduced from arXiv: 2607.06701 by the authors.

Figure 1
Figure 1. SPEAR is a Python library that can connect to, and programmatically con￾trol, any Unreal Engine (UE) application via a modular plugin architecture. SPEAR exposes over 14K unique UE functions, representing an order-of-magnitude increase in programmable functionality over existing simulators. We demonstrate the flexibility of SPEAR by using it to control 6 distinct embodied agents (each with a different action space) … view at source ↗
Figure 2
Figure 2. SPEAR includes a customizable camera sensor that can render 1920×1080 photorealistic beauty images (left) directly into a user’s NumPy array at 73 frames per second – an order of magnitude faster than existing UE plugins – while also providing ground truth image modalities that are not available in any existing UE-based simu￾lator. For example, the SPEAR camera sensor can render all of the image modalities in the Hy… view at source ↗
Figure 3
Figure 3. (a, b): We demonstrate several fundamental concepts in the SPEAR pro￾gramming model with a simple example program that spawns a set of coordinate axes in an indoor environment. In our programming model, graphs of UE work are specified as transactions. In particular, the user specifies a transaction by defining a begin_frame context followed by an end_frame context. Within each context, the user specifies a graph of … view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: We demonstrate the flexibility of our programming model by using it to im￾plement a variety of synchronization strategies found in existing simulators. For each strategy, we implement a simplified OpenAI Gym step function [17], which takes an action as input, steps a s…
Figure 5
Figure 5. Figure 5: Illustration of how the asynchronous operations in our programming model can be used to completely avoid blocking the UE game thread, thereby enabling a UE application to execute user Python code at native frame rates. Our Python client transforms a user’s Python code …
Figure 6
Figure 6. Figure 6: We demonstrate the flexibility of SPEAR by using it to programmtically ma￾nipulate the ElectricDreams sample project from Epic Games. (a): We control UE’s procedural content generation (PCG) system by translating the main PCG entity in this scene (the rock structure in…
Figure 7
Figure 7. Figure 7: SPEAR can be used in co-simulation applications with external physics simu￾lators. In this application, we interactively control the MuJoCo physics simulator [72] using the default MuJoCo viewer, e.g., by applying a force to the leftmost chair (red arrow). In real-time…
Figure 8
Figure 8. Figure 8: We demonstrate the flexibility of our camera sensor by using it to render synchronized multi-view images of a detailed human character in the MetaHumans sample project from Epic Games. Summary Our programming model is expressive because it enables users to specify comp…
Figure 9
Figure 9. Figure 9: We demonstrate the flexibility of SPEAR by using it to implement an agentic natural-language scene editing application, where a vision-and-language coding assis￾tant manipulates a UE scene by iteratively writing SPEAR programs in response to user text prompts. The Serv…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

89 extracted references · 89 canonical work pages

  1. [1]

    Epic Games Developer Documentation 5

    Blueprints visual scripting. Epic Games Developer Documentation 5

    work page

  2. [2]

    Epic Games Developer Documentation 13

    Console variables and commands. Epic Games Developer Documentation 13

    work page

  3. [3]

    NVIDIA Isaac Sim.https://developer.nvidia.com/isaac/sim2

    work page

  4. [4]

    Epic Games Developer Documentation 11

    An overview of components. Epic Games Developer Documentation 11

    work page

  5. [5]

    Panda3D.https://www.panda3d.org2

    work page

  6. [6]

    Epic Games Developer Documentation 5

    Scripting the Unreal Editor using Python. Epic Games Developer Documentation 5

    work page

  7. [7]

    Unity.https://unity.com2

    work page

  8. [8]

    Unreal Engine.https://www.unrealengine.com2

    work page

Show all 89 references
  1. [9]

    Epic Games Developer Documentation 3, 11

    Unreal Engine reflection system. Epic Games Developer Documentation 3, 11

  2. [10]

    Game World Observer (2025) 2

    Black Myth: Wukong tops 25 million copies sold as its merchandise sales skyrocket in China. Game World Observer (2025) 2

    2025

  3. [11]

    Epic Games Store News (2025) 2

    Clair Obscur takes home 9 Game Awards including Game of the Year. Epic Games Store News (2025) 2

    2025

  4. [12]

    In: NeurIPS 2024 2

    Bachmann, R., Kar, O.F., Mizrahi, D., Garjani, A., Gao, M., Griffiths, D., Hu, J., Dehghan, A., Zamir, A.: 4M-21: An any-to-any vision model for tens of tasks and modalities. In: NeurIPS 2024 2

    2024

  5. [13]

    In: ICCV 2025 2

    Bai, J., Xia, M., Fu, X., Wang, X., Mu, L., Cao, J., Liu, Z., Hu, H., Bai, X., Wan, P., Zhang, D.: ReCamMaster: Camera-controlled generative rendering from a single video. In: ICCV 2025 2

    2025

  6. [14]

    arXiv 2016 4

    Beattie, C., Leibo, J.Z., Teplyashin, D., Ward, T., Wainwright, M., Küttler, H., Lefrancq, A., Green, S., Valdés, V., Sadik, A., Schrittwieser, J., Anderson, K., York, S., Cant, M., Cain, A., Bolton, A., Gaffney, S., King, H., Hassabis, D., Legg, S., Petersen, S.: DeepMind Lab...

    2016

  7. [15]

    Journal of Artificial Intelligence Research47(1) (2013) 4

    Bellemare, M.G., Naddaf, Y., Veness, J., Bowling, M.: The arcade learning environ- ment: An evaluation platform for general agents. Journal of Artificial Intelligence Research47(1) (2013) 4

    2013

  8. [16]

    In: NeurIPS 2023 2, 9

    Bordes, F., Shekhar, S., Ibrahim, M., Bouchacourt, D., Vincent, P., Morcos, A.S.: PUG:Photorealisticandsemanticallycontrollablesyntheticdataforrepresentation learning. In: NeurIPS 2023 2, 9

    2023

  9. [17]

    arXiv 2016 5

    Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: OpenAI Gym. arXiv 2016 5

    2016

  10. [18]

    In: CoRL 2017 1, 2, 5, 7, 9, 12, 14

    Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: An open urban driving simulator. In: CoRL 2017 1, 2, 5, 7, 9, 12, 14

    2017

  11. [19]

    In: NeurIPS 2022 Datasets and Benchmarks Track 4

    Fan, L., Wang, G., Jiang, Y., Mandlekar, A., Yang, Y., Zhu, H., Tang, A., Huang, D.A., Zhu, Y., Anandkumar, A.: MineDojo: Building open-ended embodied agents with internet-scale knowledge. In: NeurIPS 2022 Datasets and Benchmarks Track 4

    2022

  12. [20]

    arXiv 2026 2

    Gabeur, V., Long, S., Peng, S., Voigtlaender, P., Sun, S., Bao, Y., Truong, K., Wang, Z., Zhou, W., Barron, J.T., Genova, K., Kannen, N., Ben, S., Li, Y., Guo, M., Yogin, S., Gu, Y., Chen, H., Wang, O., Xie, S., Zhou, H., He, K., Funkhouser, T., Alayrac, J.B., Soricut, R.: Ima...

    2026

  13. [21]

    In: NeurIPS 2021 Datasets and Benchmarks Track 2, 9 SPEAR: A Simulator for Photorealistic Embodied AI Research 17

    Gan, C., Schwartz, J., Alter, S., Mrowca, D., Schrimpf, M., Traer, J., Freitas, J.D., Kubilius, J., Bhandwaldar, A., Haber, N., Sano, M., Kim, K., Wang, E., Lingel- bach, M., Curtis, A., Feigelis, K., Bear, D.M., Gutfreund, D., Cox, D., Torralba, A., DiCarlo, J.J., Tenenbaum, ...

    2021

  14. [22]

    In: CVPR 2024 2

    Ge, Y., Tang, Y., Xu, J., Gokmen, C., Li, C., Ai, W., Martinez, B.J., Aydin, A., Anvari, M., Chakravarthy, A.K., Yu, H.X., Wong, J., Srivastava, S., Lee, S., Zha, S., Itti, L., Li, Y., Martin-Martin, R., Liu, M., Zhang, P., Zhang, R., Fei-Fei, L., Wu, J.: BEHAVIOR vision suite...

    2024

  15. [23]

    In: ICCV 2023 2

    Gong, R., Huang, J., Zhao, Y., Geng, H., Gao, X., Wu, Q., Ai, W., Zhou, Z., Terzopoulos, D., Zhu, S.C., Jia, B., Huang, S.: Arnold: A benchmark for language- grounded task learning with continuous states in realistic 3D scenes. In: ICCV 2023 2

    2023

  16. [24]

    Science Robotics 4(26) (2019) 2

    Hwangbo,J.,Lee,J.,Dosovitskiy,A.,Bellicoso,D.,Tsounis,V.,Koltun,V.,Hutter, M.: Learning agile and dynamic motor skills for legged robots. Science Robotics 4(26) (2019) 2

    2019

  17. [25]

    Science364(6443) (2019) 4

    Jaderberg, M., Czarnecki, W.M., Dunning, I., Marris, L., Lever, G., Castañeda, A.G., Beattie, C., Rabinowitz, N.C., Morcos, A.S., Ruderman, A., Sonnerat, N., Green, T., Deason, L., Leibo, J.Z., Silver, D., Hassabis, D., Kavukcuoglu, K., Grae- pel, T.: Human-level performance i...

    2019

  18. [26]

    com/wjakob/nanobind9

    Jakob, W.: nanobind: tiny and efficient C++/Python bindings.https://github. com/wjakob/nanobind9

  19. [27]

    In: IJCAI 2016 4

    Johnson, M., Hofmann, K., Hutton, T., Bignell, D.: The Malmo platform for arti- ficial intelligence experimentation. In: IJCAI 2016 4

    2016

  20. [28]

    arXiv 2018 2, 9

    Juliani, A., Berges, V.P., Teng, E., Cohen, A., Harper, J., Elion, C., Goy, C., Gao, Y., Henry, H., Mattar, M., Lange, D.: Unity: A general platform for intelligent agents. arXiv 2018 2, 9

    2018

  21. [29]

    In: SIGGRAPH 2021 Course on Advances in Real-Time Rendering and Games 2

    Karis, B., Stubbe, R., Wihlidal, G.: A deep dive into Nanite virtualized geometry. In: SIGGRAPH 2021 Course on Advances in Real-Time Rendering and Games 2

    2021

  22. [30]

    Nature 620(7976) (2023) 1

    Kaufmann, E., Bauersfeld, L., Loquercio, A., Müller, M., Koltun, V., Scara- muzza, D.: Champion-level drone racing using deep reinforcement learning. Nature 620(7976) (2023) 1

    2023

  23. [31]

    In: CoRL 2018 1

    Kaufmann, E., Loquercio, A., Ranftl, R., Dosovitskiy, A., Koltun, V., Scaramuzza, D.: Deep drone racing: Learning agile flight in dynamic environments. In: CoRL 2018 1

    2018

  24. [32]

    In: RSS 2020 1

    Kaufmann, E., Loquercio, A., Ranftl, R., Müller, M., Koltun, V., Scaramuzza, D.: Deep drone acrobatics. In: RSS 2020 1

    2020

  25. [33]

    In: CVPR 2024 2

    Ke, B., Obukhov, A., Huang, S., Metzger, N., Daudt, R.C., Schindler, K.: Re- purposing diffusion-based image generators for monocular depth estimation. In: CVPR 2024 2

    2024

  26. [34]

    In: Computa- tional Intelligence and Games 2016 4

    Kempka, M., Wydmuch, M., Runc, G., Toczek, J., Jaśkowski, W.: ViZDoom: A Doom-based AI research platform for visual reinforcement learning. In: Computa- tional Intelligence and Games 2016 4

    2016

  27. [35]

    In: ICCV 2023 2

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., Dollár, P., Girshick, R.: Segment anything. In: ICCV 2023 2

    2023

  28. [36]

    arXiv 2017 2, 9

    Kolve, E., Mottaghi, R., Han, W., VanderBilt, E., Weihs, L., Herrasti, A., Gordon, D., Zhu, Y., Gupta, A., Farhadi, A.: AI2-THOR: An interactive 3D environment for visual AI. arXiv 2017 2, 9

    2017

  29. [37]

    In: CVPR 2018 4

    Krähenbühl, P.: Free supervision from video games. In: CVPR 2018 4

    2018

  30. [38]

    In: Game Developers Con- ference 2023 2 18 M

    Langmead, A.: Procedural content generation in UE5. In: Game Developers Con- ference 2023 2 18 M. Roberts et al

    2023

  31. [39]

    Science Robotics5(47) (2020) 2

    Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., Hutter, M.: Learning quadrupedal locomotion over challenging terrain. Science Robotics5(47) (2020) 2

    2020

  32. [40]

    In: ICML 2016 2

    Lerer, A., Gross, S., Fergus, R.: Learning physical intuition of block towers by example. In: ICML 2016 2

    2016

  33. [41]

    In: CoRL 2021 4

    Li, C., Xia, F., Martín-Martín, R., Lingelbach, M., Srivastava, S., Shen, B., Vainio, K.E., Gokmen, C., Dharan, G., Jain, T., Kurenkov, A., Liu, C.K., Gweon, H., Wu, J.,Fei-Fei,L.,Savarese,S.:iGibson2.0:Object-centricsimulationforrobotlearning of everyday household tasks. In: ...

    2021

  34. [42]

    In: CoRL 2022 2

    Li, C., Zhang, R., Wong, J., Gokmen, C., Srivastava, S., Martín-Martín, R., Wang, C.,Levine,G.,Ai,W.,Martinez,B.,Yin,H.,Lingelbach,M.,Hwang,M.,Hiranaka, A., Garlanka, S., Aydin, A., Lee, S., Sun, J., Anvari, M., Sharma, M., Bansal, D., Hunter, S., Kim, K.Y., Lou, A., Matthews,...

    2022

  35. [43]

    Transactions on Robotics36(1) (2020) 1

    Loquercio, A., Kaufmann, E., Ranftl, R., Dosovitskiy, A., Koltun, V., Scaramuzza, D.: Deep drone racing: From simulation to reality with domain randomization. Transactions on Robotics36(1) (2020) 1

    2020

  36. [44]

    Science Robotics6(59) (2021) 1

    Loquercio, A., Kaufmann, E., Ranftl, R., Müller, M., Koltun, V., Scaramuzza, D.: Learning high-speed flight in the wild. Science Robotics6(59) (2021) 1

    2021

  37. [45]

    In: OOPSLA 1987 3

    Maes, P.: Concepts and experiments in computational reflection. In: OOPSLA 1987 3

    1987

  38. [46]

    arXiv 2026 4

    Magne, L., Awadalla, A., Wang, G., Xu, Y., Belofsky, J., Hu, F., Kim, J., Schmidt, L., Gkioxari, G., Kautz, J., Yue, Y., Choi, Y., Zhu, Y., Fan, L.J.: NitroGen: An open foundation model for generalist gaming agents. arXiv 2026 4

    2026

  39. [47]

    In: IJCNN 2021 2, 9

    Martinez-Gonzalez, P., Oprea, S., Castro-Vargas, J.A., Garcia-Garcia, A., Orts- Escolano, S., Garcia-Rodriguez, J., Vincze, M.: UnrealROX+: An improved tool for acquiring synthetic data from virtual 3D environments. In: IJCNN 2021 2, 9

    2021

  40. [48]

    Virtual Reality24(2019) 2, 9

    Martinez-Gonzalez, P., Oprea, S., Garcia-Garcia, A., Jover-Alvarez, A., Orts- Escolano, S., Garcia-Rodriguez, J.: UnrealROX: An extremely photorealistic vir- tual reality environment for robotics simulations and synthetic data generation. Virtual Reality24(2019) 2, 9

    2019

  41. [49]

    In: CVPR 2024 2

    Mazur, K., Bae, G., Davison, A.J.: SuperPrimitive: Scene reconstruction at a prim- itive level. In: CVPR 2024 2

    2024

  42. [50]

    Science Robotics 7(62) (2022) 2

    Miki, T., Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., Hutter, M.: Learning robust perceptive locomotion for quadrupedal robots in the wild. Science Robotics 7(62) (2022) 2

    2022

  43. [51]

    In: NeurIPS 2023 2

    Mizrahi, D., Bachmann, R., Kar, O.F., Yeo, T., Gao, M., Dehghan, A., Zamir, A.: 4M: Massively multimodal masked modeling. In: NeurIPS 2023 2

    2023

  44. [52]

    Nature518(7540) (2015) 4

    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves,A.,Riedmiller,M.,Fidjeland,A.K.,Ostrovski,G.,Petersen,S.,Beattie,C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep ...

    2015

  45. [53]

    In: CoRL 2018 1

    Müller, M., Dosovitskiy, A., Ghanem, B., Koltun, V.: Driving policy transfer via modularity and abstraction. In: CoRL 2018 1

    2018

  46. [54]

    Narkowicz, K., Costa, T.: MegaLights: Stochastic direct lighting in Unreal Engine

  47. [55]

    In: SIGGRAPH 2025 Advances in Real-Time Rendering in Games 2 SPEAR: A Simulator for Photorealistic Embodied AI Research 19

    2025

  48. [56]

    arXiv 2019 2

    OpenAI, Akkaya, I., Andrychowicz, M., Chociej, M., Litwin, M., McGrew, B., Petron, A., Paino, A., Plappert, M., Powell, G., Ribas, R., Schneider, J., Tezak, N., Tworek, J., Welinder, P., Weng, L., Yuan, Q., Zaremba, W., Zhang, L.: Solving Rubik’s Cube with a robot hand. arXiv 2019 2

    2019

  49. [57]

    OpenAI, Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., Farhi, D., Fischer, Q., Hashme, S., Hesse, C., Józefowicz, R., Gray, S., Olsson, C., Pachocki, J., Petrov, M., d. O. Pinto, H.P., Raiman, J., Salimans, T., Schlatter, J., Schneider, J., Sidor, S....

    2019

  50. [58]

    In: ICLR 2024 4

    Puig, X., Undersander, E., Szot, A., Cote, M.D., Yang, T.Y., Partsey, R., Desai, R., Clegg, A.W., Hlavac, M., Min, S.Y., Vondruš, V., Gervet, T., Berges, V.P., Turner, J.M., Maksymets, O., Kira, Z., Kalakrishnan, M., Malik, J., Chaplot, D.S., Jain, U., Batra, D., Rai, A., Mott...

    2024

  51. [59]

    In: Multimedia 2017 2, 5, 9, 10, 11, 12, 13, 14

    Qiu, W., Zhong, F., Zhang, Y., Qiao, S., Xiao, Z., Kim, T.S., Wang, Y.: UnrealCV: Virtual worlds for computer vision. In: Multimedia 2017 2, 5, 9, 10, 11, 12, 13, 14

    2017

  52. [60]

    In: ICLR 2025 2

    Ravi, N., Gabeur, V., Hu, Y.T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L., Mintun, E., Pan, J., Alwala, K.V., Carion, N., Wu, C.Y., Girshick, R., Dollár, P., Feichtenhofer, C.: SAM 2: Segment anything in images and videos. In: ICLR 2025 2

    2025

  53. [61]

    In: ECCV 2016 1, 4

    Richter, S., Vineet, V., Roth, S., Koltun, V.: Playing for data: Ground truth from computer games. In: ECCV 2016 1, 4

    2016

  54. [62]

    In: ICCV 2017 1, 4

    Richter, S.R., Hayder, Z., Koltun, V.: Playing for benchmarks. In: ICCV 2017 1, 4

    2017

  55. [63]

    In: ICCV 2021 2, 3

    Roberts, M., Ramapuram, J., Ranjan, A., Kumar, A., Bautista, M.A., Paczan, N., Webb, R., Susskind, J.M.: Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. In: ICCV 2021 2, 3

    2021

  56. [64]

    Python Enhance- ment Proposals 10

    van Rossum, G., Coghlan, A.: PEP 343: The “with” statement. Python Enhance- ment Proposals 10

  57. [65]

    arXiv 2017 4

    Savva, M., Chang, A.X., Dosovitskiy, A., Funkhouser, T., Koltun, V.: MINOS: Multimodal indoor simulator for navigation in complex environments. arXiv 2017 4

    2017

  58. [66]

    In: ICCV 2019 4

    Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., Malik, J., Parikh, D., Batra, D.: Habitat: A platform for embodied AI research. In: ICCV 2019 4

    2019

  59. [67]

    XML Press (2011) 12

    Schäling, B.: The Boost C++ Libraries. XML Press (2011) 12

    2011

  60. [68]

    In: Field and Service Robotics 2017 2, 5, 9, 12, 14

    Shah, S., Dey, D., Lovett, C., Kapoor, A.: AirSim: High-fidelity visual and physical simulation for autonomous vehicles. In: Field and Service Robotics 2017 2, 5, 9, 12, 14

    2017

  61. [69]

    In: IROS 2021 4

    Shen, B., Xia, F., Li, C., Martín-Martín, R., Fan, L., Wang, G., Pérez-D’Arpino, C., Buch, S., Srivastava, S., Tchapmi, L.P., Tchapmi, M.E., Vainio, K., Wong, J., Fei-Fei, L., Savarese, S.: iGibson 1.0: A simulation environment for interactive tasks in large realistic scenes. ...

    2021

  62. [70]

    arXiv 2025 2

    Siméoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khalidov, V., Szafraniec, M., Yi, S., Ramamonjisoa, M., Massa, F., Haziza, D., Wehrstedt, L., Wang, J., Darcet, T., Moutakanni, T., Sentana, L., Roberts, C., Vedaldi, A., Tolan, J., Brandt, J., Couprie, ...

    2025

  63. [71]

    Szelei, T.: rpclib: Modern msgpack-rpc for C++.http://rpclib.net9

  64. [72]

    Roberts et al

    Szot, A., Clegg, A., Undersander, E., Wijmans, E., Zhao, Y., Turner, J., Maestre, N.,Mukadam,M.,Chaplot,D.,Maksymets,O.,Gokaslan,A.,Vondrus,V.,Dharur, 20 M. Roberts et al. S., Meier, F., Galuba, W., Chang, A., Kira, Z., Koltun, V., Malik, J., Savva, M., Batra, D.: Habitat 2.0:...

    2021

  65. [73]

    In: IROS 2012 3, 5, 8

    Todorov, E., Erez, T., Tassa, Y.: MuJoCo: A physics engine for model-based con- trol. In: IROS 2012 3, 5, 8

    2012

  66. [74]

    In: NeurIPS 2024 2

    Tong, S., Brown, E., Wu, P., Woo, S., Middepogu, M., Akula, S.C., Yang, J., Yang, S., Iyer, A., Pan, X., Wang, Z., Fergus, R., LeCun, Y., Xie, S.: Cambrian-1: A fully open, vision-centric exploration of multimodal LLMs. In: NeurIPS 2024 2

    2024

  67. [75]

    Addison-Wesley Professional, 2 edn

    Vandevoorde, D., Josuttis, N.M., Gregor, D.: C++ Templates: The Complete Guide. Addison-Wesley Professional, 2 edn. (2017) 10

    2017

  68. [76]

    Nature575(2019) 1, 4

    Vinyals, O., Babuschkin, I., Czarnecki, W.M., Mathieu, M., Dudzik, A., Chung, J., Choi, D.H., Powell, R., Ewalds, T., Georgiev, P., Oh, J., Horgan, D., Kroiss, M., Danihelka, I., Huang, A., Sifre, L., Cai, T., Agapiou, J.P., Jaderberg, M., Vezhn- evets, A.S., Leblond, R., Pohl...

    2019

  69. [77]

    arXiv 2017 1, 4

    Vinyals, O., Ewalds, T., Bartunov, S., Georgiev, P., Vezhnevets, A.S., Yeo, M., Makhzani, A., Küttler, H., Agapiou, J., Schrittwieser, J., Quan, J., Gaffney, S., Petersen, S., Simonyan, K., Schaul, T., van Hasselt, H., Silver, D., Lillicrap, T., Calderone, K., Keet, P., Brunas...

    2017

  70. [78]

    In: CVPR 2025 2

    Wang, J., Chen, M., Karaev, N., Vedaldi, A., Rupprecht, C., Novotny, D.: VGGT: Visual geometry grounded transformer. In: CVPR 2025 2

    2025

  71. [79]

    In: CVPR 2026 2

    Wang, J., Chen, M., Zhang, S., Karaev, N., Schönberger, J., Labatut, P., Bo- janowski, P., Novotny, D., Vedaldi, A., Rupprecht, C.: VGGT-Ω. In: CVPR 2026 2

    2026

  72. [80]

    In: SIGGRAPH 2022 Advances in Real-Time Rendering in Games 2

    Wright, D., Narkowicz, K., Kelly, P.: Lumen: Real-time global illumination in Un- real Engine 5. In: SIGGRAPH 2022 Advances in Real-Time Rendering in Games 2

    2022

  73. [81]

    In: ICLR 2025 2

    Wu, W., He, H., He, J., Wang, Y., Duan, C., Liu, Z., Li, Q., Zhou, B.: MetaUrban: An embodied AI simulation platform for urban micromobility. In: ICLR 2025 2

    2025

  74. [82]

    In: CVPR 2025 2

    Wu, W., He, H., Zhang, C., He, J., Zhao, S.Z., Gong, R., Li, Q., Zhou, B.: Towards autonomous micromobility through scalable urban simulation. In: CVPR 2025 2

    2025

  75. [83]

    Nature602(2022) 1, 4

    Wurman, P.R., Barrett, S., Kawamoto, K., MacGlashan, J., Subramanian, K., Walsh, T.J., Capobianco, R., Devlic, A., Eckert, F., Fuchs, F., Gilpin, L., Khan- delwal, P., Kompella, V., Lin, H., MacAlpine, P., Oller, D., Seno, T., Sherstan, C., Thomure, M.D., Aghabozorgi, H., Barr...

    2022

  76. [84]

    In: CVPR 2018 4

    Xia, F., Zamir, A.R., He, Z.Y., Sax, A., Malik, J., Savarese, S.: Gibson Env: Real- world perception for embodied agents. In: CVPR 2018 4

    2018

  77. [85]

    In: CVPR 2020 4

    Xiang, F., Qin, Y., Mo, K., Xia, Y., Zhu, H., Liu, F., Liu, M., Jiang, H., Yuan, Y., Wang, H., Yi, L., Chang, A.X., Guibas, L.J., Su, H.: SAPIEN: A simulated part-based interactive environment. In: CVPR 2020 4

    2020

  78. [86]

    arXiv 2025 2 SPEAR: A Simulator for Photorealistic Embodied AI Research 21

    Yang, S., Yang, J., Huang, P., Brown, E., Yang, Z., Yu, Y., Tong, S., Zheng, Z., Xu, Y., Wang, M., Lu, D., Fergus, R., LeCun, Y., Fei-Fei, L., Xie, S.: Cambrian-S: Towards spatial supersensing in video. arXiv 2025 2 SPEAR: A Simulator for Photorealistic Embodied AI Research 21

    2025

  79. [87]

    In: NeurIPS 2025 2

    Ye, X., Ren, J., Zhuang, Y., He, X., Liang, Y., Yang, Y., Zhong, X., Dogra, M., Liu, E., Benavente, K., Nagaraju, R.M., Sharma, D.V., Ma, Z., Shu, T., Hu, Z., Qin, L.: SimWorld: An open-ended simulator for agents in physical and social worlds. In: NeurIPS 2025 2

    2025

  80. [88]

    In: ICCV 2025 2, 5, 9, 10, 11, 12, 13, 14

    Zhong, F., Wu, K., Wang, C., Chen, H., Ci, H., Li, Z., Wang, Y.: UnrealZoo: Enriching photo-realistic virtual worlds for embodied AI. In: ICCV 2025 2, 5, 9, 10, 11, 12, 13, 14

    2025

  81. [89]

    Zhou, B., Krähenbühl, P., Koltun, V.: Does computer vision matter for action? Science Robotics4(30) (2018) 1, 4

    2018

Pith tools

Reviewed July 10, 2026 · model on record in the stance chip above.