REVIEW 23 cited by
Unity: A General Platform for Intelligent Agents
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent advances in artificial intelligence have been driven by the presence of increasingly realistic and complex simulated environments. However, many of the existing environments provide either unrealistic visuals, inaccurate physics, low task complexity, restricted agent perspective, or a limited capacity for interaction among artificial agents. Furthermore, many platforms lack the ability to flexibly configure the simulation, making the simulated environment a black-box from the perspective of the learning system. In this work, we propose a novel taxonomy of existing simulation platforms and discuss the highest level class of general platforms which enable the development of learning environments that are rich in visual, physical, task, and social complexity. We argue that modern game engines are uniquely suited to act as general platforms and as a case study examine the Unity engine and open source Unity ML-Agents Toolkit. We then survey the research enabled by Unity and the Unity ML-Agents Toolkit, discussing the kinds of research a flexible, interactive and easily configurable general platform can facilitate.
Forward citations
Cited by 23 Pith papers
-
Affordance-Based Manipulation Planning with Text Goals and Sim-to-Real Generalisation via Real-to-Sim Image Conversion
Affordance recognition, multi-step visual effect prediction, and multimodal text matching produce robot plans that handle occlusion; real-to-sim conversion enables hardware transfer.
-
TowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents
TowerMind is a lightweight multimodal tower-defense benchmark showing that current LLM agents score 58-62% below human experts and degrade on harder levels.
-
Playstyle and Artificial Intelligence: An Initial Blueprint Through the Lens of Video Games
This dissertation formalizes playstyle as the decision-making style of an agent, introduces a discrete-state playstyle distance that distinguishes behaviors in racing games and Atari, and proposes a blueprint for usin...
-
Bot App\'etit! Exploring how Robot Morphology Shapes Perceived Affordances via a Mise en Place Scenario in a VR Kitchen
In a VR kitchen, people delegated tasks and placed robots differently based on robot morphology, yielding three hypotheses about biomorphism, perceived sensing, and robot bulk.
-
Ghost Policies: A New Paradigm for Understanding and Learning from Failure in Deep Reinforcement Learning
A proposed AR framework, Arvolution, visualizes past failed RL policies as ghosts to support failure analysis and a dual human-agent learning loop.
-
Diver-Robot Communication Dataset for Underwater Hand Gesture Recognition
A new open dataset of 30,440 underwater frames of 892 diving gestures, with synchronized glove and acoustic logs, recorded in sea and pool by ten divers at 1, 2 and 3 meters.
-
Adaptive 3D UI Placement in Mixed Reality Using Deep Reinforcement Learning
A PPO-based reinforcement learning agent learns to position a single 3D UI in simulated mixed reality, keeping it visible, collision-free, and near a moving user in both familiar and novel indoor scenes.
-
RLS3: RL-Based Synthetic Sample Selection to Enhance Spatial Reasoning in Vision-Language Models for Indoor Autonomous Perception
An RL agent generates hard synthetic spatial-reasoning examples to fine-tune VLMs, improving performance on simulated test scenes.
-
GSAVS: Gaussian Splatting-based Autonomous Vehicle Simulator
GSAVS renders all simulator assets as 3D Gaussian splats inside Unity and uses camera extrinsics to build a road spline for physics-based training.
-
Optimizing Sensor Redundancy in Sequential Decision-Making Problems
SensorOpt formulates backup sensor selection for RL policies as a budget-constrained QUBO using a second-order return approximation, and finds near-optimal configurations with Tabu Search.
-
LiDAR-EDIT: LiDAR Data Generation by Editing the Object Layouts in Real-World Scenes
A pipeline that removes and inserts vehicles in real LiDAR scans, using spherical voxelization and generative inpainting, to create controllable synthetic data with a small domain gap.
-
Harlequin: Color-driven Generation of Synthetic Data for Referring Expression Comprehension
Pre-training referring expression comprehension models on a fully synthetic, color-varied dataset generated with GLIGEN improves their fine-tuned accuracy on real benchmarks.
-
Learning Autonomous Surgical Irrigation and Suction with the da Vinci Research Kit Using Reinforcement Learning
Two reinforcement learning agents, trained in a new fluid-capable surgical simulator and transferred to a real da Vinci Research Kit, autonomously perform irrigation and suction in a simplified bench-top setup.
-
Mapping out the Space of Human Feedback for Reinforcement Learning: A Conceptual Framework
A conceptual framework classifies human feedback to RL agents along nine dimensions and seven quality criteria, unifying human-centered, interface-centered, and model-centered design perspectives.
-
Toward Sim-to-Real Directional Semantic Grasping
A double deep Q-network trained on simulated RGB images with domain randomization transfers to a real robot for directional semantic grasping of four YCB objects.
-
Robby is Not a Robber (anymore): On the Use of Institutions for Learning Normative Behavior
A robot learns to follow store norms using norm-based rewards, and the learned abstract policy transfers to a different factory domain without retraining.
-
FootsiesGym: A Fighting Game Benchmark for Two-Player Zero-Sum Imperfect-Information Games
FootsiesGym is an open-source, vectorized fighting-game benchmark for two-player zero-sum imperfect-information RL that isolates non-transitive neutral-game dynamics while remaining tractable on standard hardware.
-
SMaRCSim: Maritime Robotics Simulation Modules
SMaRCSim is a Unity-based simulation suite for maritime robotics that integrates multi-domain vehicles, mission planning, and RL training, with code released on GitHub.
-
Reward-Augmented Reinforcement Learning for Continuous Control in Precision Autonomous Parking via Policy Optimization Methods
In a custom Unity parking simulator, a milestone-augmented reward with PPO achieves 91% success, while sparse and dense proximity rewards fail to guide learning.
-
Multi-agent Reinforcement Learning for Robotized Coral Reef Sample Collection
An RL controller for coral sample collection, trained in Unity, is transferred zero-shot to a physical BlueROV2 using real-time underwater motion capture to drive the digital twin.
-
Effective Reward Specification in Deep Reinforcement Learning
A thesis presenting four methods (ASAF, TeamReg, CoachReg, constrained RL, goal-conditioned GFlowNets) that improve reward specification for deep RL through demonstrations, policy regularization, behavior constraints,...
-
Learning to Grasp from 2.5D images: a Deep Reinforcement Learning Approach
A simulated DRL agent learns to position a suction gripper over blocks using only depth-camera images, but the claimed precision is not numerically demonstrated.
-
A Review of Cooperative Multi-Agent Deep Reinforcement Learning
A review that categorizes cooperative multi-agent deep RL into independent learners, observable critics, value factorization, consensus, and communication, with errors in the taxonomy and references.
Discussion (0). Continue with ORCID to comment.