BMIL learns belief modules jointly with policies for GAIL-style imitation learning in POMDPs, outperforming separate training and standard GAIL on continuous control tasks.
Learning to Act by Predicting the Future
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
abstract
We present an approach to sensorimotor control in immersive environments. Our approach utilizes a high-dimensional sensory stream and a lower-dimensional measurement stream. The cotemporal structure of these streams provides a rich supervisory signal, which enables training a sensorimotor control model by interacting with the environment. The model is trained using supervised learning techniques, but without extraneous supervision. It learns to act based on raw sensory input from a complex three-dimensional environment. The presented formulation enables learning without a fixed goal at training time, and pursuing dynamically changing goals at test time. We conduct extensive experiments in three-dimensional simulations based on the classical first-person game Doom. The results demonstrate that the presented approach outperforms sophisticated prior formulations, particularly on challenging tasks. The results also show that trained models successfully generalize across environments and goals. A model trained using the presented approach won the Full Deathmatch track of the Visual Doom AI Competition, which was held in previously unseen environments.
verdicts
UNVERDICTED 6representative citing papers
Develops a game-theoretic estimator for true information gain in active perception that achieves sub-linear regret and shows average gains of 7% information gain and 42% error reduction across simulated and real robot experiments.
Multi-step predictive generative models form stable belief states capturing environment layout and agent pose, yielding higher data efficiency on RL tasks than model-free agents.
Classical agents outperform learning-based ones on MINOS and Stanford 3D Indoor Spaces, with learned agents weaker at collision avoidance and memory but stronger at handling ambiguity and noise.
Empirical tests in VizDoom show multiple DQN updates per step do not improve performance after learning rate adjustment, with a 4:1 update-to-step ratio optimal before significant degradation.
Offline RL promises to extract high-utility policies from static datasets but faces fundamental challenges that current methods only partially address.
citing papers explorer
-
Learning Belief Representations for Imitation Learning in POMDPs
BMIL learns belief modules jointly with policies for GAIL-style imitation learning in POMDPs, outperforming separate training and standard GAIL on continuous control tasks.
-
An Active Perception Game for Robust Exploration
Develops a game-theoretic estimator for true information gain in active perception that achieves sub-linear regret and shows average gains of 7% information gain and 42% error reduction across simulated and real robot experiments.
-
Shaping Belief States with Generative Environment Models for RL
Multi-step predictive generative models form stable belief states capturing environment layout and agent pose, yielding higher data efficiency on RL tasks than model-free agents.
-
To Learn or Not to Learn: Analyzing the Role of Learning for Navigation in Virtual Environments
Classical agents outperform learning-based ones on MINOS and Stanford 3D Indoor Spaces, with learned agents weaker at collision avoidance and memory but stronger at handling ambiguity and noise.
-
Optimal Use of Experience in First Person Shooter Environments
Empirical tests in VizDoom show multiple DQN updates per step do not improve performance after learning rate adjustment, with a 4:1 update-to-step ratio optimal before significant degradation.
-
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
Offline RL promises to extract high-utility policies from static datasets but faces fundamental challenges that current methods only partially address.