An RL agent learns to actively explore by being rewarded for inferring unobserved scene parts after short glimpse sequences, with sidekick policy learning enabling generalization to other active perception tasks.
Building Generalizable Agents with a Realistic and Rich 3D Environment
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
abstract
Teaching an agent to navigate in an unseen 3D environment is a challenging task, even in the event of simulated environments. To generalize to unseen environments, an agent needs to be robust to low-level variations (e.g. color, texture, object changes), and also high-level variations (e.g. layout changes of the environment). To improve overall generalization, all types of variations in the environment have to be taken under consideration via different level of data augmentation steps. To this end, we propose House3D, a rich, extensible and efficient environment that contains 45,622 human-designed 3D scenes of visually realistic houses, ranging from single-room studios to multi-storied houses, equipped with a diverse set of fully labeled 3D objects, textures and scene layouts, based on the SUNCG dataset (Song et.al.). The diversity in House3D opens the door towards scene-level augmentation, while the label-rich nature of House3D enables us to inject pixel- & task-level augmentations such as domain randomization (Toubin et. al.) and multi-task training. Using a subset of houses in House3D, we show that reinforcement learning agents trained with an enhancement of different levels of augmentations perform much better in unseen environments than our baselines with raw RGB input by over 8% in terms of navigation success rate. House3D is publicly available at http://github.com/facebookresearch/House3D.
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
CDAN framework uses diversity exploration and adversarial self-correction for continual RL in continuous control, evaluated on new CAM environment with NSD metric showing 18.35% NSD improvement over baseline.
Consensus recommendations for standardized evaluation measures, problem statements, and benchmarking scenarios in embodied navigation research.
A rationale is presented for developing an assistant in Minecraft to advance natural language understanding and dialogue learning.
citing papers explorer
-
Emergence of Exploratory Look-Around Behaviors through Active Observation Completion
An RL agent learns to actively explore by being rewarded for inferring unobserved scene parts after short glimpse sequences, with sidekick policy learning enabling generalization to other active perception tasks.
-
Continual Reinforcement Learning with Diversity Exploration and Adversarial Self-Correction
CDAN framework uses diversity exploration and adversarial self-correction for continual RL in continuous control, evaluated on new CAM environment with NSD metric showing 18.35% NSD improvement over baseline.
-
On Evaluation of Embodied Navigation Agents
Consensus recommendations for standardized evaluation measures, problem statements, and benchmarking scenarios in embodied navigation research.
-
Why Build an Assistant in Minecraft?
A rationale is presented for developing an assistant in Minecraft to advance natural language understanding and dialogue learning.