pith. sign in

arxiv: 1511.06581 · v3 · pith:3ZHSQ2I6new · submitted 2015-11-20 · 💻 cs.LG

Dueling Network Architectures for Deep Reinforcement Learning

Ziyu Wang , Tom Schaul , Matteo Hessel , Hado van Hasselt , Marc Lanctot , Nando de Freitas This is my paper
classification 💻 cs.LG
keywords learningreinforcementarchitectureduelingmanynetworkactionsarchitectures
0
0 comments X
read the original abstract

In recent years there have been many successes of using deep representations in reinforcement learning. Still, many of these applications use conventional architectures, such as convolutional networks, LSTMs, or auto-encoders. In this paper, we present a new neural network architecture for model-free reinforcement learning. Our dueling network represents two separate estimators: one for the state value function and one for the state-dependent action advantage function. The main benefit of this factoring is to generalize learning across actions without imposing any change to the underlying reinforcement learning algorithm. Our results show that this architecture leads to better policy evaluation in the presence of many similar-valued actions. Moreover, the dueling architecture enables our RL agent to outperform the state-of-the-art on the Atari 2600 domain.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Learning the Arrow of Time

    cs.LG 2019-07 unverdicted novelty 7.0

    Introduces a learned arrow of time in MDPs that aligns with the Jordan-Kinderlehrer-Otto notion for stochastic processes and enables practical RL utilities like reachability and side-effect detection.

  2. UAV Access Point Placement for Connectivity to a User with Unknown Location Using Deep RL

    eess.SP 2019-07 unverdicted novelty 6.0

    Deep RL positions UAV for target SINR to unknown user using SINR feedback and 3D map, achieving 90% success in ray-tracing simulations.

  3. Continual Reinforcement Learning with Diversity Exploration and Adversarial Self-Correction

    cs.LG 2019-06 unverdicted novelty 6.0

    CDAN framework uses diversity exploration and adversarial self-correction for continual RL in continuous control, evaluated on new CAM environment with NSD metric showing 18.35% NSD improvement over baseline.

  4. Growing Action Spaces

    cs.LG 2019-06 unverdicted novelty 5.0

    A curriculum of growing action spaces combined with simultaneous off-policy value estimation accelerates learning in large multi-agent action spaces.

  5. In Hindsight: A Smooth Reward for Steady Exploration

    cs.LG 2019-06 unverdicted novelty 4.0

    Adding a hindsight factor that integrates historic temporal differences into the Q-learning loss reduces overestimation and yields higher average scores than DQN, DDQN and dueling networks on ATARI games after 10 mill...

  6. Deep Reinforcement Learning for Personalized Search Story Recommendation

    cs.LG 2019-07 unverdicted novelty 3.0

    A deep RL architecture using imitation learning and reinforcement learning is proposed to model immediate and future values of search story recommendations in a Markov decision process framework.

  7. Deep Reinforcement Learning for Clinical Decision Support: A Brief Survey

    cs.LG 2019-07 unverdicted novelty 2.0

    This survey compiles deep reinforcement learning algorithms for clinical decision support, reviews case studies, and offers guidance on algorithm selection for medical applications.