· 2016
Deep Reinforcement Learning with Double Q-Learning
- 6,087citations
- 2016year
Short summary
A novel adaptation of Double Q-learning significantly reduces action value overestimations in deep reinforcement learning, improving performance on Atari games.
AI-generated from the title and abstract; the full text is not read.
Key points
- Deep Q-learning (DQN) suffers from common and performance-harming overestimations of action values.
- The Double Q-learning concept is successfully generalized to deep neural networks.
- A proposed adaptation to DQN reduces overestimations and improves performance on Atari games.
AI-generated from the title and abstract; the full text is not read.
Abstract
The popular Q-learning algorithm is known to overestimate action values under certain conditions. It was not previously known whether, in practice, such overestimations are common, whether they harm performance, and whether they can generally be prevented. In this paper, we answer all these questions affirmatively. In particular, we first show that the recent DQN algorithm, which combines Q-learning with a deep neural network, suffers from substantial overestimations in some games in the Atari 2600 domain. We then show that the idea behind the Double Q-learning algorithm, which was introduced in a tabular setting, can be generalized to work with large-scale function approximation. We propose a specific adaptation to the DQN algorithm and show that the resulting algorithm not only reduces the observed overestimations, as hypothesized, but that this also leads to much better performance on several games.
The authors' abstract, as published at the source., 2016 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Artificial Intelligence
Artificial IntelligenceComputer Science