Proceedings of the National Academy of Sciences· 2017Q1
Overcoming catastrophic forgetting in neural networks
- 7,277citations
- Q1SCImago
- 2017year
Short summary
A new method selectively slows learning on important weights, enabling neural networks to retain expertise on previously learned tasks without forgetting.
AI-generated from the title and abstract; the full text is not read.
Key points
- Neural networks typically suffer from catastrophic forgetting when learning new tasks sequentially.
- A new method selectively slows learning on weights important for old tasks.
- This allows networks to retain expertise on tasks not encountered for a long time.
- The approach is scalable and effective, demonstrated on classification and game-learning tasks.
AI-generated from the title and abstract; the full text is not read.
Abstract
The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence. Until now neural networks have not been capable of this and it has been widely thought that catastrophic forgetting is an inevitable feature of connectionist models. We show that it is possible to overcome this limitation and train networks that can maintain expertise on tasks that they have not experienced for a long time. Our approach remembers old tasks by selectively slowing down learning on the weights important for those tasks. We demonstrate our approach is scalable and effective by solving a set of classification tasks based on a hand-written digit dataset and by learning several Atari 2600 games sequentially.
The authors' abstract, as published at the source. Proceedings of the National Academy of Sciences, 2017 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Artificial Intelligence
Artificial IntelligenceComputer Science