arXiv (Cornell University)· 2012· Preprint
Improving neural networks by preventing co-adaptation of feature detectors
- 6,658citations
- 2012year
Short summary
Randomly omitting half of feature detectors during training significantly reduces overfitting in large neural networks trained on small datasets, leading to improved performance on benchmark tasks.
AI-generated from the title and abstract; the full text is not read.
Key points
- Overfitting in large neural networks trained on small datasets is reduced by randomly omitting half of feature detectors per training case.
- This 'dropout' technique prevents feature detectors from becoming overly specialized and co-dependent.
- Neurons learn to detect generally useful features, improving robustness to internal context variations.
- Dropout has been applied to many benchmark tasks, achieving new state-of-the-art results in speech and object recognition.
AI-generated from the title and abstract; the full text is not read.
Abstract
When a large feedforward neural network is trained on a small training set, it typically performs poorly on held-out test data. This "overfitting" is greatly reduced by randomly omitting half of the feature detectors on each training case. This prevents complex co-adaptations in which a feature detector is only helpful in the context of several other specific feature detectors. Instead, each neuron learns to detect a feature that is generally helpful for producing the correct answer given the combinatorially large variety of internal contexts in which it must operate. Random "dropout" gives big improvements on many benchmark tasks and sets new records for speech and object recognition.
The authors' abstract, as published at the source. arXiv (Cornell University), 2012 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Artificial Intelligence
Artificial IntelligenceComputer Science