· 2015
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
- 18,828citations
- 2015year
Short summary
A new Parametric Rectified Linear Unit (PReLU) and a robust initialization method enabled training of extremely deep neural networks, achieving 4.94% top-5 error on ImageNet classification, surpassing human-level performance.
AI-generated from the title and abstract; the full text is not read.
Key points
- Introduced Parametric Rectified Linear Unit (PReLU) for improved neural network fitting.
- Developed a robust initialization method tailored for rectifier nonlinearities.
- Achieved 4.94% top-5 test error on ImageNet 2012 classification.
- Demonstrated surpassing human-level performance on the ImageNet benchmark.
AI-generated from the title and abstract; the full text is not read.
Abstract
Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU improves model fitting with nearly zero extra computational cost and little overfitting risk. Second, we derive a robust initialization method that particularly considers the rectifier nonlinearities. This method enables us to train extremely deep rectified models directly from scratch and to investigate deeper or wider network architectures. Based on the learnable activation and advanced initialization, we achieve 4.94% top-5 test error on the ImageNet 2012 classification dataset. This is a 26% relative improvement over the ILSVRC 2014 winner (GoogLeNet, 6.66% [33]). To our knowledge, our result is the first to surpass the reported human-level performance (5.1%, [26]) on this dataset.
The authors' abstract, as published at the source., 2015 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionComputer Science