arXiv (Cornell University)· 2019· Preprint
On the Convergence of Adam and Beyond
- 1,618citations
- 2019year
Short summary
This paper identifies the exponential moving average in popular deep learning optimizers like Adam as a cause of convergence failure in certain settings, proposing new variants with 'long-term memory' that fix these issues and improve performance.
AI-generated from the title and abstract; the full text is not read.
TakeawaysIn the app
Key pointsIn the app
Ask the paperIn the app
The rest is in the Pofolia app
Takeaways, key points and questions to the paper; new summaries every day for your field. Free.
Sign in on the web to openField: Philosophy
PhilosophyArts and Humanities