arXiv (Cornell University)· 2014· Preprint
SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
- 932citations
- 2014year
Short summary
SAGA is a new optimization method that achieves faster linear convergence rates than previous incremental gradient algorithms like SAG and SVRG, and directly supports non-strongly convex composite objectives.
AI-generated from the title and abstract; the full text is not read.
Key points
- Introduces SAGA, a new incremental gradient optimization method.
- Achieves faster linear convergence rates compared to SAG and SVRG.
- Supports composite objectives with proximal operators.
- Directly handles non-strongly convex problems and adapts to strong convexity.
AI-generated from the title and abstract; the full text is not read.
Abstract
In this work we introduce a new optimisation method called SAGA in the spirit of SAG, SDCA, MISO and SVRG, a set of recently proposed incremental gradient algorithms with fast linear convergence rates. SAGA improves on the theory behind SAG and SVRG, with better theoretical convergence rates, and has support for composite objectives where a proximal operator is used on the regulariser. Unlike SDCA, SAGA supports non-strongly convex problems directly, and is adaptive to any inherent strong convexity of the problem. We give experimental results showing the effectiveness of our method.
The authors' abstract, as published at the source. arXiv (Cornell University), 2014 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Computational Mechanics
Computational MechanicsEngineering