· 2015
Effective Approaches to Attention-based Neural Machine Translation
- 8,594citations
- 2015year
Short summary
A new local attention mechanism improved English-German translation by 5.0 BLEU points over non-attentional systems, and an ensemble of attention architectures achieved a new state-of-the-art 25.9 BLEU.
AI-generated from the title and abstract; the full text is not read.
Key points
- Introduced global and local attention mechanisms for NMT.
- Local attention achieved a 5.0 BLEU point gain over non-attentional systems.
- An ensemble model with attention reached a new state-of-the-art 25.9 BLEU on WMT'15 English-German translation.
- Demonstrated effectiveness on English-German translation tasks in both directions.
AI-generated from the title and abstract; the full text is not read.
Abstract
An attentional mechanism has lately been used to improve neural machine translation (NMT) by selectively focusing on parts of the source sentence during translation. However, there has been little work exploring useful architectures for attention-based NMT. This paper examines two simple and effective classes of attentional mechanism: a global approach which always attends to all source words and a local one that only looks at a subset of source words at a time. We demonstrate the effectiveness of both approaches on the WMT translation tasks between English and German in both directions. With local attention, we achieve a significant gain of 5.0 BLEU points over non-attentional systems that already incorporate known techniques such as dropout. Our ensemble model using different attention architectures yields a new state-of-the-art result in the WMT'15 English to German translation task with 25.9 BLEU points, an improvement of 1.0 BLEU points over the existing best system backed by NMT and an n-gram reranker. 1
The authors' abstract, as published at the source., 2015 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Artificial Intelligence
Artificial IntelligenceComputer Science