· 2021
AST: Audio Spectrogram Transformer
- 1,006citations
- 2021year
Short summary
The Audio Spectrogram Transformer (AST) is the first purely attention-based model for audio classification, achieving new state-of-the-art results: 0.485 mAP on AudioSet, 95.6% accuracy on ESC-50, and 98.1% accuracy on Speech Commands V2.
AI-generated from the title and abstract; the full text is not read.
Key points
- Introduced the Audio Spectrogram Transformer (AST), the first convolution-free, purely attention-based model for audio classification.
- AST achieves new state-of-the-art results on multiple audio classification benchmarks.
- Achieved 0.485 mAP on AudioSet, 95.6% accuracy on ESC-50, and 98.1% accuracy on Speech Commands V2.
- Demonstrates the sufficiency of attention mechanisms for audio classification without reliance on CNNs.
AI-generated from the title and abstract; the full text is not read.
Abstract
In the past decade, convolutional neural networks (CNNs) have been widely adopted as the main building block for endto-end audio classification models, which aim to learn a direct mapping from audio spectrograms to corresponding labels.To better capture long-range global context, a recent trend is to add a self-attention mechanism on top of the CNN, forming a CNN-attention hybrid model.However, it is unclear whether the reliance on a CNN is necessary, and if neural networks purely based on attention are sufficient to obtain good performance in audio classification.In this paper, we answer the question by introducing the Audio Spectrogram Transformer (AST), the first convolution-free, purely attention-based model for audio classification.We evaluate AST on various audio classification benchmarks, where it achieves new state-of-the-art results of 0.485 mAP on AudioSet, 95.6% accuracy on ESC-50, and 98.1% accuracy on Speech Commands V2.
The authors' abstract, as published at the source., 2021 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Signal Processing
Signal ProcessingComputer Science