· 2018
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
- 12,611citations
- 2018year
Short summary
Deep features from neural networks, trained on ImageNet, significantly outperform traditional metrics like PSNR and SSIM in judging perceptual image similarity, a finding that holds across various network architectures and training methods.
AI-generated from the title and abstract; the full text is not read.
Key points
- Deep features from neural networks are superior to traditional metrics (PSNR, SSIM) for perceptual image similarity.
- VGG network features trained on ImageNet classification tasks show strong performance.
- The effectiveness of deep features extends across different network architectures and levels of supervision (supervised, self-supervised, unsupervised).
- Perceptual similarity appears to be an emergent property of deep visual representations.
AI-generated from the title and abstract; the full text is not read.
Abstract
While it is nearly effortless for humans to quickly assess the perceptual similarity between two images, the underlying processes are thought to be quite complex. Despite this, the most widely used perceptual metrics today, such as PSNR and SSIM, are simple, shallow functions, and fail to account for many nuances of human perception. Recently, the deep learning community has found that features of the VGG network trained on ImageNet classification has been remarkably useful as a training loss for image synthesis. But how perceptual are these so-called "perceptual losses"? What elements are critical for their success? To answer these questions, we introduce a new dataset of human perceptual similarity judgments. We systematically evaluate deep features across different architectures and tasks and compare them with classic metrics. We find that deep features outperform all previous metrics by large margins on our dataset. More surprisingly, this result is not restricted to ImageNet-trained VGG features, but holds across different deep architectures and levels of supervision (supervised, self-supervised, or even unsupervised). Our results suggest that perceptual similarity is an emergent property shared across deep visual representations.
The authors' abstract, as published at the source., 2018 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionComputer Science