EnhanceNet
Single Image Super-Resolution
Through Automated Texture Synthesis
|
|
|
input image |
highest PSNR |
our result |
|
Abstract
Single image super-resolution is the task of inferring a high-resolution image
from a single low-resolution input. Traditionally, the performance of
algorithms for this task is measured using pixel-wise reconstruction measures
such as peak signal-to-noise ratio (PSNR) which have been shown to correlate
poorly with the human perception of image quality. As a result, algorithms
minimizing these metrics tend to produce over-smoothed images that lack
high-frequency textures and do not look natural despite yielding high PSNR
values.
We propose a novel application of automated texture synthesis in combination
with a perceptual loss focusing on creating realistic textures rather than
optimizing for a pixel-accurate reproduction of ground truth images during
training. By using feed-forward fully convolutional neural networks in an
adversarial training setting, we achieve a significant boost in image quality
at high magnification ratios. Extensive experiments on a number of datasets
show the effectiveness of our approach, yielding state-of-the-art results in
both quantitative and qualitative benchmarks.
|
Downloads
Results
|
|
|
|
bicubic |
ENet-E |
ENet-PAT |
ground truth |
|
|
|
|
bicubic |
ENet-E |
ENet-PAT |
ground truth |
|
Comparison with other methods
|
|
|
|
|
|
bicubic |
Glasner |
Kim |
SCSR |
SelfEx |
SRCNN |
|
|
|
|
|
|
PSyCo |
VDSR |
DRCN |
ENet-E |
ENet-PAT |
ground truth |
|
Comparison of previous state of the art at 2x (75% pixels missing)
with our method at 4x super-resolution (94% pixels missing)
|
|
|
2x downsampled input |
2x downsampled input |
4x downsampled input |
|
|
|
|
|
|
2x VDSR |
2x DRCN |
4x with ENet-PAT |
|
References
[DRCN] J. Kim, J. Kwon Lee, and K. Mu Lee. Deeply-recursive convolutional network for image super-resolution. In CVPR, 2016.
[Glasner] D. Glasner, S. Bagon, and M. Irani. Super-resolution from a single image. In ICCV, 2009.
[Kim] K. I. Kim and Y. Kwon. Single-image super-resolution using sparse regression and natural image prior. IEEE TPAMI, 2010.
[PSyCo] E. Perez-Pellitero, J. Salvador, J. Ruiz-Hidalgo, and B. Rosenhahn. PSyCo: Manifold span reduction for super resolution. In CVPR, 2016.
[SCSR] J. Yang, J. Wright, T. Huang, and Y. Ma. Image super-resolution as sparse representation of raw image patches. In CVPR, 2008.
[SelfEx] J.-B. Huang, A. Singh, and N. Ahuja. Single image super-resolution from transformed self-exemplars. In CVPR, 2015.
[SRCNN] C. Dong, C. C. Loy, K. He, and X. Tang. Learning a deep convolutional network for image super-resolution. In ECCV, 2014.
[VDSR] J. Kim, J. Kwon Lee, and K. Mu Lee. Accurate image super-resolution using very deep convolutional networks. In CVPR, 2016.