Fine-grained Attention and Feature-sharing Generative Adversarial Networks for Single Image Super-Resolution

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

View graph of relations

Author(s)

  • Yitong Yan
  • Chuangchuang Liu
  • Changyou Chen
  • Xianfang Sun
  • Longcun Jin
  • Xinyi Peng

Detail(s)

Original languageEnglish
Journal / PublicationIEEE Transactions on Multimedia
Online published12 Mar 2021
Publication statusOnline published - 12 Mar 2021

Abstract

Traditional super-resolution (SR) methods by minimize the mean square error usually produce images with oversmoothed and blurry edges, due to the lack of high-frequency details. In this paper, we propose two novel techniques within the generative adversarial network framework to encourage generation of photo-realistic images for image super-resolution. Firstly, instead of producing a single score to discriminate real and fake images, we propose a variant, called Fine-grained Attention Generative Adversarial Network (FASRGAN), to discriminate each pixel of real and fake images. FASRGAN adopts a UNetlike network as the discriminator with two outputs: an image score and an image score map. The score map has the same spatial size as the HR/SR images, serving as the fine-grained attention to represent the degree of reconstruction difficulty for each pixel. Secondly, instead of using different networks for the generator and the discriminator, we introduce a feature-sharing variant (denoted as Fs-SRGAN) for both the generator and the discriminator. The sharing mechanism can maintain model express power while making the model more compact, and thus can improve the ability of producing high-quality images. Quantitative and visual comparisons with state-of-the-art methods on benchmark datasets demonstrate the superiority of our methods. We further apply our super-resolution images for object recognition, which further demonstrates the effectiveness of our proposed method. The code is available at https://github.com/Rainyfish/FASRGAN-and-Fs-SRGAN.

Research Area(s)

  • image super-resolution, feature-sharing, Fine-grained attention, generative adversarial network