Generative Adversarial Network-Based Intra Prediction for Video Coding

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

29 Scopus Citations
View graph of relations

Author(s)

Detail(s)

Original languageEnglish
Article number8744274
Pages (from-to)45-58
Journal / PublicationIEEE Transactions on Multimedia
Volume22
Issue number1
Online published24 Jun 2019
Publication statusPublished - Jan 2020

Abstract

In this paper, a novel intra prediction method is proposed to improve the video coding performance, in which the generative adversarial network (GAN) is adopted to intelligently remove the spatial redundancy with the inference process. The proposed GAN-based method improves the prediction by exploiting more information and generating more flexible prediction patterns. In particular, the intra prediction is modeled as an inpainting task, which is accomplished with the GAN model to fill in the missing part by conditioning on the available reconstructed pixels. As such, the learned GAN model is incorporated into both video encoder and decoder, and the rate-distortion optimization is performed for the competition between GAN-based intra prediction and traditional angular-based intra prediction to achieve better coding performance. The proposed scheme is implemented into the high-efficiency video coding test model (HM 16.17) and the versatile video coding test model (VTM 1.1). The experimental results show that the proposed algorithm can achieve 6.6%, 7.5%, and 7.5% under HM 16.17 and 6.75%, 7.63%, and 7.65% under VTM 1.1 bit rate savings on average for luma and chroma components in the intra coding scenario.

Research Area(s)

  • Generative adversarial network, high efficiency video coding, inpainting, intra prediction, versatile video coding

Citation Format(s)

Generative Adversarial Network-Based Intra Prediction for Video Coding. / Zhu, Linwei; Kwong, Sam; Zhang, Yun et al.

In: IEEE Transactions on Multimedia, Vol. 22, No. 1, 8744274, 01.2020, p. 45-58.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review