Cycle Encoding of a StyleGAN Encoder for Improved Reconstruction and Editability

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with host publication)peer-review

2 Scopus Citations
View graph of relations

Author(s)

  • Liujuan Cao
  • Zhenguo Yang
  • Qing Li
  • Rongrong Ji

Related Research Unit(s)

Detail(s)

Original languageEnglish
Title of host publicationMM '22
Subtitle of host publicationProceedings of the 30th ACM International Conference on Multimedia
PublisherAssociation for Computing Machinery
Pages2032-2041
ISBN (Print)978-1-4503-9203-7
Publication statusPublished - 10 Oct 2022

Publication series

NameMM 2022 - Proceedings of the 30th ACM International Conference on Multimedia

Conference

Title30th ACM International Conference on Multimedia (MM 2022)
Location
PlacePortugal
CityLisbon
Period10 - 14 October 2022

Abstract

GAN inversion aims to invert an input image into the latent space of a pre-trained GAN. Despite the recent advances in GAN inversion, there remain challenges to mitigate the tradeoff between distortion and editability, i.e. reconstructing the input image accurately and editing the inverted image with a small visual quality drop. The recently proposed pivotal tuning model makes significant progress towards reconstruction and editability, by using a two-step approach that first inverts the input image into a latent code, called pivot code, and then alters the generator so that the input image can be accurately mapped into the pivot code. Here, we show that both reconstruction and editability can be improved by a proper design of the pivot code. We present a simple yet effective method, named cycle encoding, for a high-quality pivot code. The key idea of our method is to progressively train an encoder in varying spaces according to a cycle scheme: WW+→W. This training methodology preserves the properties of both W and W+ spaces, i.e. high editability of W and low distortion of W+. To further decrease the distortion, we also propose to refine the pivot code with an optimization-based method, where a regularization term is introduced to reduce the degradation in editability. Qualitative and quantitative comparisons to several state-of-the-art methods demonstrate the superiority of our approach. © 2022 Copyright held by the owner/author(s).

Research Area(s)

  • cycle encoding, GAN, GAN inversion, image manipulation

Citation Format(s)

Cycle Encoding of a StyleGAN Encoder for Improved Reconstruction and Editability. / Mao, Xudong; Cao, Liujuan; Gnanha, Aurele Tohokantche et al.
MM '22: Proceedings of the 30th ACM International Conference on Multimedia. Association for Computing Machinery, 2022. p. 2032-2041 (MM 2022 - Proceedings of the 30th ACM International Conference on Multimedia).

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with host publication)peer-review