Semantic Inpainting with Multi-dimensional Adversarial Network and Wasserstein Distance

Haodi Wang; Libin Jiao; Rongfang Bie; Hao Wu

doi:10.1007/978-3-030-60636-7_7

Semantic Inpainting with Multi-dimensional Adversarial Network and Wasserstein Distance

Haodi Wang, Libin Jiao, Rongfang Bie, Hao Wu^*

^*Corresponding author for this work

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

Abstract

Inpainting represents a procedure which can restore the lost parts of an image based upon the residual information. We present an inpainting network that consists of an Encoder-Decoder pipeline and a multi-dimensional adversarial network. The Encoder-Decoder pipeline extracts features from the input image with missing area and learns these features. Through unsupervised learning, the pipeline can predict and fill the missing region with the most reasonable content. Meanwhile the multi-dimensional adversarial network identifies the difference between the ground truth and the generated images both in detail and in general. Compared with the traditional training procedure, our model combines with Wasserstein Distance that enhances the stability of network training. The network is training specifically on street view images and not only performs a satisfying outcome, but also shows competitiveness when comparing with existing methods. © 2020, Springer Nature Switzerland AG.

Original language	English
Title of host publication	Pattern Recognition and Computer Vision - Third Chinese Conference, PRCV 2020, Proceedings, Part III
Editors	Yuxin Peng, Qingshan Liu, Huchuan Lu, Zhenan Sun, Chenglin Liu, Xilin Chen, Hongbin Zha, Jian Yang
Publisher	Springer, Cham
Pages	78-88
Number of pages	11
ISBN (Electronic)	9783030606367
ISBN (Print)	9783030606350
DOIs	https://doi.org/10.1007/978-3-030-60636-7_7
Publication status	Published - 2020
Externally published	Yes
Event	3rd Chinese Conference on Pattern Recognition and Computer Vision (PRCV 2020) - Nanjing, China Duration: 16 Oct 2020 → 18 Oct 2020

Publication series

Name	Lecture Notes in Computer Science
Volume	12307
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	3rd Chinese Conference on Pattern Recognition and Computer Vision (PRCV 2020)
Place	China
City	Nanjing
Period	16/10/20 → 18/10/20

Funding

This research is sponsored by National Natural Science Foundation of China (No. 61571049, 61371185, 61401029, 11401028, 61472044, 61472403, 61601033) and the Fundamental Research Funds for the Central Universities (No. 2014KJJCB32, 2013NT57) and by SRF for ROCS, SEM and China Postdoctoral Science Foundation Funded Project (No. 2016M590337).

Research Keywords

Inpainting
Multi-dimensional discriminator
Wasserstein distance

Access Output

10.1007/978-3-030-60636-7_7

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Wang, H., Jiao, L., Bie, R., & Wu, H. (2020). Semantic Inpainting with Multi-dimensional Adversarial Network and Wasserstein Distance. In Y. Peng, Q. Liu, H. Lu, Z. Sun, C. Liu, X. Chen, H. Zha, & J. Yang (Eds.), Pattern Recognition and Computer Vision - Third Chinese Conference, PRCV 2020, Proceedings, Part III (pp. 78-88). (Lecture Notes in Computer Science; Vol. 12307). Springer, Cham. https://doi.org/10.1007/978-3-030-60636-7_7

Wang, Haodi ; Jiao, Libin ; Bie, Rongfang et al. / Semantic Inpainting with Multi-dimensional Adversarial Network and Wasserstein Distance. Pattern Recognition and Computer Vision - Third Chinese Conference, PRCV 2020, Proceedings, Part III. editor / Yuxin Peng ; Qingshan Liu ; Huchuan Lu ; Zhenan Sun ; Chenglin Liu ; Xilin Chen ; Hongbin Zha ; Jian Yang. Springer, Cham, 2020. pp. 78-88 (Lecture Notes in Computer Science).

@inproceedings{a49c65f3ef3e4bd89000a5f0ecce60d2,

title = "Semantic Inpainting with Multi-dimensional Adversarial Network and Wasserstein Distance",

abstract = "Inpainting represents a procedure which can restore the lost parts of an image based upon the residual information. We present an inpainting network that consists of an Encoder-Decoder pipeline and a multi-dimensional adversarial network. The Encoder-Decoder pipeline extracts features from the input image with missing area and learns these features. Through unsupervised learning, the pipeline can predict and fill the missing region with the most reasonable content. Meanwhile the multi-dimensional adversarial network identifies the difference between the ground truth and the generated images both in detail and in general. Compared with the traditional training procedure, our model combines with Wasserstein Distance that enhances the stability of network training. The network is training specifically on street view images and not only performs a satisfying outcome, but also shows competitiveness when comparing with existing methods. {\textcopyright} 2020, Springer Nature Switzerland AG.",

keywords = "Inpainting, Multi-dimensional discriminator, Wasserstein distance",

author = "Haodi Wang and Libin Jiao and Rongfang Bie and Hao Wu",

year = "2020",

doi = "10.1007/978-3-030-60636-7_7",

language = "English",

isbn = "9783030606350",

series = "Lecture Notes in Computer Science",

publisher = "Springer, Cham",

pages = "78--88",

editor = "Yuxin Peng and Qingshan Liu and Huchuan Lu and Zhenan Sun and Chenglin Liu and Xilin Chen and Hongbin Zha and Jian Yang",

booktitle = "Pattern Recognition and Computer Vision - Third Chinese Conference, PRCV 2020, Proceedings, Part III",

note = "3rd Chinese Conference on Pattern Recognition and Computer Vision (PRCV 2020) ; Conference date: 16-10-2020 Through 18-10-2020",

}

Wang, H, Jiao, L, Bie, R & Wu, H 2020, Semantic Inpainting with Multi-dimensional Adversarial Network and Wasserstein Distance. in Y Peng, Q Liu, H Lu, Z Sun, C Liu, X Chen, H Zha & J Yang (eds), Pattern Recognition and Computer Vision - Third Chinese Conference, PRCV 2020, Proceedings, Part III. Lecture Notes in Computer Science, vol. 12307, Springer, Cham, pp. 78-88, 3rd Chinese Conference on Pattern Recognition and Computer Vision (PRCV 2020), Nanjing, China, 16/10/20. https://doi.org/10.1007/978-3-030-60636-7_7

Semantic Inpainting with Multi-dimensional Adversarial Network and Wasserstein Distance. / Wang, Haodi; Jiao, Libin; Bie, Rongfang et al.
Pattern Recognition and Computer Vision - Third Chinese Conference, PRCV 2020, Proceedings, Part III. ed. / Yuxin Peng; Qingshan Liu; Huchuan Lu; Zhenan Sun; Chenglin Liu; Xilin Chen; Hongbin Zha; Jian Yang. Springer, Cham, 2020. p. 78-88 (Lecture Notes in Computer Science; Vol. 12307).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Semantic Inpainting with Multi-dimensional Adversarial Network and Wasserstein Distance

AU - Wang, Haodi

AU - Jiao, Libin

AU - Bie, Rongfang

AU - Wu, Hao

PY - 2020

Y1 - 2020

N2 - Inpainting represents a procedure which can restore the lost parts of an image based upon the residual information. We present an inpainting network that consists of an Encoder-Decoder pipeline and a multi-dimensional adversarial network. The Encoder-Decoder pipeline extracts features from the input image with missing area and learns these features. Through unsupervised learning, the pipeline can predict and fill the missing region with the most reasonable content. Meanwhile the multi-dimensional adversarial network identifies the difference between the ground truth and the generated images both in detail and in general. Compared with the traditional training procedure, our model combines with Wasserstein Distance that enhances the stability of network training. The network is training specifically on street view images and not only performs a satisfying outcome, but also shows competitiveness when comparing with existing methods. © 2020, Springer Nature Switzerland AG.

AB - Inpainting represents a procedure which can restore the lost parts of an image based upon the residual information. We present an inpainting network that consists of an Encoder-Decoder pipeline and a multi-dimensional adversarial network. The Encoder-Decoder pipeline extracts features from the input image with missing area and learns these features. Through unsupervised learning, the pipeline can predict and fill the missing region with the most reasonable content. Meanwhile the multi-dimensional adversarial network identifies the difference between the ground truth and the generated images both in detail and in general. Compared with the traditional training procedure, our model combines with Wasserstein Distance that enhances the stability of network training. The network is training specifically on street view images and not only performs a satisfying outcome, but also shows competitiveness when comparing with existing methods. © 2020, Springer Nature Switzerland AG.

KW - Inpainting

KW - Multi-dimensional discriminator

KW - Wasserstein distance

UR - http://www.scopus.com/inward/record.url?scp=85094149090&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85094149090&origin=recordpage

U2 - 10.1007/978-3-030-60636-7_7

DO - 10.1007/978-3-030-60636-7_7

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9783030606350

T3 - Lecture Notes in Computer Science

SP - 78

EP - 88

BT - Pattern Recognition and Computer Vision - Third Chinese Conference, PRCV 2020, Proceedings, Part III

A2 - Peng, Yuxin

A2 - Liu, Qingshan

A2 - Lu, Huchuan

A2 - Sun, Zhenan

A2 - Liu, Chenglin

A2 - Chen, Xilin

A2 - Zha, Hongbin

A2 - Yang, Jian

PB - Springer, Cham

T2 - 3rd Chinese Conference on Pattern Recognition and Computer Vision (PRCV 2020)

Y2 - 16 October 2020 through 18 October 2020

ER -

Wang H, Jiao L, Bie R, Wu H. Semantic Inpainting with Multi-dimensional Adversarial Network and Wasserstein Distance. In Peng Y, Liu Q, Lu H, Sun Z, Liu C, Chen X, Zha H, Yang J, editors, Pattern Recognition and Computer Vision - Third Chinese Conference, PRCV 2020, Proceedings, Part III. Springer, Cham. 2020. p. 78-88. (Lecture Notes in Computer Science). Epub 2020 Oct 13. doi: 10.1007/978-3-030-60636-7_7

Semantic Inpainting with Multi-dimensional Adversarial Network and Wasserstein Distance

Abstract

Publication series

Conference

Funding

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this