Interactive Sketch-Based Normal Map Generation with Deep Neural Networks

Wanchao Su; Dong Du; Xin Yang; Shizhe Zhou; Hongbo Fu

Interactive Sketch-Based Normal Map Generation with Deep Neural Networks

Wanchao Su, Dong Du, Xin Yang, Shizhe Zhou, Hongbo Fu

School of Creative Media

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

1239 Downloads (CityUHK Scholars)

Abstract

High-quality normal maps are important intermediates for representing complex shapes. In this paper, we propose an interactive system for generating normal maps with the help of deep learning techniques. Utilizing the Generative Adversarial Network (GAN) framework, our method produces high quality normal maps with sketch inputs. In addition, we further enhance the interactivity of our system by incorporating user-specified normals at selected points. Our method generates high quality normal maps in real time. Through comprehensive experiments, we show the effectiveness and robustness of our method. A thorough user study indicates the normal maps generated by our method achieve a lower perceptual difference from the ground truth compared to the alternative methods.

Original language	English
Title of host publication	Proceedings of ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (i3D)
Place of Publication	New York
Publisher	Association for Computing Machinery
ISBN (Print)	123-4567-24-567/08/06
Publication status	Published - May 2018
Event	ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (i3D 2018) - Ubisoft Montréal, Montreal, Canada Duration: 15 May 2018 → 18 May 2018 http://i3dsymposium.github.io/2018/index.html

Conference

Conference	ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (i3D 2018)
Place	Canada
City	Montreal
Period	15/05/18 → 18/05/18
Internet address	http://i3dsymposium.github.io/2018/index.html

Bibliographical note

Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

Research Keywords

Sketch
Normal Map
Point Hints
Generative Adversarial Network
Wasserstein Distance

Publisher's Copyright Statement

COPYRIGHT TERMS OF DEPOSITED POSTPRINT FILE: © ACM 2018. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Proceedings of the ACM on Computer Graphics and Interactive Techniques

Access Output

Post-print.pdfPost-print, 2.81 MBLicence: Unspecified

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

GRF: Data-Driven 3D Interpretation of Freehand Drawings
FU, H. (Principal Investigator / Project Coordinator)
1/11/14 → 1/04/19
Project: Research

Cite this

@inproceedings{d0848f799c7644ef959d26d6c1581b07,

title = "Interactive Sketch-Based Normal Map Generation with Deep Neural Networks",

abstract = "High-quality normal maps are important intermediates for representing complex shapes. In this paper, we propose an interactive system for generating normal maps with the help of deep learning techniques. Utilizing the Generative Adversarial Network (GAN) framework, our method produces high quality normal maps with sketch inputs. In addition, we further enhance the interactivity of our system by incorporating user-specified normals at selected points. Our method generates high quality normal maps in real time. Through comprehensive experiments, we show the effectiveness and robustness of our method. A thorough user study indicates the normal maps generated by our method achieve a lower perceptual difference from the ground truth compared to the alternative methods.",

keywords = "Sketch, Normal Map, Point Hints, Generative Adversarial Network, Wasserstein Distance",

author = "Wanchao Su and Dong Du and Xin Yang and Shizhe Zhou and Hongbo Fu",

note = "Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).; ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (i3D 2018) ; Conference date: 15-05-2018 Through 18-05-2018",

year = "2018",

month = may,

language = "English",

isbn = "123-4567-24-567/08/06",

booktitle = "Proceedings of ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (i3D)",

publisher = "Association for Computing Machinery",

address = "United States",

url = "http://i3dsymposium.github.io/2018/index.html",

}

Interactive Sketch-Based Normal Map Generation with Deep Neural Networks. / Su, Wanchao; Du, Dong; Yang, Xin et al.
Proceedings of ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (i3D). New York: Association for Computing Machinery, 2018. 4.

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Interactive Sketch-Based Normal Map Generation with Deep Neural Networks

AU - Su, Wanchao

AU - Du, Dong

AU - Yang, Xin

AU - Zhou, Shizhe

AU - Fu, Hongbo

N1 - Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

PY - 2018/5

Y1 - 2018/5

N2 - High-quality normal maps are important intermediates for representing complex shapes. In this paper, we propose an interactive system for generating normal maps with the help of deep learning techniques. Utilizing the Generative Adversarial Network (GAN) framework, our method produces high quality normal maps with sketch inputs. In addition, we further enhance the interactivity of our system by incorporating user-specified normals at selected points. Our method generates high quality normal maps in real time. Through comprehensive experiments, we show the effectiveness and robustness of our method. A thorough user study indicates the normal maps generated by our method achieve a lower perceptual difference from the ground truth compared to the alternative methods.

AB - High-quality normal maps are important intermediates for representing complex shapes. In this paper, we propose an interactive system for generating normal maps with the help of deep learning techniques. Utilizing the Generative Adversarial Network (GAN) framework, our method produces high quality normal maps with sketch inputs. In addition, we further enhance the interactivity of our system by incorporating user-specified normals at selected points. Our method generates high quality normal maps in real time. Through comprehensive experiments, we show the effectiveness and robustness of our method. A thorough user study indicates the normal maps generated by our method achieve a lower perceptual difference from the ground truth compared to the alternative methods.

KW - Sketch

KW - Normal Map

KW - Point Hints

KW - Generative Adversarial Network

KW - Wasserstein Distance

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 123-4567-24-567/08/06

BT - Proceedings of ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (i3D)

PB - Association for Computing Machinery

CY - New York

T2 - ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (i3D 2018)

Y2 - 15 May 2018 through 18 May 2018

ER -

Interactive Sketch-Based Normal Map Generation with Deep Neural Networks

Abstract

Conference

Bibliographical note

Research Keywords

Publisher's Copyright Statement

Access Output

Other Access Options

Permanent Link

Fingerprint

Projects

GRF: Data-Driven 3D Interpretation of Freehand Drawings

Cite this