iOrthoPredictor : model-guided deep prediction of teeth alignment

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

4 Scopus Citations
View graph of relations

Author(s)

  • Lingchen YANG
  • Zefeng SHI
  • Yiqian WU
  • Xiang LI
  • Kun ZHOU
  • Youyi ZHENG

Related Research Unit(s)

Detail(s)

Original languageEnglish
Article number216
Journal / PublicationACM Transactions on Graphics
Volume39
Issue number6
Online publishedNov 2020
Publication statusPublished - Dec 2020

Link(s)

Abstract

In this paper, we present iOrthoPredictor, a novel system to visually predict teeth alignment in photographs. Our system takes a frontal face image of a patient with visible malpositioned teeth along with a corresponding 3D teeth model as input, and generates a facial image with aligned teeth, simulating a real orthodontic treatment effect. The key enabler of our method is an effective disentanglement of an explicit representation of the teeth geometry from the in-mouth appearance, where the accuracy of teeth geometry transformation is ensured by the 3D teeth model while the in-mouth appearance is modeled as a latent variable. The disentanglement enables us to achieve fine-scale geometry control over the alignment while retaining the original teeth appearance attributes and lighting conditions. The whole pipeline consists of three deep neural networks: a U-Net architecture to explicitly extract the 2D teeth silhouette maps representing the teeth geometry in the input photo, a novel multilayer perceptron (MLP) based network to predict the aligned 3D teeth model, and an encoder-decoder based generative model to synthesize the in-mouth appearance conditional on the original teeth appearance and the aligned teeth geometry. Extensive experimental results and a user study demonstrate that iOrthoPredictor is effective in qualitatively predicting teeth alignment, and applicable to the orthodontic industry.

Research Area(s)

  • generative networks, image synthesis, orthodontics, teeth alignment

Citation Format(s)

iOrthoPredictor : model-guided deep prediction of teeth alignment. / YANG, Lingchen; SHI, Zefeng; WU, Yiqian; LI, Xiang; ZHOU, Kun; FU, Hongbo; ZHENG, Youyi.

In: ACM Transactions on Graphics, Vol. 39, No. 6, 216, 12.2020.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

Download Statistics

No data available