IDEA-Net : Dynamic 3D Point Cloud Interpolation via Deep Embedding Alignment

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)peer-review

View graph of relations

Detail(s)

Original languageEnglish
Title of host publicationProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Subtitle of host publicationCVPR 2022
Pages6338-6347
Number of pages10
Publication statusPublished - Jun 2022

Conference

Title2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022)
LocationHybrid
PlaceUnited States
CityNew Orleans
Period19 - 24 June 2022

Abstract

This paper investigates the problem of temporally interpolating dynamic 3D point clouds with large non-rigid deformation. We formulate the problem as estimation of point-wise trajectories (i.e., smooth curves) and further reason that temporal irregularity and under-sampling are two major challenges. To tackle the challenges, we propose IDEA-Net, an end-to-end deep learning framework, which disentangles the problem under the assistance of the explicitly learned temporal consistency. Specifically, we propose a temporal consistency learning module to align two consecutive point cloud frames point-wisely, based on which we can employ linear interpolation to obtain coarse trajectories/in-between frames. To compensate the high-order nonlinear components of trajectories, we apply aligned feature embeddings that encode local geometry properties to regress point-wise increments, which are combined with the coarse estimations. We demonstrate the effectiveness of our method on various point cloud sequences and observe large improvement over state-of-the-art methods both quantitatively and visually. Our framework can bring benefits to 3D motion data acquisition. The source code is publicly available at https://github.com/ZENGYIMING-EAMON/IDEA-Net.git.

Citation Format(s)

IDEA-Net : Dynamic 3D Point Cloud Interpolation via Deep Embedding Alignment. / Zeng, Yiming; Qian, Yue; Zhang, Qijian; Hou, Junhui; Yuan, Yixuan; He, Ying.

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition: CVPR 2022. 2022. p. 6338-6347.

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)peer-review