Semi‐Supervised 3D Shape Recognition via Multimodal Deep Co‐training

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

View graph of relations


Related Research Unit(s)


Original languageEnglish
Pages (from-to)279-289
Journal / PublicationComputer Graphics Forum
Issue number7
Publication statusPublished - Oct 2020


3D shape recognition has been actively investigated in the field of computer graphics. With the rapid development of deep learning, various deep models have been introduced and achieved remarkable results. Most 3D shape recognition methods are supervised and learn only from the large amount of labeled shapes. However, it is expensive and time consuming to obtain such a large training set. In contrast to these methods, this paper studies a semi‐supervised learning framework to train a deep model for 3D shape recognition by using both labeled and unlabeled shapes. Inspired by the co‐training algorithm, our method iterates between model training and pseudo‐label generation phases. In the model training phase, we train two deep networks based on the point cloud and multi‐view representation simultaneously. In the pseudo‐label generation phase, we generate the pseudo‐labels of the unlabeled shapes using the joint prediction of two networks, which augments the labeled set for the next iteration. To extract more reliable consensus information from multiple representations, we propose an uncertainty‐aware consistency loss function to combine the two networks into a multimodal network. This not only encourages the two networks to give similar predictions on the unlabeled set, but also eliminates the negative influence of the large performance gap between the two networks. Experiments on the benchmark ModelNet40 demonstrate that, with only 10% labeled training data, our approach achieves competitive performance to the results reported by supervised methods.

Research Area(s)

  • CCS Concepts, • Computing methodologies → Shape analysis