Semantic Face Compression for Metaverse : A Compact 3D Descriptor Based Approach

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

2 Scopus Citations
View graph of relations

Author(s)

Detail(s)

Original languageEnglish
Pages (from-to)8978-8982
Number of pages5
Journal / PublicationIEEE Transactions on Circuits and Systems for Video Technology
Volume34
Issue number9
Online published19 Apr 2024
Publication statusPublished - Sept 2024

Abstract

The metaverse, a 3D virtual world, requires efficient interactive avatar communication. To achieve this goal, we envision a new metaverse paradigm for virtual avatar faces and develop semantic face compression with compact 3D facial descriptors. The paradigm comprises a compression framework that transmits 3D face descriptors for semantic compression and applications based on the semantic descriptors. The fundamental principle is that the communication of virtual avatar faces primarily emphasizes the conveyance of semantic information. In light of this, the proposed scheme offers the advantages of being highly flexible, efficient, and semantically meaningful. The promise of the proposed paradigm is also demonstrated by performance comparisons with the state-of-the-art video coding standard, Versatile Video Coding. A significant improvement in terms of rate-accuracy performance has been achieved. The proposed scheme is expected to enable numerous applications especially for real-time communication in the metaverse, such as digital human communication based on machine analysis, and to form the cornerstone of interactions. © 2024 IEEE.

Research Area(s)

  • 3D descriptor, emotion recognition, face identification, intelligent machine task, Metaverse, real-time communication, semantic face compression

Bibliographic Note

Research Unit(s) information for this publication is provided by the author(s) concerned.