A Hybrid Random Sets/Dynamic Bayesian Network Approach to Expressive Speech-Driven Facial Animation

Description

Facial expression is vital to the underlying intent of the delivered speech in communications. Creating expressive facial animation remains one of the main challenges in computer animation due to the variability and flexibility of facial behaviors. Current facial animation system may have difficulties in achieving a condensed and meaningful mechanism of representing expressions. To generate fine-detailed expressions, motion synthesis models require a large repository of audio-visual data for learning. The missing or imperfect data will inevitably affect the animation quality. In this project the researchers will propose a hybrid random sets/dynamic Bayesian network (RS/DBN) approach that learns a handy and robust model from a modest amount of audio-visual data and synthesizes expressive facial animation with lifelike performance. They will investigate the topological model structures, effective and efficient inference and learning algorithms, coarsening schemes and develop a speech-driven expressive talking face system. This project will find significant applications in interactive media systems, entertainment, e-commerce, and education.