TY - GEN
T1 - Speech animation using coupled hidden Markov models
AU - Xie, Lei
AU - Liu, Zhi-Qiang
N1 - Publication details (e.g. title, author(s), publication statuses and dates) are captured on an “AS IS” and “AS AVAILABLE” basis at the time of record harvesting from the data source. Suggestions for further amendments or supplementary information can be sent to [email protected].
PY - 2006
Y1 - 2006
N2 - We present a novel speech animation approach using coupled hidden Markov models (CHMMs). Different from the conventional HMMs that use a single state chain to model the audio-visual speech with tight inter-modal synchronization, we use the CHMMs to model the asynchrony, different discriminative abilities, and temporal coupling between the audio speech and the visual speech, which are important factors for animations looking natural. Based on the audio-visual CHMMs, visual animation parameters are predicted from audio through an EM-based audio to visual conversion algorithm. Experiments on the JEWEL AV database show that compared with the conventional HMMs, the CHMMs can output visual parameters that are much closer to the actual ones. Explicit modelling of audio-visual speech is promising in speech animation. © 2006 IEEE.
AB - We present a novel speech animation approach using coupled hidden Markov models (CHMMs). Different from the conventional HMMs that use a single state chain to model the audio-visual speech with tight inter-modal synchronization, we use the CHMMs to model the asynchrony, different discriminative abilities, and temporal coupling between the audio speech and the visual speech, which are important factors for animations looking natural. Based on the audio-visual CHMMs, visual animation parameters are predicted from audio through an EM-based audio to visual conversion algorithm. Experiments on the JEWEL AV database show that compared with the conventional HMMs, the CHMMs can output visual parameters that are much closer to the actual ones. Explicit modelling of audio-visual speech is promising in speech animation. © 2006 IEEE.
UR - http://www.scopus.com/inward/record.url?scp=34047240820&partnerID=8YFLogxK
UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-34047240820&origin=recordpage
U2 - 10.1109/ICPR.2006.1074
DO - 10.1109/ICPR.2006.1074
M3 - RGC 32 - Refereed conference paper (with host publication)
SN - 0769525210
SN - 9780769525211
VL - 1
T3 - Proceedings - International Conference on Pattern Recognition
SP - 1128
EP - 1131
BT - Proceedings - 18th International Conference on Pattern Recognition, ICPR 2006
PB - IEEE
T2 - 18th International Conference on Pattern Recognition, ICPR 2006
Y2 - 20 August 2006 through 24 August 2006
ER -