A real-time automatic lipreading system

Research output: Journal Publications and ReviewsRGC 22 - Publication in policy or professional journal

23 Scopus Citations
View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Journal / PublicationProceedings - IEEE International Symposium on Circuits and Systems
Volume2
Publication statusPublished - 2004

Conference

Title2004 IEEE International Symposium on Cirquits and Systems - Proceedings
PlaceCanada
CityVancouver, BC
Period23 - 26 May 2004

Abstract

It's well known that visual information such as lip shape and its movement can indicate what the speaker is talking about. In this paper, we present an automatic lipreading system solely using visual information for recognizing isolated English digits, from 0 to 9. A parameter set of a 14-point ASM lip model is used to describe the outer lip contour. The inner mouth information such as the teeth region and the mouth opening are also extracted. With appropriate normalization, the feature vectors containing the normalized outer lip features, inner mouth features and also their first order derivatives are obtained for training the HMM word models. Experiments have been carried out to investigate the recognition performance using our visual feature set compared with other traditional visual feature representations. An accuracy of 93% for speaker dependent recognition and 84% for speaker independent recognition is achieved using our visual feature representation. A real-time automatic lipreading system has been successfully implemented on a 1.9-GHz PC.