A real-time automatic lipreading system

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)22_Publication in policy or professional journal

22 Scopus Citations
View graph of relations


Related Research Unit(s)


Original languageEnglish
Journal / PublicationProceedings - IEEE International Symposium on Circuits and Systems
Publication statusPublished - 2004


Title2004 IEEE International Symposium on Cirquits and Systems - Proceedings
CityVancouver, BC
Period23 - 26 May 2004


It's well known that visual information such as lip shape and its movement can indicate what the speaker is talking about. In this paper, we present an automatic lipreading system solely using visual information for recognizing isolated English digits, from 0 to 9. A parameter set of a 14-point ASM lip model is used to describe the outer lip contour. The inner mouth information such as the teeth region and the mouth opening are also extracted. With appropriate normalization, the feature vectors containing the normalized outer lip features, inner mouth features and also their first order derivatives are obtained for training the HMM word models. Experiments have been carried out to investigate the recognition performance using our visual feature set compared with other traditional visual feature representations. An accuracy of 93% for speaker dependent recognition and 84% for speaker independent recognition is achieved using our visual feature representation. A real-time automatic lipreading system has been successfully implemented on a 1.9-GHz PC.