Simulating a smartboard by real-time gesture detection in lecture videos

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

11 Scopus Citations
View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Article number4540197
Pages (from-to)926-935
Journal / PublicationIEEE Transactions on Multimedia
Volume10
Issue number5
Publication statusPublished - Aug 2008

Abstract

Gesture plays an important role for recognizing lecture activities in video content analysis. In this paper, we propose a real-time gesture detection algorithm by integrating cues from visual, speech and electronic slides. In contrast to the conventional "complete gesture" recognition, we emphasize detection by the prediction from "incomplete gesture". Specifically, intentional gestures are predicted by the modified hidden Markov model (HMM) which can recognize incomplete gestures before the whole gesture paths are observed. The multimodal correspondence between speech and gesture is exploited to increase the accuracy and responsiveness of gesture detection. In lecture presentation, this algorithm enables the on-the-fly editing of lecture slides by simulating appropriate camera motion to highlight the intention and flow of lecturing. We develop a real-time application, namely simulated smartboard, and demonstrate the feasibility of our prediction algorithm using hand gesture and laser pen with simple setup without involving expensive hardware. © 2008 IEEE.

Research Area(s)

  • Gesture detection, Lecture video, Real-time simulated smartboard

Citation Format(s)

Simulating a smartboard by real-time gesture detection in lecture videos. / Wang, Feng; Ngo, Chong-Wah; Pong, Ting-Chuen.

In: IEEE Transactions on Multimedia, Vol. 10, No. 5, 4540197, 08.2008, p. 926-935.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review