Content parsing of home videos by motion analysis

Abstract

Due to the increasing use of hand-held camcorders, an explosion of home video data is already underway. Home videos, by nature, are unedited, unstructured and lack of story-line. They contain unrestricted content domain which usually mixes together with irregular camera motions. These features have made conventional video processing techniques inappropriate for the analysis of home videos. In this thesis, we propose new techniques for the automatic parsing of home video content to support effective indexing and browsing. These techniques cover two major issues: video object analysis (VOA) and content parsing (CP). In VOA, a new algorithm is proposed for automatic object initialization. This algorithm is based on motion discriminant analysis formulated through 3D tensor representation and robust clustering. It is capable of rapidly selecting the best few frames in videos to start object initialization. Multiple object initialization is handled based on the formulation of Bayesian filter in data association and the effective temporal selection schemes. The former outlines how to initialize while the latter decides when to initialize. By the proposed object initialization, we improve the start-of-the-art EM object segmentation and mean shift tracking algorithms with automatic initialization. The distribution of shot length in home videos can typically last for several minutes. In CP, we propose a motion-based approach to decompose long shots into small segments called “snippets” by efficiently getting rid of jerky camera motions, fast pans and zooms in shots. Based on snippet representation, we incorporate object initialization and a MWBG (maximum weighted bipartite graph) pattern matching algorithm to parse the objects in videos, which is seamlessly integrated with VOA. In addition, to facilitate content browsing, we also propose an algorithm for the selective stabilization of video objects by motion segmentation and Kalman stabilizer.

Date of Award	15 Jul 2005
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Chong Wah NGO (Supervisor)