Scalable Partial Near-Duplicate Search and Its Applications to Topical Video Browsing

With massive amount of videos available on web, commercial search engines use to return a long list of hundreds or even thousands of videos in response to a query topic. These videos are sorted according to relevance and then presented to users. As a consequence, users often need to painstakingly explore the search list for a gist of the search results, and are subjected to repeatedly watching similar clips which has been viewed previously. This eventually brings out the research challenges of large-scale video browsing and search exploration:How to efficiently browse hundreds or even thousands of videos at a glance?How to navigate videos and efficiently locate the events of interest under a topic?This project addresses the aforementioned challenges from three aspects. First, we consider the unique role of partial near-duplicates (PNDs) in bridging videos. PNDs are similar clips that are manipulated and inserted into videos for serving different purposes. Different from full near-duplicates, mining PNDs requires temporal localization of clips and scalability can become a big problem with huge volume of videos. Our proposed solution for this problem is based upon the search of minimal yet informative local features for scalable indexing. Rapid localization of PNDs by accounting temporal and geometric consistency checking is also addressed. Second, drawn upon the manipulation clues of PNDs, we propose the possible solution of mining iconic clips and their time stamps. The iconic clips, with proper chaining and visualization interface, are expected to provide an overview of search results in terms of topic evolution, and changes of context and perspective. Third, we consider a more granular level of browsing by building topic structures based on burst event detection and topic threading. Clues from PNDs, analysis from multi-modality processing, and knowledge from external sources will be jointly exploited for this task. In addition, visualization techniques which allow efficient navigation of topic structures for search exploration will be researched.The significance of this project is twofold. First, we address the timely issue and challenge of scalable video browsing, which is still a relatively new research topic since the proliferation of social media. Second, we investigate large-scale mining of near-duplicates, which is a hot research topic along the development of Web-2.0. Different from most existing researches which employ full near-duplicates for search diversification, we address the localization of partial near-duplicates in large-scale and the exploitation of PNDs for topical video browsing.