Multimedia-based Hyperlink Construction and Modeling for Web Video Search

With the exponential growth of social media such as the video and photo sharing websites YouTube and Flickr, the volume of multimedia data being transmitted and searched on the Internet has increased tremendously. In this project, the researchers will propose innovative techniques for web-scale multimedia search by video hyperlinking, novelty ranking and multi-video summarization. Video hyperlinking is to inter-relate video segments which are near-duplicate and similar in terms of content and context. The hyperlinks form a cross-referencing network that facilitates the assessment of video importance and novelty. This feature is particularly critical for video search engines in Web 2.0 environment where near-duplicate videos are largely available but treated independently during search. Novelty ranking is a sorting strategy that gives higher rank to videos which are relevant to query and also novel to previously watched videos. This task requires the similarity measure of content novelty. Query-dependent summarization is to generate hyperlinked summaries that review the set of videos relevant to a given query. The summaries allow the interactive browsing and searching of user-preference clips by traversing hyperlinks. In face with the popularity of Web 2.0, these three tasks (hyperlinking, ranking, summarization) are timely for efficient web video search experience, are new under the context of social media network, and are challenging for requiring scalable techniques to thread redundancy, discover content novelty, and summarize multiple videos online. This project will research the techniques and algorithms in hyperlink construction, modeling and utilization for web video search and interactive browsing. Rich media information including video content, context and social network will be analyzed for these purposes. The major tasks of the project includeDeveloping video hyperlinking techniques by video content, context and social network.In particular, the techniques for scalable threading of visual near-duplicates will be investigated by exploiting the 3D space-time local interest points. A hierarchical framework which performs filtering by 3D visual vocabulary and matching by point-totrajectory projection will be studied for robust and efficient threading.Developing algorithms for multimedia-based novelty ranking by modeling hyperlink and video content.Most existing algorithms consider only query relevancy. The developed algorithm will emphasize both the relevancy and novelty aspects of search results.Developing techniques for online multi-video summarization.The summaries are planned to be query-dependent and enhanced with hyperlinks for interactive video navigation. To generate compact and easy-to-understand summaries, the researchers will jointly explore hyperlink modeling and content analysis by video event/object understanding.