Topical video browsing by cross-media and cross-source linking

Abstract

A huge number of videos are being uploaded to the Internet every second. Existing commercial search engines are still largely using surrounding texts for video search, which map textual user queries to the textual contexts of Web videos (e.g., titles and descriptions). Results are popularly shown in a traditional list view, which is boring and is not sufficient for visualizing multiple semantic facets of complex queries. In addition, video search is often treated as an isolated function, without being connected to other media like news documents. Despite the growing popularity of videos on the Web, the search and browsing functions of the commercial search engines did not change too much over the past decade. This motivated the works presented in this thesis. We investigate three important and correlated issues to advance video search and browsing: 1) cross-media hyper-linking to smartly link videos with valuable information from other types of media; 2) hierarchical video search result navigation, which revolutionizes the traditional list view by organizing relevant videos of complex search topics using semantically hierarchical structures; and 3) user-log analysis for mining iconic videos of historically hot events, important time stamps and hot topics, where we use Celebrity search as a show case to analyze real user logs from Microsoft Bing search. For cross-media hyper-linking, we propose to explore the rapidly growing social media information. By synchronizing the heterogeneous information sources (e.g., Google news and trends), we significantly enrich video search results and present a well-organized browsing structure for users to navigate. The key techniques to reach this goal include content mining and selection from videos as well as space-time alignment of multiple knowledge media. Several interfaces such as time-line based and video-island based are implemented. Videos related to a complex event always contain multiple content facets such as torch relay" and "games" for the event "Beijing Olympics". Organizing video search result in a highly structured way is always preferred to get a good understanding of a search topic in a very short time, which is missing in current commercial engines. Therefore the second issue investigated in this thesis is to design a good hierarchical video search result navigation system. The semantic hierarchy of a popular user query can be easily obtained by searching comprehensive knowledge bases on the Web like the Wikipedia. To pick the best suitable videos for each node of the hierarchy, we define three important criteria, namely relevance, uniqueness and diversity. Both textual and visual similarities are exploited to maximize the three criteria. The last research issue studied in this thesis is to analyze the valuable user log to understand and make use of real user preferences. We take celebrity-type queries as an example, which occupies 10% traffics of Microsoft Bing video search. User log from six months of Bing search engine is used to mine historically hot events for each celebrity. The mined events are automatically assigned with tags such as time and hot degree, and also accompanied with a few iconic video examples. This useful information is then in turn utilized for improving the relevancy of new searches and the way of visualizing search results. Extensive evaluations were conducted for each of the three issues on large scale real datasets, with very promising results observed. We hope that the works presented in this thesis are helpful towards the challenging goal of devising the next-generation of Internet video search engines, which is of extremely high demands.

Date of Award	14 Feb 2014
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Chong Wah NGO (Supervisor)