Multimedia Hyperlinking: From Similarity Ranking to Entity Linking

Link traversal has been a usual practice for web page surfing. Similar practice on videos,nevertheless, is yet to be realized as audio-visual content are relatively difficult to editthan html pages. The idea of multimedia hyperlinking is to insert links betweenfragments of videos such that shared entities (e.g., object, person and event) areconnected for efficient content navigation. Manual creation of hyperlinks, particularlyfor large and unfamiliar archive of multimedia content, is expected to be a tediousexperience. The ambition of this project is to research techniques and algorithms toautomate video hyperlinking – a timely problem in multimedia driven by applicationssuch as detail-on-demand browsing and video advertising.Creation of hyperlinks involves two major steps. Starting from a large video collection,the process digs into underlying multimedia content shared among video fragments andsubsequently establishes links among them for content-based navigation. Browsingbehaviors are considered along the process such that, by traversing the links, userinformation need can be satisfied while diverse perspectives of the information areexposed to users. Different from information searching where relevant items are rankedwith user search intent in mind, hyperlinking serves in the scenarios of detail-on-demandand exploratory browsing without user knowledge in advance. Hence, thecreated linkages are expected to be error-free as much as possible with diversification incontent coverage, versus searching where right information are strived to be ranked ashigh as possible. Achieving this goal is by no means easy due to the need of techniquesin reliable multimedia content mining and link selection, which remain largely openchallenges. Yet this is achievable due to feasibility of million-scale visual entities miningdemonstrated recently and new results from vision and database communities regardingmeaningful nearest neighbor search, which can be applied for link selection. This projectconsiders hyperlinking beyond search ranking by researching issues in mining of multimodalentities and the statistical (i.e., hubness and local intrinsic dimensionality) andcorrelative properties of entities in leading to robust hyperlinking practical for differentvideo browsing scenarios.The project targets for “linking things inside multimedia archive”, enabling videocontent more easily accessible and consumable – a problem becoming increasinglyimportant due to massive video data growth. This project will contribute by investigatingdifferent aspects influencing the utility of hyperlinking, including anchor popularity,linking uncertainty, information coverage versus diversification, which are seldomstudied and yet to be integrally explored in the literature.