Approach for video retrieval by video clip

Video clip retrieval plays a critical role in the content-based video retrieval. Two major concerns in this issue are: (1) automatic segmentation and retrieval of similar video clips from video database; (2) similarity ranking of similar video clips. Motivated by the maximal matching and optimal matching in graph theory, a novel approach is proposed for video clip retrieval based on matching theory. To tackle the clip segmentation and retrieval, the retrieval process is divided into two phases: shot-based retrieval and clip-based retrieval. In shot-based retrieval, a shot is temporally partitioned into several sub-shots based on motion content. The similarity among shots is measured according to the color content of sub-shots. In clip-based retrieval, candidates of similar video clips are selected by modeling the continuity of similar shots. Maximal matching based on Hungarian algorithm is then adopted to obtain the final similar video clips. To rank the similarity of the selected video clips, four different factors: visual similarity, granularity, interference and temporal order of shots are taken into consideration. These factors are modeled by optimal matching based on Kuhn-Munkres algorithm and dynamic programming. Experimental results indicate that the proposed approach is effective and efficient in retrieving and ranking similar video clips.