Integrating color and spatial features for content-based video retrieval

In this paper, we present a novel scheme for content-based video retrieval by exploring the spatio-temporal information. A shot with significant content changes can be segmented into several subshots that are of coherent content, and shot similarity measure for video retrieval can be computed from the similarity between corresponding subshots. To characterize the temporal content variations in one shot, we developed two descriptors: Dominant Color Histograms (DCH) and Spatial Structure Histograms (SSH). By fusing temporal information into color content, DCH for a "group of frames"(GoF) are trying to capture the dominant colors with long durations, which would be the colors of the focused objects or background. SSH is a set of features extracted from color-blob maps to describe spatial information for one individual frame. Experimental results on real-world sports video prove that our proposed approach achieve the best performance on the average recall (AR) and average normalized modified retrieval rank (ANMRR) for video shot retrievals.