Modeling local interest points for semantic detection and video search at TRECVID 2006

Local interest points (LIPs) and their features have been shown to obtain surprisingly good results in object detection and recognition. Its effectiveness and scalability, however, have not been seriously addressed in large-scale multimedia database, for instance TRECVID benchmark. The goal of our works is to investigate the role and performance of LIPs, when coupling with multi-modality features, for high-level feature extraction and automatic video search. In high-level feature extraction, we explore LIPs with both local description and spatial distribution for characterizing and sketching semantic concepts respectively. Two visual dictionaries, based upon universal visual keywords and concept-based visual keywords, are generated for experiments. The 39 concepts are learnt by SVM in vector space model with the support of both dictionaries. In addition, the distribution of LIPs is also exploited for detection with the multi-resolution and embedded Earth Mover's Distance settings. We submit six runs by incorporating the two properties of LIPs with other modalities including grid-based color moment and wavelet texture.