A novel saliency prediction method based on fast radial symmetry transform and its extensions

Abstract

Computational visual attention model (saliency prediction model) predicts where humans look when they attend to images. It has applications such as surveillance system, video compression, robotic vision and game production. It has been widely studied by computer scientists, psychologists and neuroscientists in the past decades. Visual psychological research supports the idea that visual attention is affected by symmetry. In this dissertation, we propose a novel visual attention model based on the automatic scale selection, Fast Radial Symmetry Transform (FRST) and its extensions. Based on the observation that edge plays an important role in attracting visual fixations, we apply edge detection with Automatic scale selection Edge detection (AE) to predict visual saliency. AE has the ability of removing textures and background noise in images; hence it is able to locate human fixations more accurately. It is combined with the Coherent Visual Attention (CVA) model via linear regression. The new model has good performance on a large and popular eye-tracking dataset. The one-tailed paired t-test indicates that the combined model outperforms eight state-of-the-art visual attention models highly significantly. The results of the AE+CVA model and related visual psychological research indicate that visual attention is more affected by the presence of symmetry features than the edge feature. Motivated by these findings, we apply the Fast Radial Symmetry Transform (FRST) to saliency prediction. The new approach does not require a whole set of visual features (intensity, color, orientation, etc.) as in most previous works, but uses only symmetry and center bias (CB) to explain human fixations. A comparative study of FRST+CB and 11 other state-of-the-art approaches on saliency prediction over three popular eye-tracking datasets is presented. It is shown that the new model is comparable to the best performing approaches in the literature. It is also shown to have higher prediction accuracy and lower computational complexity than an existing saliency prediction method based on symmetry proposed by Kootstra et al. There are many kinds of symmetries and more general forms of symmetries that exist in nature. Thus our research is directed towards generalizing the symmetries that can be detected. The generalized FRST (GFRST) and the Scale-selection FRST (SSFRST) are proposed to further enhance the performance of FRST on saliency prediction. The GFRST extends FRST via affine transformation so that it can detect symmetry under parallel projection in a fast end effective manner. It is shown to have better prediction accuracy than FRST and outperform the state-of-the-art GBVS method on the MIT1003 dataset. On the other hand, the SSFRST includes scale-selection when detecting symmetry. It can locate the salient regions precisely in some images. For quantitative results, it is shown to rank 1st in all metrics in both the LeMeur dataset and the MIT1003 dataset among state-of-the-art multi-scale saliency prediction approaches. It is shown to rank 1st in both the linear correlation coefficient score and the similarity score and 2nd in EMD on the MIT1003 dataset among all state-of-the-art approaches. We also study the relationships between faces, symmetry and visual attention. We find that FRST is comparable to the Viola-Jones face detector on the task of predicting visual attention. Our research confirms that a strong relationship exists between faces and human attention. When systematically analyzing the effect of symmetry on drawing visual attention based on experimental results over an eye-tracking dataset, we find that while symmetry may be an important feature for visual attention, its effect on drawing attention depends on the identity of the object itself. Thus in constrast to the advocation of Kootstra et al., a more complex visual attention mechanism than symmetry is needed. Our findings are consistent with recent findings in visual psychology literature.

Date of Award	15 Jul 2015
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Shiu Yin Kelvin YUEN (Supervisor) & Hong YAN (Supervisor)