Automatic image dataset construction from click-Through logs using deep neural network

Labelled image datasets are the backbone for high-level im-age understanding tasks with wide application scenarios, and continuously drive and evaluate the progress of fea-ture designing and supervised learning models. Recently, the million scale labelled image dataset further contributes to the rebirth of deep convolutional neural network and by-pass manual designing handcraft features. However, the con-struction process of image dataset is mainly manual-based and quite labor intensive, which often take years' efforts to construct a million scale dataset with high quality. In this paper, we propose a deep learning based method to construc-t large scale image dataset in an automatic way. Specifically, word representation and image representation are learned in a deep neural network from large amount of click-Through logs, and further used to define word-word similarity and image-word similarity. These two similarities are used to automatize the two labor intensive steps in manual-based image dataset construction: query formation and noisy im-age removal. With a new proposed cross convolutional filter regularizer, we can construct a million scale image dataset in one week. Finally, two image datasets are constructed to verify the effectiveness of the method. In addition to scale, the automatically constructed dataset has compara-ble accuracy, diversity and cross-dataset generalization with manually labelled image datasets. © 2015 ACM.