Context-based multi-label image annotation

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)Not applicablepeer-review

15 Scopus Citations
View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Title of host publicationCIVR 2009 - Proceedings of the ACM International Conference on Image and Video Retrieval
Pages224-230
Publication statusPublished - 2009

Conference

TitleACM International Conference on Image and Video Retrieval, CIVR 2009
PlaceGreece
CitySantorini Island
Period8 - 10 July 2009

Abstract

This paper presents a novel context-based keyword propagation method for automatic image annotation. We follow the idea of keyword propagation and formulate image annotation as a multi-label learning problem, which is further resolved efficiently by linear programming. In this way, our method can exploit the context between keywords during keyword propagation. Unlike the popular relevance models that treat each keyword independently, our method can simultaneously propagate multiple keywords (i.e. labels) from the training images to the test images using their similarities. Moreover, we present a new 2D string kernel, called spatial spectrum kernel, to take into account another type of context when defining the similarity between images for keyword propagation. Each image is first denoted as a 2D sequence of visual keywords which are obtained through dividing images into blocks and then clustering these blocks, and a spatial spectrum kernel is then proposed to measure the 2D sequence similarity based on shared occurrences of s-length 1D subsequences through decomposing each 2D sequence into two parallel 1D sequences (i.e. the row-wise and column-wise ones). That is, we incorporate the context between visual keywords into the similarity between images (i.e. 2D sequences) used for keyword propagation. Experiments on two standard image databases demonstrate that the proposed method for automatic image annotation outperforms the state-of-the-art methods. Copyright 2009 ACM.

Research Area(s)

  • Automatic image annotation, Kernel methods, Keyword propagation, Multilabel learning, Visual keywords

Citation Format(s)

Context-based multi-label image annotation. / Lu, Zhiwu; Ip, Horace H. S.; He, Qizhen.

CIVR 2009 - Proceedings of the ACM International Conference on Image and Video Retrieval. 2009. p. 224-230.

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)Not applicablepeer-review