Rich Image Description Based on Regions

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 32 - Refereed conference paper (with host publication)peer-review

5 Scopus Citations
View graph of relations

Author(s)

  • Xinhang SONG
  • Xiong LV
  • Shuqiang JIANG
  • Qixiang YE
  • Jianbin JIAO

Related Research Unit(s)

Detail(s)

Original languageEnglish
Title of host publicationProceedings of the 23rd Annual ACM Conference on Multimedia
Pages1315-1318
Publication statusPublished - 26 Oct 2015

Conference

Title The 23rd Annual ACM Conference on Multimedia
PlaceAustralia
CityBrisbane
Period26 - 30 October 2015

Abstract

Automatically describing the content of an image is a fundamental problem in artificial intelligence that connects computer vision and natural language processing. In contrast to the previous image description methods that focus on describing the whole image, this paper presents a method of generating rich image descriptions from image regions. First, we detect regions with R-CNN (regions with convolutional neural network features) framework. We then utilize the RNN (recurrent neural networks) to generate sentences for image regions. Finally, we propose an optimization method to select one suitable region. The proposed model generates several sentence description of regions in an image, which has sufficient representative power of the whole image and contains more detailed information. Comparing to general image level description, generating more specific and accurate sentences on the different regions can satisfy more personal requirements for different people. Experimental evaluations validate the effectiveness of the proposed method.

Research Area(s)

  • Image Description, Object Detection, Region Optimization, Convolutional Neural Networks, Recurrent Neural Networks

Citation Format(s)

Rich Image Description Based on Regions. / ZHANG, Xiaodan; SONG, Xinhang; LV, Xiong et al.
Proceedings of the 23rd Annual ACM Conference on Multimedia. 2015. p. 1315-1318.

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 32 - Refereed conference paper (with host publication)peer-review