Rich Image Description Based on Regions
Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Title of host publication | Proceedings of the 23rd Annual ACM Conference on Multimedia |
Pages | 1315-1318 |
Publication status | Published - 26 Oct 2015 |
Conference
Title | The 23rd Annual ACM Conference on Multimedia |
---|---|
Place | Australia |
City | Brisbane |
Period | 26 - 30 October 2015 |
Link(s)
Abstract
Automatically describing the content of an image is a fundamental problem in artificial intelligence that connects computer vision and natural language processing. In contrast to the previous image description methods that focus on describing the whole image, this paper presents a method of generating rich image descriptions from image regions. First, we detect regions with R-CNN (regions with convolutional neural network features) framework. We then utilize the RNN (recurrent neural networks) to generate sentences for image regions. Finally, we propose an optimization method to select one suitable region. The proposed model generates several sentence description of regions in an image, which has sufficient representative power of the whole image and contains more detailed information. Comparing to general image level description, generating more specific and accurate sentences on the different regions can satisfy more personal requirements for different people. Experimental evaluations validate the effectiveness of the proposed method.
Research Area(s)
- Image Description, Object Detection, Region Optimization, Convolutional Neural Networks, Recurrent Neural Networks
Citation Format(s)
Rich Image Description Based on Regions. / ZHANG, Xiaodan; SONG, Xinhang; LV, Xiong et al.
Proceedings of the 23rd Annual ACM Conference on Multimedia. 2015. p. 1315-1318.
Proceedings of the 23rd Annual ACM Conference on Multimedia. 2015. p. 1315-1318.
Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review