Incorporating Side Information by Adaptive Convolution
Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Title of host publication | 31st Conference on Neural Information Processing Systems (NIPS 2017) |
Editors | I. Guyon, S. Bengio, U.V. Luxburg |
Pages | 3868-3878 |
Publication status | Published - Dec 2017 |
Publication series
Name | Advances in Neural Information Processing Systems |
---|---|
ISSN (Print) | 1049-5258 |
Conference
Title | 31st Annual Conference on Neural Information Processing Systems (NIPS 2017) |
---|---|
Location | Long Beach Convention Center |
Place | United States |
City | Long Beach |
Period | 4 - 9 December 2017 |
Link(s)
Abstract
Computer vision tasks often have side information available that is helpful to solve the task. For example, for crowd counting, the camera perspective (e.g., camera angle and height) gives a clue about the appearance and scale of people in the scene. While side information has been shown to be useful for counting systems using traditional hand-crafted features, it has not been fully utilized in counting systems based on deep learning. In order to incorporate the available side information, we propose an adaptive convolutional neural network (ACNN), where the convolution filter weights adapt to the current scene context via the side information. In particular, we model the filter weights as a low-dimensional manifold within the high-dimensional space of filter weights. The filter weights are generated using a learned "filter manifold" sub-network, whose input is the side information. With the help of side information and adaptive weights, the ACNN can disentangle the variations related to the side information, and extract discriminative features related to the current context (e.g. camera perspective, noise level, blur kernel parameters). We demonstrate the effectiveness of ACNN incorporating side information on 3 tasks: crowd counting, corrupted digit recognition, and image deblurring. Our experiments show that ACNN improves the performance compared to a plain CNN with a similar number of parameters. Since existing crowd counting datasets do not contain ground-truth side information, we collect a new dataset with the ground-truth camera angle and height as the side information.
Citation Format(s)
Incorporating Side Information by Adaptive Convolution. / Kang, Di; Dhar, Debarun; Chan, Antoni B.
31st Conference on Neural Information Processing Systems (NIPS 2017). ed. / I. Guyon; S. Bengio; U.V. Luxburg. 2017. p. 3868-3878 (Advances in Neural Information Processing Systems).
31st Conference on Neural Information Processing Systems (NIPS 2017). ed. / I. Guyon; S. Bengio; U.V. Luxburg. 2017. p. 3868-3878 (Advances in Neural Information Processing Systems).
Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review