Accurate and Compact Convolutional Neural Networks with Trained Binarization

Research output: Conference Papers (RGC: 31A, 31B, 32, 33)32_Refereed conference paper (no ISBN/ISSN)peer-review

View graph of relations

Related Research Unit(s)

Detail(s)

Original languageEnglish
Number of pages13
Publication statusPublished - Sep 2019

Conference

Title30th British Machine Vision Conference (BMVC 2019)
PlaceUnited Kingdom
CityCardiff
Period9 - 12 September 2019

Abstract

Although convolutional neural networks (CNNs) are now widely used in various computer vision applications, its huge resource demanding on parameter storage and computation makes the deployment on mobile and embedded devices difficult. Recently, binary convolutional neural networks are explored to help alleviate this issue by quantizing both weights and activations with only 1 single bit. However, there may exist a noticeable accuracy degradation when compared with full-precision models. In this paper, we propose an improved training approach towards compact binary CNNs with higher accuracy. Trainable scaling factors for both weights and activations are introduced to increase the value range. These scaling factors will be trained jointly with other parameters via backpropagation. Besides, a specific training algorithm is developed including tight approximation for derivative of discontinuous binarization function and L2 regularization acting on weight scaling factors. With these improvements, the binary CNN achieves 92.3% accuracy on CIFAR-10 with VGG-Small network. On ImageNet, our method also obtains 46.1% top-1 accuracy with AlexNet and 54.2% with Resnet-18 surpassing previous works.

Citation Format(s)

Accurate and Compact Convolutional Neural Networks with Trained Binarization. / Xu, Zhe; Cheung, Ray C.C.

2019. Paper presented at 30th British Machine Vision Conference (BMVC 2019), Cardiff, United Kingdom.

Research output: Conference Papers (RGC: 31A, 31B, 32, 33)32_Refereed conference paper (no ISBN/ISSN)peer-review