Approximation Analysis for Feature Extraction by Deep Convolutional Neural Networks

Description

In recent decades, deep learning, which is accomplished through deep neural networks (DNNs), has achieved remarkable breakthroughs and captured state-of-the-art performance in various applications, like speech recognition, computer vision, naturallanguage processing. Among various architectures, deep convolutional neural networks(CNNs) are of special interest, which has been widely employed in numerous practical learning tasks such as classification of images, image captioning, and controlpolicylearning to play the board game Go. Despite the great success of deep learning, it is often deployed as a ``black box,'' to obscurely accomplish an input-output data fitting. The obscurity leads to the lack of interpretability and subsequent performance guarantees (e.g., stableness, accuracy,generalizability, robustness, and transferability, etc.) in deep network learning. Growing curiosity and concern about the black-box nature arise for the time being. Two of the central questions are 1) what the intermediate features or knowledge extracted by thenetwork about the underlying information of data and 2) how it makes meaningful tasks like classification, detection, to be achieved finally. To open the ''black-boxes'' of deep networks, in this proposal we shall study feature extraction in classification learning and investigate the approximation analysis of feature extractors generated by deep CNNs. Precisely, we shall explore principled measures to evaluate the goodness of feature representations of source labeled data, which will be discriminative among multiple classes and compressible within a single class. Then theoretical analysis based on approximation theory and coding theory would be investigated to reveal the correspondence of the goodness of feature extractions to the classification error. The desired results will have an impact both for the theoretical analysis community and for the real practitioners in industries. In addition, the analysis produced may provide guidance for network design and algorithms that are more efficient and accurate.