Deep learning is a powerful tool for artificial intelligence and has demonstrated its power in speech recognition, computer vision, natural language processing, and many other areas of practical applications. Compared with its high efficiency in processing big data, its theoretical understanding is way behind. There is an urgent need to develop rigorous mathematical analysis for deep learning algorithms. The goal of this project is to make some contributions to the mathematical theory of deep learning from the viewpoints of wavelet analysis and approximation theory. Our first target is to develop some wavelet analysis of convolutional deep neural networks activated by ReLU (the rectified linear unit) and to estimate the approximation ability of the generated deep learning algorithm. Our idea is to construct a special convolutional deep net to reproduce some ridge splines which are continuous linear splines composed with homogeneous linear functions, and to study the approximation ability of the generated hypothesis space. This can be achieved by initial layers of convolutions to generate a rich family of homogeneous linear functions and deeper layers of activations to generate linear splines with knot sets depending on the depth of the deep net. Our second target is to investigate the approximation of radial functions by fully connected deep neural networks and convolutional deep neural networks activated by ReLU and sigmoidal functions. We expect to derive approximation orders by approximating a univariate polynomial of degree k by a shallow neural network of width k+1, and then approximate the univariate function generating the radial function by scaled shifts of the activation function in convolutional deep nets or multi-layer nets with tight widths. Methods and ideas from wavelet analysis, learning theory, and approximation by radial basis functions and from shift-invariant spaces play an essential role in our research.