Learning Theory Approach to Minimum Error Entropy Criteria and Approximation Analysis of Some Learning Schemes

Least squares method is an important computational tool in various fields of science and engineering, and has been well developed in learning theory. An information theoretic alternative of this method is minimum error entropy criteria based on entropy and high order statistics. This project aims at mathematical analysis of minimum error entropy criteria by a learning theory approach and approximation analysis of some learning schemes dealing with multivariate data. We shall first show that an empirical risk minimization algorithm for minimizing empirical error entropy is consistent when the scaling parameter becomes large. Analysis of possible limits will be conducted by some ideas from wavelet analysis when the parameter becomes small. Then we shall study regularization schemes for multivariate quantile regression and ranking problems by error decomposition techniques. The group effect and sparsity of the elastic net in various settings will be analyzed. Methods from approximation theory and properties of reproducing kernel Hilbert spaces will be applied to deal with various loss functions in our analysis. Finally some learning theory problems on peptides, alleles, and binding affinity arising from bioinformatics will be considered and an approximation theory problem on positive and positive semidefinite reproducing kernels will be discussed.