Approximation Analysis of Information Theoretic Learning and Ranking Type Learning Problems

Description

Theory of learning with the classical least squares loss has been well developed in mathematics, based on probability analysis, statistics, and approximation theory. Information theoretic learning is a different learning framework using descriptors from information theory to substitute the conventional statistical descriptors of variance and covariance in the least squares method for processing non-Gaussian noise. Minimum error entropy is such a principle using entropies in this framework. Ranking type learning problems aim at efficient algorithms involving sample pairs, which is different from methods for regression or classification. The purpose of this project is to develop rigorous mathematical analysis for some problems in these two topics by methods and ideas from approximation theory and wavelet analysis. We shall first establish error analysis for minimum error entropy algorithms in both empirical risk minimization and regularization settings. Analysis for information theoretic learning algorithms induced by Wasserstein metric will also be provided by scaling-based approximation schemes. We shall then make Fourier analysis for some ranking type learning algorithms. Minimizers of the generalization errors associated with the scoring function-based ranking losses and with the metric and similarity learning will be characterized in terms of the generalized Fourier transform by ideas from the study of radial basis functions. Robustness analysis of ranking type regularization schemes will be carried out. Finally some interesting approximation theory problems arising from learning theory involving correntropy, additive models, Wasserstein metric-based approximation, and positive definite kernels will be investigated.