Approximation with Reproducing Kernel Hilbert Spaces and Error Bounds for Some Learning Schemes

Methods from approximation theory and ideas from wavelets are very useful in learning theory. Analyzing and designing learning algorithms raises new theoretical problems for approximation theory and reproducing kernel Hilbert spaces (RKHS). In this project we shall first apply techniques and ideas from approximation theory and wavelet analysis to derive confidence-based error bounds for some learning schemes. Learning rates will be presented for least square regularized regression in RKHS with unbounded sampling processes and for regularized regression withℓ1penalty, by means of concentration inequalities. We shall then consider some approximation theory problems arising from learning schemes in RKHSs. Approximation theory properties of minimizing functions of the generalization error associated with ranking loss functions will be investigated and error analysis for related ranking algorithms will be conducted. High order approximation schemes for one-class support vector machines will be analyzed, and error bounds for the learning algorithm will be provided. For a general regularization scheme and the related online algorithm, the approximation in the RKHS will be investigated. The restricted isometry property in compressed sensing will be considered by methods from learning theory.