Multi-kernel regularized classifiers

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

131 Scopus Citations
View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)108-134
Journal / PublicationJournal of Complexity
Volume23
Issue number1
Publication statusPublished - Feb 2007

Abstract

A family of classification algorithms generated from Tikhonov regularization schemes are considered. They involve multi-kernel spaces and general convex loss functions. Our main purpose is to provide satisfactory estimates for the excess misclassification error of these multi-kernel regularized classifiers when the loss functions achieve the zero value. The error analysis consists of two parts: regularization error and sample error. Allowing multi-kernels in the algorithm improves the regularization error and approximation error, which is one advantage of the multi-kernel setting. For a general loss function, we show how to bound the regularization error by the approximation in some weighted Lq spaces. For the sample error, we use a projection operator. The projection in connection with the decay of the regularization error enables us to improve convergence rates in the literature even for the one-kernel schemes and special loss functions: least-square loss and hinge loss for support vector machine soft margin classifiers. Existence of the optimization problem for the regularization scheme associated with multi-kernels is verified when the kernel functions are continuous with respect to the index set. Concrete examples, including Gaussian kernels with flexible variances and probability distributions with some noise conditions, are used to illustrate the general theory. © 2006 Elsevier Inc. All rights reserved.

Research Area(s)

  • Classification algorithm, Convex loss function, Misclassification error, Multi-kernel regularization scheme, Regularization error and sample error

Citation Format(s)

Multi-kernel regularized classifiers. / Wu, Qiang; Ying, Yiming; Zhou, Ding-Xuan.

In: Journal of Complexity, Vol. 23, No. 1, 02.2007, p. 108-134.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review