TY - JOUR
T1 - Support vector machine soft margin classifiers
T2 - Error analysis
AU - Chen, Di-Rong
AU - Wu, Qiang
AU - Ying, Yiming
AU - Zhou, Ding-Xuan
PY - 2004/9/1
Y1 - 2004/9/1
N2 - The purpose of this paper is to provide a PAC error analysis for the q-norm soft margin classifier, a support vector machine classification algorithm. It consists of two parts: regularization error and sample error. While many techniques are available for treating the sample error, much less is known for the regularization error and the corresponding approximation error for reproducing kernel Hilbert spaces. We are mainly concerned about the regularization error. It is estimated for general distributions by a K-functional in weighted Lq spaces. For weakly separable distributions (i.e., the margin may be zero) satisfactory convergence rates are provided by means of separating functions. A projection operator is introduced, which leads to better sample error estimates especially for small complexity kernels. The misclassification error is bounded by the V-risk associated with a general class of loss functions V. The difficulty of bounding the offset is overcome. Polynomial kernels and Gaussian kernels are used to demonstrate the main results. The choice of the regularization parameter plays an important role in our analysis.
AB - The purpose of this paper is to provide a PAC error analysis for the q-norm soft margin classifier, a support vector machine classification algorithm. It consists of two parts: regularization error and sample error. While many techniques are available for treating the sample error, much less is known for the regularization error and the corresponding approximation error for reproducing kernel Hilbert spaces. We are mainly concerned about the regularization error. It is estimated for general distributions by a K-functional in weighted Lq spaces. For weakly separable distributions (i.e., the margin may be zero) satisfactory convergence rates are provided by means of separating functions. A projection operator is introduced, which leads to better sample error estimates especially for small complexity kernels. The misclassification error is bounded by the V-risk associated with a general class of loss functions V. The difficulty of bounding the offset is overcome. Polynomial kernels and Gaussian kernels are used to demonstrate the main results. The choice of the regularization parameter plays an important role in our analysis.
KW - Approximation error
KW - Misclassification error
KW - q-norm soft margin classifier
KW - Regularization error
KW - Support vector machine classification
UR - http://www.scopus.com/inward/record.url?scp=84879394399&partnerID=8YFLogxK
UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-84879394399&origin=recordpage
M3 - RGC 21 - Publication in refereed journal
SN - 1533-7928
VL - 5
SP - 1143
EP - 1175
JO - Journal of Machine Learning Research
JF - Journal of Machine Learning Research
ER -