Boosted Kernel Ridge Regression : Optimal Learning Rates and Early Stopping
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Article number | 46 |
Journal / Publication | Journal of Machine Learning Research |
Volume | 20 |
Online published | Feb 2019 |
Publication status | Published - 2019 |
Link(s)
Attachment(s) | Documents
Publisher's Copyright Statement
|
---|---|
Document Link | Links
|
Link to Scopus | https://www.scopus.com/record/display.uri?eid=2-s2.0-85072647947&origin=recordpage |
Permanent Link | https://scholars.cityu.edu.hk/en/publications/publication(529f1b08-d4b8-4411-bc37-f4888604a385).html |
Abstract
In this paper, we introduce a learning algorithm, boosted kernel ridge regression (BKRR), that combines L2-Boosting with the kernel ridge regression (KRR). We analyze the learning performance of this algorithm in the framework of learning theory. We show that BKRR provides a new bias-variance trade-off via tuning the number of boosting iterations, which is different from KRR via adjusting the regularization parameter. A (semi-)exponential bias-variance trade-off is derived for BKRR, exhibiting a stable relationship between the generalization error and the number of iterations. Furthermore, an adaptive stopping rule is proposed, with which BKRR achieves the optimal learning rate without saturation.
Research Area(s)
- learning theory, kernel ridge regression, boosting, integral operator, ITERATED TIKHONOV REGULARIZATION, SPECTRAL ALGORITHMS, GRADIENT, APPROXIMATION, PARAMETER, OPERATORS, THEOREM, CHOICE
Citation Format(s)
Boosted Kernel Ridge Regression: Optimal Learning Rates and Early Stopping. / Lin, Shao-Bo; Lei, Yunwen; Zhou, Ding-Xuan.
In: Journal of Machine Learning Research, Vol. 20, 46, 2019.
In: Journal of Machine Learning Research, Vol. 20, 46, 2019.
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review
Download Statistics
No data available