Sparse multiple kernel learning: Minimax rates with random projection

Wenqi Lu; Zhongyi Zhu; Rui Li; Heng Lian

doi:10.1016/j.jspi.2023.106142

Sparse multiple kernel learning: Minimax rates with random projection

Wenqi Lu, Zhongyi Zhu, Rui Li^*, Heng Lian

^*Corresponding author for this work

Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review

Abstract

In kernel-based learning, the random projection method, also called random sketching, has been successfully used in kernel ridge regression to reduce the computational burden in the big data setting, and at the same time retain the minimax convergence rate. In this work, we consider its use in sparse multiple kernel learning problems where a closed-form optimizer is not available, which poses significant technical challenges, for which the existing results do not carry over directly. Even when random projection is not used, our risk bound improves on the existing results in several aspects. We also illustrate the use of random projection via some numerical examples. © 2023 Elsevier B.V.

Original language	English
Article number	106142
Journal	Journal of Statistical Planning and Inference
Volume	231
Online published	27 Dec 2023
DOIs	https://doi.org/10.1016/j.jspi.2023.106142
Publication status	Published - Jul 2024

Funding

The authors sincerely thank the editor, associate editor, and two reviewers for their insightful and constructive comments that significantly improved the manuscript. The research of Wenqi Lu is supported by NSFC, China ( 12301343 ), Tianjin Municipal Natural Science Foundation, China ( 22JCQNJC01670 ), and Fundamental Research Funds for the Central Universities, Nankai University, China ( 63231192 ). The research of Zhongyi Zhu is supported by NSFC, China ( 12331009 , 12071087 ). The research of Rui Li is supported by Humanities and Social Sciences Fund of Ministry of Education 23YJA910003 . The research of Heng Lian is partially supported by NSFC, China 12371297 at CityU Shenzhen Research Institute, NSF of Jiangxi Province, China under Grant 20223BCJ25017 , and by Hong Kong RGC general research fund 11300519 , 11300721 and 11311822 , and by CityU internal grant 7006014 and 9680239 .

Research Keywords

Convergence rate
Kernel method
Random projection
Reproducing kernel Hilbert space

RGC Funding Information

RGC-funded

Access Output

10.1016/j.jspi.2023.106142

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@article{e975cbf6a9ba4229a593e1c792fde82c,

title = "Sparse multiple kernel learning: Minimax rates with random projection",

abstract = "In kernel-based learning, the random projection method, also called random sketching, has been successfully used in kernel ridge regression to reduce the computational burden in the big data setting, and at the same time retain the minimax convergence rate. In this work, we consider its use in sparse multiple kernel learning problems where a closed-form optimizer is not available, which poses significant technical challenges, for which the existing results do not carry over directly. Even when random projection is not used, our risk bound improves on the existing results in several aspects. We also illustrate the use of random projection via some numerical examples. {\textcopyright} 2023 Elsevier B.V.",

keywords = "Convergence rate, Kernel method, Random projection, Reproducing kernel Hilbert space",

author = "Wenqi Lu and Zhongyi Zhu and Rui Li and Heng Lian",

year = "2024",

month = jul,

doi = "10.1016/j.jspi.2023.106142",

language = "English",

volume = "231",

journal = "Journal of Statistical Planning and Inference",

issn = "0378-3758",

publisher = "Elsevier B.V.",

}

TY - JOUR

T1 - Sparse multiple kernel learning

T2 - Minimax rates with random projection

AU - Lu, Wenqi

AU - Zhu, Zhongyi

AU - Li, Rui

AU - Lian, Heng

PY - 2024/7

Y1 - 2024/7

N2 - In kernel-based learning, the random projection method, also called random sketching, has been successfully used in kernel ridge regression to reduce the computational burden in the big data setting, and at the same time retain the minimax convergence rate. In this work, we consider its use in sparse multiple kernel learning problems where a closed-form optimizer is not available, which poses significant technical challenges, for which the existing results do not carry over directly. Even when random projection is not used, our risk bound improves on the existing results in several aspects. We also illustrate the use of random projection via some numerical examples. © 2023 Elsevier B.V.

AB - In kernel-based learning, the random projection method, also called random sketching, has been successfully used in kernel ridge regression to reduce the computational burden in the big data setting, and at the same time retain the minimax convergence rate. In this work, we consider its use in sparse multiple kernel learning problems where a closed-form optimizer is not available, which poses significant technical challenges, for which the existing results do not carry over directly. Even when random projection is not used, our risk bound improves on the existing results in several aspects. We also illustrate the use of random projection via some numerical examples. © 2023 Elsevier B.V.

KW - Convergence rate

KW - Kernel method

KW - Random projection

KW - Reproducing kernel Hilbert space

UR - http://www.scopus.com/inward/record.url?scp=85181870092&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85181870092&origin=recordpage

U2 - 10.1016/j.jspi.2023.106142

DO - 10.1016/j.jspi.2023.106142

M3 - RGC 21 - Publication in refereed journal

SN - 0378-3758

VL - 231

JO - Journal of Statistical Planning and Inference

JF - Journal of Statistical Planning and Inference

M1 - 106142

ER -

Sparse multiple kernel learning: Minimax rates with random projection

Abstract

Funding

Research Keywords

RGC Funding Information

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

GRF: Low-rank Nonparametric Regression and Application to Reinforcement Learning

GRF: Distributed Estimation with Random Projection in Reproducing Kernel Hilbert Spaces

GRF: Low-rank tensor as a Dimension Reduction Tool in Complex Data Analysis

Cite this

Sparse multiple kernel learning: Minimax rates with random projection

Abstract

Funding

Research Keywords

RGC Funding Information

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Projects

GRF: Low-rank Nonparametric Regression and Application to Reinforcement Learning

GRF: Distributed Estimation with Random Projection in Reproducing Kernel Hilbert Spaces

GRF: Low-rank tensor as a Dimension Reduction Tool in Complex Data Analysis

Cite this