Improved Learning Rates of a Functional Lasso-type SVM with Sparse Multi-Kernel Representation

Shaogao Lv; Junhui Wang; Jiankun Liu; Yong Liu

Improved Learning Rates of a Functional Lasso-type SVM with Sparse Multi-Kernel Representation

Shaogao Lv
, Junhui Wang
, Jiankun Liu
, Yong Liu^*

^*Corresponding author for this work

City University of Hong Kong

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

6 Citations (Scopus)

Abstract

In this paper, we provide theoretical results of estimation bounds and excess risk upper bounds for support vector machine (SVM) with sparse multi-kernel representation. These convergence rates for multi-kernel SVM are established by analyzing a Lasso-type regularized learning scheme within composite multi-kernel spaces. It is shown that the oracle rates of convergence of classifiers depend on the complexity of multi-kernels, the sparsity, a Bernstein condition and the sample size, which significantly improve on previous results even for the additive or linear cases. In summary, this paper not only provides unified theoretical results for multi-kernel SVMs, but also enriches the literature on high-dimensional nonparametric classification.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 34
Subtitle of host publication	35th Conference on Neural Information Processing Systems (NeurIPS 2021)
Editors	M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, J. Wortman Vaughan
Publisher	Neural Information Processing Systems (NeurIPS)
Pages	21467-21479
Volume	26
ISBN (Print)	9781713845393
Publication status	Published - Dec 2021
Event	35th Conference on Neural Information Processing Systems (NeurIPS 2021) - Virtual, Los Angeles, United States Duration: 6 Dec 2021 → 14 Dec 2021 https://nips.cc/virtual/2021/index.html https://papers.nips.cc/paper/2021 https://media.neurips.cc/Conferences/NeurIPS2021/NeurIPS_2021_poster.pdf https://www.proceedings.com/63069.html

Publication series

Name	Advances in Neural Information Processing Systems
ISSN (Print)	1049-5258

Conference

Conference	35th Conference on Neural Information Processing Systems (NeurIPS 2021)
Place	United States
City	Los Angeles
Period	6/12/21 → 14/12/21
Internet address	https://nips.cc/virtual/2021/index.html https://papers.nips.cc/paper/2021 https://media.neurips.cc/Conferences/NeurIPS2021/NeurIPS_2021_poster.pdf https://www.proceedings.com/63069.html

Access Output

https://papers.nips.cc/paper/2021/hash/b31df16a88ce00fed951f24b46e08649-Abstract.html

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Lv, S., Wang, J., Liu, J., & Liu, Y. (2021). Improved Learning Rates of a Functional Lasso-type SVM with Sparse Multi-Kernel Representation. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, & J. W. Vaughan (Eds.), Advances in Neural Information Processing Systems 34: 35th Conference on Neural Information Processing Systems (NeurIPS 2021) (Vol. 26, pp. 21467-21479). (Advances in Neural Information Processing Systems). Neural Information Processing Systems (NeurIPS). https://papers.nips.cc/paper/2021/hash/b31df16a88ce00fed951f24b46e08649-Abstract.html

Lv, Shaogao ; Wang, Junhui ; Liu, Jiankun et al. / Improved Learning Rates of a Functional Lasso-type SVM with Sparse Multi-Kernel Representation. Advances in Neural Information Processing Systems 34: 35th Conference on Neural Information Processing Systems (NeurIPS 2021). editor / M. Ranzato ; A. Beygelzimer ; Y. Dauphin ; P.S. Liang ; J. Wortman Vaughan. Vol. 26 Neural Information Processing Systems (NeurIPS), 2021. pp. 21467-21479 (Advances in Neural Information Processing Systems).

@inproceedings{13c8267b37f0417fbc6db40bd0d3d22f,

title = "Improved Learning Rates of a Functional Lasso-type SVM with Sparse Multi-Kernel Representation",

abstract = "In this paper, we provide theoretical results of estimation bounds and excess risk upper bounds for support vector machine (SVM) with sparse multi-kernel representation. These convergence rates for multi-kernel SVM are established by analyzing a Lasso-type regularized learning scheme within composite multi-kernel spaces. It is shown that the oracle rates of convergence of classifiers depend on the complexity of multi-kernels, the sparsity, a Bernstein condition and the sample size, which significantly improve on previous results even for the additive or linear cases. In summary, this paper not only provides unified theoretical results for multi-kernel SVMs, but also enriches the literature on high-dimensional nonparametric classification.",

author = "Shaogao Lv and Junhui Wang and Jiankun Liu and Yong Liu",

year = "2021",

month = dec,

language = "English",

isbn = "9781713845393",

volume = "26",

series = "Advances in Neural Information Processing Systems",

publisher = "Neural Information Processing Systems (NeurIPS)",

pages = "21467--21479",

editor = "M. Ranzato and A. Beygelzimer and Y. Dauphin and P.S. Liang and Vaughan, \{J. Wortman\}",

booktitle = "Advances in Neural Information Processing Systems 34",

address = "United States",

note = "35th Conference on Neural Information Processing Systems (NeurIPS 2021) ; Conference date: 06-12-2021 Through 14-12-2021",

url = "https://nips.cc/virtual/2021/index.html, https://papers.nips.cc/paper/2021, https://media.neurips.cc/Conferences/NeurIPS2021/NeurIPS\_2021\_poster.pdf, https://www.proceedings.com/63069.html",

}

Lv, S, Wang, J, Liu, J & Liu, Y 2021, Improved Learning Rates of a Functional Lasso-type SVM with Sparse Multi-Kernel Representation. in M Ranzato, A Beygelzimer, Y Dauphin, PS Liang & JW Vaughan (eds), Advances in Neural Information Processing Systems 34: 35th Conference on Neural Information Processing Systems (NeurIPS 2021). vol. 26, Advances in Neural Information Processing Systems, Neural Information Processing Systems (NeurIPS), pp. 21467-21479, 35th Conference on Neural Information Processing Systems (NeurIPS 2021), Los Angeles, United States, 6/12/21. <https://papers.nips.cc/paper/2021/hash/b31df16a88ce00fed951f24b46e08649-Abstract.html>

Improved Learning Rates of a Functional Lasso-type SVM with Sparse Multi-Kernel Representation. / Lv, Shaogao; Wang, Junhui; Liu, Jiankun et al.
Advances in Neural Information Processing Systems 34: 35th Conference on Neural Information Processing Systems (NeurIPS 2021). ed. / M. Ranzato; A. Beygelzimer; Y. Dauphin; P.S. Liang; J. Wortman Vaughan. Vol. 26 Neural Information Processing Systems (NeurIPS), 2021. p. 21467-21479 (Advances in Neural Information Processing Systems).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Improved Learning Rates of a Functional Lasso-type SVM with Sparse Multi-Kernel Representation

AU - Lv, Shaogao

AU - Wang, Junhui

AU - Liu, Jiankun

AU - Liu, Yong

PY - 2021/12

Y1 - 2021/12

N2 - In this paper, we provide theoretical results of estimation bounds and excess risk upper bounds for support vector machine (SVM) with sparse multi-kernel representation. These convergence rates for multi-kernel SVM are established by analyzing a Lasso-type regularized learning scheme within composite multi-kernel spaces. It is shown that the oracle rates of convergence of classifiers depend on the complexity of multi-kernels, the sparsity, a Bernstein condition and the sample size, which significantly improve on previous results even for the additive or linear cases. In summary, this paper not only provides unified theoretical results for multi-kernel SVMs, but also enriches the literature on high-dimensional nonparametric classification.

AB - In this paper, we provide theoretical results of estimation bounds and excess risk upper bounds for support vector machine (SVM) with sparse multi-kernel representation. These convergence rates for multi-kernel SVM are established by analyzing a Lasso-type regularized learning scheme within composite multi-kernel spaces. It is shown that the oracle rates of convergence of classifiers depend on the complexity of multi-kernels, the sparsity, a Bernstein condition and the sample size, which significantly improve on previous results even for the additive or linear cases. In summary, this paper not only provides unified theoretical results for multi-kernel SVMs, but also enriches the literature on high-dimensional nonparametric classification.

UR - https://www.scopus.com/pages/publications/85131786530

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85131786530&origin=recordpage

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9781713845393

VL - 26

T3 - Advances in Neural Information Processing Systems

SP - 21467

EP - 21479

BT - Advances in Neural Information Processing Systems 34

A2 - Ranzato, M.

A2 - Beygelzimer, A.

A2 - Dauphin, Y.

A2 - Liang, P.S.

A2 - Vaughan, J. Wortman

PB - Neural Information Processing Systems (NeurIPS)

T2 - 35th Conference on Neural Information Processing Systems (NeurIPS 2021)

Y2 - 6 December 2021 through 14 December 2021

ER -

Lv S, Wang J, Liu J, Liu Y. Improved Learning Rates of a Functional Lasso-type SVM with Sparse Multi-Kernel Representation. In Ranzato M, Beygelzimer A, Dauphin Y, Liang PS, Vaughan JW, editors, Advances in Neural Information Processing Systems 34: 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Vol. 26. Neural Information Processing Systems (NeurIPS). 2021. p. 21467-21479. (Advances in Neural Information Processing Systems).

Improved Learning Rates of a Functional Lasso-type SVM with Sparse Multi-Kernel Representation

Abstract

Publication series

Conference

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this