Debiased Distributed Learning for Sparse Partial Linear Models in High Dimensions
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Article number | 2 |
Journal / Publication | Journal of Machine Learning Research |
Volume | 23 |
Online published | Dec 2021 |
Publication status | Published - 2022 |
Link(s)
Attachment(s) | Documents
Publisher's Copyright Statement
|
---|---|
Document Link | Links
|
Link to Scopus | https://www.scopus.com/record/display.uri?eid=2-s2.0-85124227641&origin=recordpage |
Permanent Link | https://scholars.cityu.edu.hk/en/publications/publication(3e74449a-e860-4c96-927e-56f851f755f0).html |
Abstract
Although various distributed machine learning schemes have been proposed recently for purely linear models and fully nonparametric models, little attention has been paid to distributed optimization for semi-parametric models with multiple structures (e.g. sparsity, linearity and nonlinearity). To address these issues, the current paper proposes a new communication-efficient distributed learning algorithm for sparse partially linear models with an increasing number of features. The proposed method is based on the classical divide and conquer strategy for handling big data and the computation on each subsample consists of a debiased estimation of the doubly regularized least squares approach. With the proposed method, we theoretically prove that our global parametric estimator can achieve the optimal parametric rate in our semi-parametric model given an appropriate partition on the total data. Specifically, the choice of data partition relies on the underlying smoothness of the nonparametric component, and it is adaptive to the sparsity parameter. Finally, some simulated experiments are carried out to illustrate the empirical performances of our debiased technique under the distributed setting.
Research Area(s)
- Big data, Distributed learning, High dimensions, Reproducing kernel Hilbert space (RKHS), Semi-parametric models
Citation Format(s)
Debiased Distributed Learning for Sparse Partial Linear Models in High Dimensions. / Lv, Shaogao; Lian, Heng.
In: Journal of Machine Learning Research, Vol. 23, 2, 2022.
In: Journal of Machine Learning Research, Vol. 23, 2, 2022.
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review
Download Statistics
No data available