Low-rank Nonparametric Regression and Application to Reinforcement Learning

Project: Research

View graph of relations

Description

The proposed research project will develop novel dimension reduction methods for nonparametric regression, based on the concept of low-rankness that has so far been prevalently applied for parametric regression. It is well-known that nonparametric regression suffers from curse of dimensionality, and parametric and semiparametric regression can be more efficient in practice, which however involves structural assumptions that are hard to verify. Using truncated singular value decomposition for the integral operator corresponding to a bivariate function, which can be extended via tensor decomposition to multivariate functions, the estimation of a generalnonparametric function can be reduced to the estimation of multiple univariate functions. More specifically, by using series estimation such as splines or wavelets for the singular functions, together with a nuclear norm regularization on the coefficient parameters, the estimation is the same as a convex parametric matrix/tensor regression problem in the existing literature. Theoretically, by carefully dealing with the bias in function approximation, as well as the effective covariates consisting of basis functions, we will aim to establish faster convergence rates for the nonparametric regression problem. Furthermore, as a mathematically more elegant approach by posing our problem in the reproducing kernel Hilbert space (RKHS), we can regularize the nonparametric function estimation using the trace norm of the integral operator, resulting in a problem similar to kernel ridge regression where the Hilbert space norm isreplaced by the trace norm. Theoretical analysis is the main challenge in this proposed estimator in the RKHS. Finally, we apply the proposed methodology to action-value function estimation in reinforcement learning with general state/action space, where thecovariates are naturally separated into the state variables and the action variables, and thus the low-rank approach can be used. 

Detail(s)

Project number9043383
Grant typeGRF
StatusNot started
Effective start/end date1/01/23 → …