On Convergence Rate of Adaptive Multiscale Value Function Approximation for Reinforcement Learning

Tao Li; Quanyan Zhu

doi:10.1109/MLSP.2019.8918816

On Convergence Rate of Adaptive Multiscale Value Function Approximation for Reinforcement Learning

Tao Li, Quanyan Zhu

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

10 Citations (Scopus)

Abstract

In this paper, we propose a generic framework for devising an adaptive approximation scheme for value function approximation in reinforcement learning, which introduces multiscale approximation. The two basic ingredients are multiresolution analysis as well as tree approximation. Starting from simple refinable functions, multiresolution analysis enables us to construct a wavelet system from which the basis functions are selected adaptively, resulting in a tree structure. Furthermore, we present the convergence rate of our multiscale approximation which does not depend on the regularity of basis functions. © 2019 IEEE.

Original language	English
Title of host publication	2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP)
Publisher	IEEE
Number of pages	6
ISBN (Electronic)	978-1-7281-0824-7
DOIs	https://doi.org/10.1109/MLSP.2019.8918816
Publication status	Published - 2019
Externally published	Yes
Event	29th IEEE International Workshop on Machine Learning for Signal Processing (MLSP) - Pittsburgh, United States Duration: 13 Oct 2019 → 16 Oct 2019

Publication series

Name	IEEE International Workshop on Machine Learning for Signal Processing, MLSP
Volume	2019-October
ISSN (Print)	2161-0363
ISSN (Electronic)	2161-0371

Conference

Conference	29th IEEE International Workshop on Machine Learning for Signal Processing (MLSP)
Place	United States
City	Pittsburgh
Period	13/10/19 → 16/10/19

Funding

This research is supported in part by National Science Foundation (NSF) under grant ECCS-1847056, CNS-1544782, and SES-1541164, and in part by ARO grant W911NF1910041.

Research Keywords

multiresolution analysis
Multiscale approximation
n-term approximation
reinforcement learning
tree approximation
wavelets

Access Output

10.1109/MLSP.2019.8918816

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Li, T., & Zhu, Q. (2019). On Convergence Rate of Adaptive Multiscale Value Function Approximation for Reinforcement Learning. In 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP) Article 8918816 (IEEE International Workshop on Machine Learning for Signal Processing, MLSP; Vol. 2019-October). IEEE. https://doi.org/10.1109/MLSP.2019.8918816

@inproceedings{f606413d40014b90b9c00fbe614b5a30,

title = "On Convergence Rate of Adaptive Multiscale Value Function Approximation for Reinforcement Learning",

abstract = "In this paper, we propose a generic framework for devising an adaptive approximation scheme for value function approximation in reinforcement learning, which introduces multiscale approximation. The two basic ingredients are multiresolution analysis as well as tree approximation. Starting from simple refinable functions, multiresolution analysis enables us to construct a wavelet system from which the basis functions are selected adaptively, resulting in a tree structure. Furthermore, we present the convergence rate of our multiscale approximation which does not depend on the regularity of basis functions. {\textcopyright} 2019 IEEE.",

keywords = "multiresolution analysis, Multiscale approximation, n-term approximation, reinforcement learning, tree approximation, wavelets",

author = "Tao Li and Quanyan Zhu",

year = "2019",

doi = "10.1109/MLSP.2019.8918816",

language = "English",

series = "IEEE International Workshop on Machine Learning for Signal Processing, MLSP",

publisher = "IEEE",

booktitle = "2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP)",

address = "United States",

note = "29th IEEE International Workshop on Machine Learning for Signal Processing (MLSP) ; Conference date: 13-10-2019 Through 16-10-2019",

}

Li, T & Zhu, Q 2019, On Convergence Rate of Adaptive Multiscale Value Function Approximation for Reinforcement Learning. in 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP)., 8918816, IEEE International Workshop on Machine Learning for Signal Processing, MLSP, vol. 2019-October, IEEE, 29th IEEE International Workshop on Machine Learning for Signal Processing (MLSP), Pittsburgh, United States, 13/10/19. https://doi.org/10.1109/MLSP.2019.8918816

On Convergence Rate of Adaptive Multiscale Value Function Approximation for Reinforcement Learning. / Li, Tao; Zhu, Quanyan.
2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP). IEEE, 2019. 8918816 (IEEE International Workshop on Machine Learning for Signal Processing, MLSP; Vol. 2019-October).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - On Convergence Rate of Adaptive Multiscale Value Function Approximation for Reinforcement Learning

AU - Li, Tao

AU - Zhu, Quanyan

PY - 2019

Y1 - 2019

N2 - In this paper, we propose a generic framework for devising an adaptive approximation scheme for value function approximation in reinforcement learning, which introduces multiscale approximation. The two basic ingredients are multiresolution analysis as well as tree approximation. Starting from simple refinable functions, multiresolution analysis enables us to construct a wavelet system from which the basis functions are selected adaptively, resulting in a tree structure. Furthermore, we present the convergence rate of our multiscale approximation which does not depend on the regularity of basis functions. © 2019 IEEE.

AB - In this paper, we propose a generic framework for devising an adaptive approximation scheme for value function approximation in reinforcement learning, which introduces multiscale approximation. The two basic ingredients are multiresolution analysis as well as tree approximation. Starting from simple refinable functions, multiresolution analysis enables us to construct a wavelet system from which the basis functions are selected adaptively, resulting in a tree structure. Furthermore, we present the convergence rate of our multiscale approximation which does not depend on the regularity of basis functions. © 2019 IEEE.

KW - multiresolution analysis

KW - Multiscale approximation

KW - n-term approximation

KW - reinforcement learning

KW - tree approximation

KW - wavelets

UR - http://www.scopus.com/inward/record.url?scp=85077700076&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85077700076&origin=recordpage

U2 - 10.1109/MLSP.2019.8918816

DO - 10.1109/MLSP.2019.8918816

M3 - RGC 32 - Refereed conference paper (with host publication)

T3 - IEEE International Workshop on Machine Learning for Signal Processing, MLSP

BT - 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP)

PB - IEEE

T2 - 29th IEEE International Workshop on Machine Learning for Signal Processing (MLSP)

Y2 - 13 October 2019 through 16 October 2019

ER -

On Convergence Rate of Adaptive Multiscale Value Function Approximation for Reinforcement Learning

Abstract

Publication series

Conference

Funding

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this