Preconditioned temporal difference learning

Hengshuai Yao; Zhi-Qiaug Liu

Preconditioned temporal difference learning

Hengshuai Yao
, Zhi-Qiaug Liu

School of Creative Media

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

15 Citations (Scopus)

Abstract

This paper extends many of the recent popular policy evaluation algorithms to a generalized framework that includes least-squares temporal difference (LSTD) learning, least-squares policy evaluation (LSPE) and a variant of incremental LSTD (iLSTD). The basis of this extension is a preconditioning technique that solves a stochastic model equation. This paper also studies three significant issues of the new framework: it presents a new rule of step-size that can he computed online, provides an iterative way to apply preconditioning, and reduces the complexity of related algorithms to near that of temporal difference (TD) learning. Copyright 2008 by the author(s)/owner(s).

Original language	English
Title of host publication	Proceedings of the 25th International Conference on Machine Learning
Pages	1208-1215
Publication status	Published - 2008
Event	25th International Conference on Machine Learning - Helsinki, Finland Duration: 5 Jul 2008 → 9 Jul 2008

Conference

Conference	25th International Conference on Machine Learning
Place	Finland
City	Helsinki
Period	5/07/08 → 9/07/08

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@inproceedings{d6538f80307047d08d88d1625f0743c4,

title = "Preconditioned temporal difference learning",

abstract = "This paper extends many of the recent popular policy evaluation algorithms to a generalized framework that includes least-squares temporal difference (LSTD) learning, least-squares policy evaluation (LSPE) and a variant of incremental LSTD (iLSTD). The basis of this extension is a preconditioning technique that solves a stochastic model equation. This paper also studies three significant issues of the new framework: it presents a new rule of step-size that can he computed online, provides an iterative way to apply preconditioning, and reduces the complexity of related algorithms to near that of temporal difference (TD) learning. Copyright 2008 by the author(s)/owner(s).",

author = "Hengshuai Yao and Zhi-Qiaug Liu",

year = "2008",

language = "English",

isbn = "9781605582054",

pages = "1208--1215",

booktitle = "Proceedings of the 25th International Conference on Machine Learning",

note = "25th International Conference on Machine Learning ; Conference date: 05-07-2008 Through 09-07-2008",

}

TY - GEN

T1 - Preconditioned temporal difference learning

AU - Yao, Hengshuai

AU - Liu, Zhi-Qiaug

PY - 2008

Y1 - 2008

N2 - This paper extends many of the recent popular policy evaluation algorithms to a generalized framework that includes least-squares temporal difference (LSTD) learning, least-squares policy evaluation (LSPE) and a variant of incremental LSTD (iLSTD). The basis of this extension is a preconditioning technique that solves a stochastic model equation. This paper also studies three significant issues of the new framework: it presents a new rule of step-size that can he computed online, provides an iterative way to apply preconditioning, and reduces the complexity of related algorithms to near that of temporal difference (TD) learning. Copyright 2008 by the author(s)/owner(s).

AB - This paper extends many of the recent popular policy evaluation algorithms to a generalized framework that includes least-squares temporal difference (LSTD) learning, least-squares policy evaluation (LSPE) and a variant of incremental LSTD (iLSTD). The basis of this extension is a preconditioning technique that solves a stochastic model equation. This paper also studies three significant issues of the new framework: it presents a new rule of step-size that can he computed online, provides an iterative way to apply preconditioning, and reduces the complexity of related algorithms to near that of temporal difference (TD) learning. Copyright 2008 by the author(s)/owner(s).

UR - https://www.scopus.com/pages/publications/56449123618

UR - https://www.scopus.com/pages/publications/56449123618#tab=impact

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9781605582054

SP - 1208

EP - 1215

BT - Proceedings of the 25th International Conference on Machine Learning

T2 - 25th International Conference on Machine Learning

Y2 - 5 July 2008 through 9 July 2008

ER -

Preconditioned temporal difference learning

Abstract

Conference

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this