Regime-switching control problems and related Markov chain approximation methods

Abstract

The stochastic control problems and two-player stochastic differential game problems with regime switching are two of the most important problems in control theory and generally in mathematical finance. They have been received much attention for many applications in different fields, such as pursuit-evasion games, queuing systems in heavy traffic, risk-sensitive control, and constrained optimization problems, since formulated in late 1970s. Various approximation methods for stochastic control have been studied and corresponding convergence analyses have been done. In particular, the Markov chain approximation method is a powerful and widely applied in numerical problems for controlled stochastic processes. Generally speaking, the approximating Markov chain starts by approximating the original controlled process by an appropriate controlled Markov chain on a state space. The approximation parameters are denoted by h and δ and the original cost functional is approximated such that it's suitable for the defined Markov chain. Moreover, the Markov chain also keeps track of the regimes when dealing with the stochastic problem with regimeswitching. In many cases, the time interval in the stochastic control problem is bounded by a finite number T, which makes the problem is a solution of a parabolic partial differential equation. The main goal of this thesis is to make some contributions to the Markov chain approximation methods to time-dependent regime-switching stochastic control problems on a finite time horizon as well as the convergence of the algorithms by means of weak convergence methods. Furthermore, the application of this numerical scheme expands to the underlying game problems. The sufficient conditions for the existence of a saddle point of a discrete Markov game constructed by Markov chain approximation of stochastic differential games in a general setup are provided. In addition, numerical solutions of several examples are provided for demonstration purpose.

Date of Award	2 Oct 2013
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Kam Moon Lester LIU (Supervisor), Qingshuo SONG (Supervisor) & Nicolas PRIVAULT (Supervisor)