Markov Chain Approximation of Path Dependent Hybrid Control System

Description

Since the extensive development of the stochastic control theory in the 1960s, establishing practically and environmentally reasonable mathematical models is definitely one of the most important and complicated issues in the modern decision making. As one of the developments in this vein, the hybrid system modeling has drawn considerable attentions from both researchers and practitioners in recent decades due to its distinct capability to capture the features of random environments. Such examples can be found in mathematical finance, ecosystem modeling, manufacturing systems, and risk theory, to name just a few. In particular, the systems often display qualitative structural changes by Markov chain, and it turns out to be quite versatile in capturing these inherent randomness.Owing to the complex nature of control problems, the solutions are highly nonlinear and closed-form solutions are not obtainable. Thus, the numerical methods become a viable alternative. Two major types of numerical methods include probabilistic method (e.g., Markov chain approximation initiated by Kushner [see Kushner(1990)] and numerical PDE method (e.g., Finite different/element method). Both developments are highly dependent on the dynamic programming principle. However, dynamic programming principle may not be true for a class of path-dependent stochastic hybrid control problems.In this project, we will take up the challenges of numerical method on a class of path-dependent stochastic hybrid control systems. We aim to design and analyze a feasible Markov chain approximation to the path-dependent stochastic control problem in a mathematically rigorous manner. The main difficulty is that, due to the lack of dynamic programming principle, one cannot expect the conventional Markov chain approximation method or finite difference method to be worked out. Different from the existing theory of the Markov chain approximations, we develop an alternative approach via weak convergence for this purpose, which requires the actual computations on an appropriate topology equivalent to conventional Skorohod topology. Not only does this proof enable us to establish the convergence of the path-dependent control problems, but also it leads to sufficient conditions for the non-occurrence of tangency problem.One advantage of Markov chain approximation over numerical PDE is the flexibility of the construction of various types of good Markov chains. The aforementioned theoretical study of convergence makes it possible to open up a round of new investigations on the comparison of various Markov chain approximations on path-dependent control problems. In this direction, a multilevel approach on Markov chain approximation will be investigated to multidimensional problems to reduce computational cost.