Convergence rates of Markov chain approximation methods for controlled diffusions with stopping

This work is concerned with rates of convergence of numerical methods using Markov chain approximation for controlled diffusions with stopping (the first exit time from a bounded region). In lieu of considering the associated finite difference schemes for Hamilton-Jacobi-Bellman (HJB) equations, a purely probabilistic approach is used. There is an added difficulty due to the boundary condition, which requires the continuity of the first exit time with respect to the discrete parameter. To prove the convergence of the algorithm by Markov chain approximation method, a tangency problem might arise. A common approach uses certain conditions to avoid the tangency problem. Here, by modifying the value function, it is demonstrated that the tangency problem will not arise in the sense of convergence in probability and in L ¹. In addition, controlled diffusions with a discount factor is also treated. © 2010 Institute of Systems Science, Academy of Mathematics and Systems Science, CAS and Springer-Verlag Berlin Heidelberg.