Large Deviations For Markov Decision Process And Related Problems

Markov decision process(MDP)is a class of stochastic control systems.The theory and fundamental conceptions were first advanced by Bellman in 1950 s.Many results have been obtained since 1960 s.In the research of Markov decision model,the core problems are the existence and solution of the optimal strategy and the analysis of the optimal function.In this dissertation,we study a specific Markov decision model defined by iteration.The increments of the model are affected by random noise,and we employ random actions to control the influence of the noise.We advance a new approach to analyze the model by studying the limit distributions of the path.Then we discuss the method of solving the optimal strategies and obtaining the corresponding optimal values of several optimization problems with practical background.During the research,we prove that under certain conditions,the distributions of the path satisfy the large deviation principle,while under weaker conditions,the distributions of the path satisfy the law of large numbers.According to the results of the distributions of the path,we propose a new numerical method to solve a class of differential equations with discontinuous structure.The main contributions of this dissertation are as follows.1.By employing stochas-tic actions to control the stochastic iterative system,we propose a new kind of optimiza-tion problem.2.The optimal strategy is found by studying the influence of strategies on limit path.3.We study the behavior of large deviation in the process of limit path,and analyze the optimization problem through the large deviation rate function.4.We apply a new transformation technique to transforming a complex system into a simple system while proving that the distributions of the path satisfy the large deviation principle.5.For a class of differential equations with discontinuous structure,we prove the existence of solutions of this kind of differential equations with the help of probability theory,and propose a new stochastic version of Euler method.