The Optimal Solution Of Risk-sensitive Markov Control Processes

The optimal control problem is to determine a control policy that optimizes(i.e.,maximizes or minimizes)the performance criterion in a dynamical system.Markov control process is one of stochastic dynamical systems.The risk-sensitive average cri-terion is a popular criterion for optimization of stochastic dynamical systems in recent fifty years.Usual Markov control processes methodologies include the following two:(1)contractive mappings;(2)vanishing discount approach.Cavazos-Cadena(2009)put forward another analysis way,and its main results are obtained by an appropriate selec-tion of the relatively independent parameter and employ basic probabilistic and analysis principles.For a nonnegative cost function with compact support,the existence of a unique bounded solution of the optimality equation is proved by introducing relative value function and studying a parameterized expected-total cost problem with a stop-ping time.This article includes a brief historical perspective of the research efforts in this area,and then gives an emphasis and detailed introduction on Cavazos-Cadena' s method and applies this method on optimality equation under average reward criterion.Concerning Markov control processes on a denumerable state space and compact action sets,the performance of a control policy is measured by the risk-sensitive average cost criterion.To insure the solution existence of optimality equation in a class of risk-sensitive Markov control processes,the following assumptions are necessary:(i)the simultaneous Doeblin condition holds,and(ii)the system is communicating under the action of each stationary deterministic policy.Working within this framework,the aim is to verify for a nonnegative cost function with compact support,the optimal average cost function is constant.The optimal average cost function has a tight connection to the optimality equation,that is,the bounded solution of the optimality equation is the minimum value of average cost function and the policy which satisfies the optimality equation is the optimal stationary deterministic policy.Thus,the problem of making the average cost minimum switches into solving the optimality equation.