Some Results Arising From Discrete-Time Stochastic LQ Optimal Control Problem

The study on the topic of linear quadratic optimal control problem for deterministic linear state space systems (LQ problem for short) was initiated by a lot of researchers, and was stepped into a new era due to the work of Kalman. With the development of modern control theory, scientists have been trying to approach the LQ problem from different perspectives. The conducted research on the LQ problem does not only deepen people's understanding concerning it, but widen the field to which the LQ theory is applicable.In recent decades, both the LQ problem for It(o|^) stochastic systems and the LQ problem for discrete-time stochastic systems have been extensively investigated by researchers. Moreover, the study revealed that for It(o|^) stochastic systems, even if the control weighting matrix is indefinite, the corresponding problem can still be guaranteed to be well-posed. It was also justified in mathematical finance that these theoretical results enable us to have a deeper understanding of the mean-variance portfolio theory.Although there are some fundamental differences between the LQ problem for deterministic state space systems and the LQ problem for stochastic systems, we have to admit one point, i.e., some existed methods which were made use of to study the deterministic problem can also be applied to the stochastic case. On the basis of this idea, we combine in this paper the asymptotic analysis method with the numerical integration technique to concentrate on the infinite horizon discrete-time stochastic LQ problems.It is worth mentioning that during our discussion, we will prove, by following the way which appeared in It(o|^) stochastic control theory, the Popov-Belevith-Hautus criterion for the discrete-time stochastic systems affiliated with a measurement output to be exactly observable. Moreover, we will introduce the notion of exact detectability, and discuss its relationship with exact observability coupled with its properties.Last but not least, we will provide in each section some numerical examples to justify the developed theoretical results.