Large And Moderate Deviations For The Estimation Of Entropy

Entropy is a measure of uncertainty and information.It is widely used in physics,chemistry,information science,the social and economic field and so on.In early 1959,the[2]has proved the central limit theorem for the estimation of entropy of independent and identically distributed random variables valued in finite dimensional space.After in 1966,the central limit theorem for the estimation of entropy rate of ergodic homogeneous Markov process with finite state space is proved in[17].This paper mainly studies large and moderate deviations for the estimation of entropy.The large and moderate deviations theory is a very fruitful branch of the limit theory of probability.It deals with another kind of limit problem,which is different from the central limit theorem and the precision of the law of large numbers.Using some classical theory,this paper proves large and moderate deviations for the estimation of entropy in two kinds of different cases,independent and identically distributed random variables and Markov process with finite state space,and calculate the rate function,which is an extension of to the limit theorem of entropy.