Some Researches On Theoretical Problems In Information Theory

In this paper,by using a new method of studying the strong limit theorem,the generalized strong deviation theorem and entropy theorem of nonhomogeneous Markov chain are discussed.The estimation of lower bound of the relative entropy of the arbitrary information sources relative to nonhomogeneous Markov chain is obtained.The lower bound of the relative entropy of nonhomogeneous Markov chain indexed by tree is also obtained.The basic approach is to construct a likelihood ratio(or moving likelihood ratio)with a parameter and then use the classical tools of Markov inequality and Borel-Cantelli lemma to get the almost sure convergence of likelihood ratio(or generalized likelihood ratio).Finally,by taking proper limit of the parameter,the proof of the theorems is completed.The first chapter introduces the basic concept in information,the methods and the background of the strong deviation theorem and the content of this paper briefly.The second chapter presents some basic concepts,definition,methods and some lemmas used in this article.In the third chapter,taking Markov measure as a reference measure and considering moving sample relative entropy as a random deviation between arbitrary random variables and Markov chains and by restricting the moving relative entropy,a subset of the sample space is given.On this subset,the strong deviation theorems(i.e.limit theorem represented by inequalities)of moving average for functions of two random variables is obtained.In the fourth chapter,the approximation of arbitrary information source by Markov chains is studied.A estimation of the lower bound of moving relative entropy of the sample presented by the relative frequency of sample function is obtained.In the fifth chapter,by limiting the difference between the sample function of Markov chain indexed by trees and its expectation of reference measure,a lower bound of the relative entropy of the Markov chain indexed trees is given.In particular,when the sample function satisfies the law of large numbers,the lower bound of the relative entropy of the Markov chains indexed by trees is 0.