Structure Learning Of Dynamic System In Complex Environment

Structure learning of dynamic system in complex environment mainly focuses on the cognition of complex dynamic system by means of non-parametric inference and learning in the framework of non-parametric Bayes. Bayesian non-parametric analysis is the most cutting-edge topics in the field of machine learning. It can automatically infer an adequate model size and complexity from data without too much knowledge on the underlying populations of research objects. It is regarded as the most powerful tool to complex dynamic system structure learning with wide applications in the fields of audio signal processing, image processing, text semantic analysis, biomedical signal processing and so on.The research on non-parametric Bayes mostly focuses on the complex and random object. Its data set may either come from a complex random measure distribution, or from a complex random function. The uncertainty propagation of complex random object would be far more complex than of parameterized distribution in parametric Bayes. Although the Bayesian non-parametric framework defines a priori for such a potential process, which helps researchers handle the data set under the general Bayesian framework, the traditional system analysis may not be efficient. MCMC provides a general approximation to the posterior process. In many contexts, how to formulate an effective inference algorithm of MCMC in infinite dimension is a difficulty for complex and dynamic system structure learning. Aiming to solve this problem, the research work is divided into the following three parts in this dissertation:The first section investigates on how to extract useful signal in a linear dynamic systems with complex noise. Firstly, a fresh method of non-parametric Bayes block sampling and inference is proposed to identify the multi-model distribution of randomly complicate noise. The main idea of proposed method is: In the temporal sampling, the samples can be updated at one time under the principle of conditional independence, which is usually simple and more effective than the separate update, and approximate the posterior distribution of noise. Based on the sound identification of noise, it is further analyzed on the effect of such noise model on extracting useful signal in the scenario of high dimension. With the Chinese restaurant process mixture model (Chinese Restaurant Process Mixture Model, referred to as CRPMM), it is more convenient to have access to Markov chain samples. With the combination of CRPMM and Kalman filtering, a higher accuracy can be achieved as well as approximation on the space of distribution. The simulation results and the experiment result of Dirichlet speech enhancement show the efficiency of the proposed algorithms.The second section resolves the non-parametric regression and prediction in complex dynamic system. Gaussian process model based on a mixture of Dirichlet process is proposed to reveal the intrinsic mechanism of multi-model of complex dynamic system architecture data. As for the difference between the mean structure and covariance structure of sparsity, parametric a priori and non-parametric a priori are designed based on the hybrid sampling framework of Polya urn sampling and over-relaxed sliced sampling. The hybrid sampling will not only be implemented under the unified Metropolis-Hasting probability evaluation criteria , but also be able to overcome the shortcomings of Gaussian random walk, Markov chain samples can be quickly and easily got started. The effect of the algorithm is validated by the simulation and the experiment result of interpolation /extrapolation prediction of the ECGs.The third section main deals with methods of extracting information on completely exchangeable text. The graph model representation of Latent Dirichlet allocation (Latent Dirichlet Allocation, LDA) model is provided; Bayesian information criteria (Bayesian Information Criterion, BIC) of model selection is set up on the number of theme of exchangeable text; Metropolis-Hastings algorithm is also provided, which is a sampling based on the independence; In the end, the compromise between the convergence of algorithms and complexity of BIC is discussed as for its effect on the convergence. The real data set experiment of the texts shows that the proposed algorithm is efficient.The main work in the dissertation is described as follows:1) A non-parametric algorithm is proposed to identify the noise based on block sampling. This method can approximate the noise distribution directly on the space of distribution, and therefore more efficient than the point estimation ones on the space of Gaussian distribution. The sampling based on conditional independence instead of the correlation improves convergence speed and has wide adaptivity in the identifying non-sparse noise. 2) The mixture of Gaussian process model with finite dimension is extended to a model of infinite dimension and non-parametric regression analysis under DP+GP model can avoid deriving iterative formulation of fixed point.3) The scheme of hybrid sampling can overcome the shortcomings of Gaussian random walks in the non-parametric regression and prediction analysis.4) The M-H algorithm based on independent sampling according to BIC can identify the number of topics within the exchangeable data set, and avoids setting a number in advance in the training datasets in LDA model.This work was supported by the ShangHai Science Foundation under grant No.05JC14026.