Research On Parameter Learning Of Bayesian Networks From Small Data Sets

A Bayesian network(BN)is typically a type of probabilistic model that combines probability theory and graphical model theory.BN has drawn more and more attention in AI domain.As a probabilistic model,BN enables concise representation of joint probability.BN has been applied to deal with issues,including disease diagnosis,gene analysis,fault diagnosis,intelligent control,signal processing,audio identification,video identification,target recognition,target tracking,earthquake prediction,parole assessment,educational measurement,network security,and ecosystem modeling.Data mining is one of the main approaches to construct BNs.However,collecting a large amount of data is difficult for some decision-making problems,such as earthquake prediction,parole assessment,rare disease diagnosis and amputation decision.Therefore,learning BNs from small data sets is a topic that attracts wide attention worldwide.In this thesis,we focus on parameter learning of BNs from small data sets.With analysis of exisiting parameter learning methods,such as qualitative-constraint-based learning methods,model-based methods and other methods,we extend existing research on BN parameter learning,especially learning with qualitative parameter constraints.The main work is summarized as follows:(1)An adaptive BN parameter learning algorithm has been proposed.Constrained maximum likelihood(CML)and qualitative maximum a posterior(QMAP)algorithms are two advanced approaches,which suit all types of existing parameter constraints.However,those two approaches dominate each other when sample size,constraint number and true-parameter location varies.That makes it tough to choose between those two algorithms.To solve the above problem,first,CML and QMAP algorithms are used to learn BN parameters.Then,sample weight,constraint weight,and parameter-location weight are defined and calculated by rejection-acceptance sampling and spatial maximum a posterior analysis.Finally,new parameters are computed as the weighted values of CML and QMAP solutions and an adaptive BN parameter learning algorithm is proposed.(2)A spatially maximum a posteriori(SMAP)estimation algorithm has been proposed.By now,most of the existing parameter learning algorithms take parameter learning problem as an exact optimization problem and regard the optimal solutions as the final parameters.However,due to the scarcity of data,objective functions constructed from the data,such as likelihood function and entropy function,are not accurate.Therefore,parameters computed from the objective functions do not approach the true parameters well.To solve the above problem,we visualize possible parameters with high-dimensional parallel coordinate system,define a new type of interval constraint and propose a spatially maximum a posteriori algorithm.(3)A further constrained qualitatively maximum a posteriori(FC-QMAP)estimation algorithm has been proposed.MAP(Maximum a Posteriori)estimation that utilizes both sample data and domain knowledge has been well studied in the literatures.Among all the MAP-based algorithms,the QMAP(Qualitatively Maximum a Posteriori)algorithm exhibits the best learning performance.However,when the data is insufficient,the estimation given by the QMAP often fails to satisfy all the convex parameter constraints,and this has made the overall QMAP estimation unreliable.To solve the above problem and to further improve the learning accuracy,we regulates QMAP estimation by replacing data estimation with a further constrained estimation via convex optimization and propose a further constrained qualitatively maximum a posteriori algorithm.(4)A constrained maximum a posteriori(CMAP)estimation algorithm has been proposed.Equivalent sample size(ESS)is a key factor in maximum a posteriori estimation.In reality,it is intractable for the domain experts to define ESS values.If the ESS value is too small,the data dominates the posteriori estimation and the posteriori estimation is often biased when the data is insufficient.If the ESS value is too large,the prior dominates the posteriori estimation and information contained in data might be overlook.To solve the above problem,we derive ESS constraints from the parameter constraints,determine the optimal ESS values by cross-validation and propose a constrained maximum a posteriori estimation algorithm.(5)In modern battlefield,it is tough for UAVs(Unmanned Aerial Vehicles)get enough target data and therefore threat assessment based on insufficient data is not reliable.To solve the above problem,by combing military expert knowledge and data,manned-unmmanned aerial vehicle threat assessment model is constructed using the approaches proposed in this thesis.The experiments show that expert knowledge has been well incorporated into the threat assessment model and the performance of the model has been significantly improved.