| Classification is an essential task in data mining and is widely used in many real-world applications.In recent years,researchers try to learn different classifiers for good performance,such as Bayesian methods,decision tree methods,and support vector machine methods,etc.Among them,Bayesian methods have become one of the most popular methods due to its strong expressive ability,good interpretability and high robustness.Bayesian methods are based on probability theory and graph theory,which use Bayesian networks to express the dependencies between attributes,and have a solid mathematical foundation.Bayesian network classifier is a special type of Bayesian network designed for classification.Researchers have proven that learning an optimal Bayesian network classifier is NP-Hard.Therefore,in order to simplify the calculation of joint probability,a commonly used method is to introduce the attribute conditional independence assumption,i.e.,to assume that all attributes are fully independent given the class,and the simplified model is called Naive Bayes(NB).NB is simple,efficient and robust,and is selected as one of the top 10 algorithms for data mining,but the required attribute conditional independence assumption hardly holds true in real-world applications.In order to relax this assumption,researchers have proposed many different improvements,which can be generally divided into three categories:structure-oriented improvements,instance-oriented improvements,and attribute-oriented improvements.Among them,attribute-oriented improvements can be further divided into attribute selection and attribute weighting.In recent years,attribute weighting has attracted extensive attentions from researchers due to its flexibility and effectiveness.However,most of the existing methods only focus on learning a weight for each attribute,which is still relatively coarse-grained in three aspects:(1)They ignore the influence of different instances on the accuracy of probability estimation;(2)They ignore the influence of different attribute values and different class labels on attribute weights;(3)They ignore the limitations of the expert-defined attribute spaces.To solve these three problems,this dissertation aims to improve the classification performance of existing attribute weighting methods,which takes fine-grained attribute weighted NB as the research object and proposes six novel models.Besides,we observe their effectiveness through a series of experiments.The main contents and innovations of this dissertation include:(1)To solve the problem that existing methods ignore the influence of different instances on the accuracy of probability estimation,we unified the influence of instances into the framework of attribute weighting,and proposed three instance and attribute weighted NB models:attribute and instance weighted NB(AIWNB),Collaboratively Weighted NB(CWNB),and Fine Tuned Attribute Weighted NB(FTAWNB).Specifically,AIWNB learns a weight for each attribute and a weight for each instance respectively,which not only relaxes the unrealistic attribute conditional independence assumption,but also obtains more reliable probability estimates.Nonetheless,AIWNB ignores the interaction between attribute and instance.To solve this problem,CWNB embeds the learned instance weights into the optimization process of attribute weighting and collaboratively optimizes their weights to make the learned conditional probabilities optimal.Different from these two methods,FTAWNB does not learn a specific weight for each training instance,but pays more attention to the misclassified training instances,and uses the posterior probability error between the predicted class label and the real class label to iteratively adjust the conditional probability,so as to improve the accuracy of probability estimates as well as the classification performance of the built model.Besides,we also observed the performance of these models in the problem of ozone level classification,land cover classification and eryhemato-squamous disease classification.(2)To solve the problem that existing methods ignore the influence of different attribute values and different class labels on attribute weights,we proposed a model called Class-specific Attribute Value Weighted NB(CAVWNB).Specifically,CAVWNB extends attribute weights from one-dimensional vector to two-dimensional matrix by considering the influence of different attribute values and different class labels,and learns specific weights for each attribute value and each class label respectively.To optimize the two dimensional weight matrix,CAVWNB uses the objective function of maximizing the conditional log-likelihood or minimizing the mean square error.At the same time,in order to avoid the risk of overfitting caused by the increase of parameter number,L2regularization is added to the objective function.By expanding the attribute weights,CAVWNB can learn different weights of different attribute values and different class labels,reflecting data characteristics in a more fine-grained manner,and improves the classification performance of the model.Besides,we also observed the performance of this model in the problem of Connect-4Opening game result prediction.(3)To solve the problem that existing methods ignore the limitations of the expert-defined attribute space,we considered generating some new attributes to expand the expert-defined attribute space,and proposed two attribute expanded-based weighted NB model:Attribute Augmented and Weighted NB(A~2WNB),and Multi-view Attribute Weighted NB(MAWNB).Specifically,A~2WNB constructs the latent attribute space and fuses it with the expert-defined attribute space to perform attribute augmentation.Whilst,MAWNB constructs two label views and combines them with expert-defined attribute views to form multi views.Finally,to avoid attribute redundancy,attribute weighting is conducted in the augmented attribute space and in each view,respectively.Through attribute expansion and attribute weighting,it not only improves the discriminative ability of the expert-defined attribute space,but also avoids the risk of attribute redundancy,and improves the classification performance of the model.Besides,we also observed the performance of these models in the problem of nursery school application result prediction and oil-water layer classification. |
