Research And Application Of Feature Selection Algorithm Based On Information Measure

The dimension of datasets' features increases dramatically,and it not only leads to larger amount of computation,but also may cause “the curse of dimensionality”.Therefore,it is necessary to reduce the dimension of feature set.Feature selection is a great way of dimensionality reduction and the metrics include information,distance and so on.Research on feature selection algorithms based on information meaure is conducted and some algorithms are applied to steganalysis for validating the application.The major contributions and innovations are as follows:(1)To solve the problem that the algorithms based on relevance and redundancy cannot guarantee selecting the feature with maximal relevance minimal redundancy,a feature ranking selection algorithm based on equal interval division and minimal-redundancy-maximal-relevance is proposed.The proposed algorithm first adopts mutual information between features and the class label to measure relevance,and selects the feature with maximum relevance.Then it adopts mutual information between features and the class label to measure relevance,and exploits mutual information between features to measure redundancy and combines the method of equal interval division and ranking to select features.The proposed algorithms can have the priority to select the feature with maximal relevance minimal redundancy and achieve better feature selection effectiveness.(2)For solving the problem that several feature ranking selection algorithms based on three-dimensional mutual information do not consider three-dimensional mutual information among features,a feature ranking selection algorithm based on equal interval division and conditional mutual information is proposed.The proposed algorithm first calculates mutual information between features and the class label,and selects the feature with maximum.Then it calcluates conditional mutual information among features and the class label,and selects the feature with maximum.Following that,it calculates conditional mutual information among the class label and features as well as conditional mutual information among features,and combines the method of equal interval division and ranking to select features.The proposed algorithm considers conditional mutual information among features,and the selected features can obtain better classification accuracy.(3)For the problem that some relevant features are considered as redundant and removed in several feature subset selection algorithms,a feature subset selection algorithm based on equal interval division and three-way interaction information is proposed.The proposed algorithm first employs symmetric uncertainty between the class label and features to measure relevance and remove irrelevant features.Then,it adopts two measures,three-way interaction information and symmetric uncertainty,and the method of equal interval division and ranking to measure redundancy,and removes redundant features.The proposed algorithm reduces the situation that relevant features are considered as redundant and removed,and improves feature selection performance.(4)To solve the problem that several steganalytic feature selection algorithms have heavy computation,feature selection algorithms based on feature module are proposed.Feature module is made up of many features,and mutual information and classification accuracy of SVM classifier are employed to select modules.The proposed algorithms are applied to steganalysis and exploited to handle spatial-domain steganalytic features.The proposed algorithms have the priority to select more features with synergy and reduce the computation,and they can achieve better feature selection effectiveness in case of less time-consuming.