| Rough Set theory has a wide range of application in dealing with fuzzy classifi-cation, attribute reduction and other basic issues of data mining. Rough Set Attribute Reduction is not very effective actually, when facing with information explosion, the quick and efficient processing of large-scale data necessary for every Data Mining algo-rithm. How to improve the efficiency of the reduction method is the mostly considered of in this paper.This paper mainly proposes a parallelling rough set attribute reduction method based on attribute significance to improve the efficiency, and designs a data preprocess-ing method based on canonical correlation analysis to further improve the efficiency of the algorithm and reduction effect. The main contents are as follows:1. This paper proposes a feature fusion method CCAFusion based on canoni-cal correlation analysis. First, splitting a collection of the attributes set into multiple sub-sets of attributes, regarding each sub-set as a sub-view of the origin object; Then, adopting canonical correlation to analyze these sub-sets to get the feature correlation information between each view; Finally, fusing the features of each view into one view according to the correlation. Experimental results show that this method can reduce feature dimension effectively and improve the classification accuracy of data.2. This paper proposes a parallelized rough set attribute reduction algorithm P -RoughReduction. First, designing a rough set attribute reduction algorithm based on attribute significance according to rough set theory; then finding out that calculating attribute importance is actually an independent sequence, which meets the condition for parallel computing. Giving a parallel implementation of the rough set reduction algorithm in MapReduce distributed programming framework. The experimental re-sults show that in Hadoop cluster, the efficiency of the proposed parallelized reduction algorithm improved Significantly.This paper merges CCAFusion algorithm and P-RoughReduction algorithms together, presenting a mixed attribute reduction method P-CCARoughReduction. The experiment result shows that this hybrid algorithm can not only have good opera-tional efficiency and improving the reduction effect significantly.Finally, this algorithm is applied to massive data mining toolbox Dodo in data preprocessing module. |
