Parallel And Distributed Mining Algorithms On Simulation Data

Simulation experiments generate massive simulation data, in order to extract useful information and knowledge, understand and improve the system better, help the deciders make decisions, can use data mining method in simulation data analysis. Because the simulation data has high dimensions and big scale, parallel mining algorithm is needed to improve efficiency. And also these data is stored in distribute place, centralizing the data will be very expansive and may be not safe, distributed mining algorithm should be considered. The context of this paper is given as fallow.Due to the instant and distribute attribute of simulation data, the need of parallel and distributed data mining is proposed. According to the basic process of data mining, we conclude the common process of mining simulation data. Find the rules and optimism is two important needs in simulation, there are two favorite mining method correspondingly, association rules and decision tree. They can find the rules in the system and optimize the simulation system.We use association rule method to find rules in simulation experiment. We make some improvement in Apriori algorithm. First, use the vector data structure, reduce the space of the input dataset, so it can be put into the main memory and avoid the overhead caused by scanning database many times, the efficiency is improved. Second, use vector container instead of hash tree to save candidate set, reduce the space complexity of algorithm. Consider the large scale of simulation data, we use the same parallel formulation of CD (Count Distribute) algorithm to make the algorithm paralleled, and design experiments to test the scalability of our algorithm. Consider the need for simulation optimize, we use decision tree method.Owning to the distribution of the simulation data, we first study on the meta-learning based distributed classifier, then compare two parallel formulations of constructing decision tree, called synchronous tree construction approach and partitioned tree construction. We implement the synchronous tree construction approach based ID3 algorithm, and use the simulation instance to verify the algorithm.