Study On Dynamic Parallel Computing Based On Probabilistic Rough Set And Its Application In Spark Platform

Stream computing is a method of data calculation in dynamic environment.According to the flow direction of dynamic data,stream computing can be divided into three methods: incremental,decremental and incremental or decremental in parallel.In addition,the parallel method can effectively improve the processing efficiency of dynamic data and the speed of knowledge updating.Spark is a typical parallel computing framework,which can effectively handle the computation of complex data.The research and application of parallel computing has become a new hot issue.The three-way decisions making theory of probabilistic rough set is a tool to deal with uncertain problems.It can discover hidden knowledge and reveal potential rules by mathematical calculation and reasoning of data.In recent years,three-way decisions have made notable achievements in both theoretical research and practical application.Dynamic decision making in three-way decisions making under dynamic environment is the key issue similarly.However,the current research in this direction is mainly based on incremental learning algorithm,and research of algorithms based on parallel computing method are less.The main work is as follows: Firstly,in the dynamic decision environment,we study the dynamic increased and decreased mechanism of one single data object in information system,and based on the thought of time division multiplexing,a dynamic increased and decreased parallel computing algorithm based on three-way decisions is proposed.Experiments resluts on the Spark platform show that comparing to the classical computing algorithm,the dynamic increased and decreased parallel computing algorithm has more effectiveness;Secondly,since the dynamic increased and decreased parallel computing algorithm is actually pseudo-parallel,we make some improvement on the basis of this research: By means of real time memory computing,a single object dynamic memory based on sliding window is given,which reflects the flow calculation model of this kind of data change mode of unified modeling,and according to the above dynamic model,the single data object is increased and decreased parallelly,so that the condition probability and three regions of three-way decisions are deduced quickly,finally a novel dynamic parallel computing algorithm based on three-way decisions is proposed;Experiments resluts on the Spark platform show that comparing to the classical computing algorithm,the dynamic parallel computing algorithm has better performance and stability.