Research On The Application Of Data Processing Technology In Smart Grid

With continuous development and deep-going implement of smart grid construction, accumulated grid data is growing exponentially. How to find out “treasure” from these historical data has become an urgent requirement in the process of construction of strong smart grid. The technology of big data’s storage and processing brings new opportunities to smart grid data mining. It can be a great solution to the problem which is caused by bid data mining with the advantages of excellent clustering feature of Hadoop platform, powerful computing ability of MapReduce and the storage capacity of HDFS.It has designed and implemented an association rules mining system based on Hadoop platform with combination of Hadoop platform and association rule mining technology. According to the actual needs of the grid data, this thesis takes the use of K-Means clustering algorithm to do the discrete processing of continuous data, and then employs the improved FP-Growth algorithm in mining association. This thesis mainly solves three problems below: Firstly, based on MapReduce, it proposes an improved parallel FP-Growth algorithm which introduces Matrix Storage affairs,reduces the number of database scanning, and saves memory space. Meanwhile, it designs IDFPTree data structure to reduce memory space in runtime and improves the efficiency of the algorithm. Secondly, it implements the K-Means algorithm and FP-Growth algorithm which is based on MapReduce framework, and transplants them to the Hadoop platform. Thirdly, it constructs a smart grid data association rule mining system based on MapReduce programming framework by using B / S architecture. The client is a browser which is responsible for the interaction between the user and the system. The server is built on Hadoop platform which is responsible for the tasks raised by users with distributed computing.This thesis applies the designed system into analyzing marketing data in power enterprises. It finds out the strong association rule between the classification properties of the users(user properties, market properties, etc.) and decision properties(bucketed electricity consumption). Thus, it reveals the law of electricity sales which can not be found in traditional ways,and it is significant to the power marketing analysis.