Research On Massive Power Quality Data Cloud Computing Platform Based On Hadoop

With the growth of science and technology development and the national economy, increasing demand for electricity, while the increasingly high demand for power quality. In order to improve power quality, sampling frequency of data acquisition system for power quality needs to significantly improve and the scope needs to greatly expand. It will emerge a massive power quality monitoring data. Currently, power quality analysis and calculation of power system generally relies on a centralized computing platform located in dispatch center. For large-scale power quality data, the calculation speed is slow, the efficiency of task execution is low and the analysis result has a serious lag. The data management based on traditional relational database cannot meet their requirements, or need to pay a high cost. How to process massive monitoring data reliably and quickly becomes an important issue in the analysis of power quality.This thesis is based on the existing power quality analysis platform. Taking advantage of the mass data storage and processing of Hadoop, to carry out the design and implementation of massive power quality data cloud computing platform based on Hadoop. In data storage, this thesis designed and implemented a scalable and highly reliable large-scale data storage platform based on HBase, also combined each characteristic of HBase and SQL Server to store different types of data. It not only enhanced the platform performance, but also the platform can be more easily integrated into existing power quality monitoring system. In the data processing, this thesis designed and implemented the power quality analysis module based on MapReduce and put forward some parameters optimization method for the MapReduce job to run operations. It enhanced the analysis process concurrency and improved the efficiency of calculation and analysis of power quality. In addition, the platform set up a hot standby node for high availability to the single point of failure in HDFS.In the experimental section, compared to the traditional power quality analysis and processing, this thesis calculated steady-state harmonic power quality indicators for different time periods to verify that the correctness and superiority of the power quality analysis platform based on Hadoop.