Load Modeling Accurate Data Acquisition Based On Cloud Computing

With the development of economy,increasing demand for electricity and scale-up of network,it meets high requirements for security and stability of power supply.In order to improve operation stability,essentiality for accurate load model has drawn much attention.High-accuracy load data collected by field devices is essential for load modeling.However,sampling rate of disturbance events,the number of monitoring devices,has raise up a question that much data need to be processed.Currently,electric monitoring data generally relies on centralized computing platform located in the dispatch center whose computing forces and storage capability is limited.The data management based on traditional relational database cannot meet requirements or need to pay high cost.How to process massive monitoring data reliably and quickly has become a significance issue in analysis of load modeling.This paper focus on Hadoop ecosystem,which is relied on computing fabric MapReduce and storage structure HDFS.The core of platform is Hive database and HBase is also implemented to realize fast query.Design and implementation of massive power load data cloud computing platform based on Hadoop has been carried out.In data storage,this paper analyze COMTRADE file format of different monitoring devices,design structure of Hive table to process data from different sources.HBase is a NoSQL database and has the characteristics of fault tolerance,scalability and reliability.This paper design HBase table to store COMTRADE files.The designs take advantage of database’s different properties and improve the integration degree of platform.In query optimization,Hive query language(HQL)will be converted to MapReduce job whose process time is long.The Hive tables are mapped to HBase that can take advantage of Hive and HBase respectively.The Hive is designed to store sampling data,which is stored in RDBMS commonly and NoSQL data is stored in HBase.The design reduces time delay and increases efficiency.In power load computation and data conversion,a ETL tool,Sqoop,is employed that can turn ETL into MapReduce job and it eases programming work.A parallel DFT algorithm is employed to reduce computing load in electric parameters extraction.In the experimental section,comparison between Hive and MySQL in query,comparison between Hive and HBase in database loading,comparison between API and MapReduce in database loading are studied.The experiments show that superiority of power load processing platform based on Hadoop is proved.