Research On Key Technologies Of Multi-source Heterogeneous Data Fusion And Processing For Intelligent Distribution Network

Faced with the trend of more complex data types and sharp increase in the number of data in distribution networks,sub-topic 4 of "Research and Application of Wide Area Measurement and Control Technology in Power Distribution Network" in the national key research and development plan requires the information integration of massive multi-source heterogeneous data of distribution network,the construction of panoramic information model of multi-source heterogeneous data of distribution network,and construction of multi-source heterogeneous data fusion and processing platform for intelligent distribution network.Based on this background,the main contents of this paper are as follows:(1)Using multithreading,database batch processing and other technologies,a user-configurable data rectification mechanism is designed and implemented to help users preprocess multi-source heterogeneous data of distribution network according to their specific needs,thus solving the problem of many types of dirty data in multi-source heterogeneous data.(2)Based on the data characteristics of high jitter frequency of PMU data and strong complementarity between PMU data and SCADA data in power distribution network,a PMU-SCADA data time proofreadingand and fusion method based on Lagrange interpolation and exponential smoothing is proposed,which solves the problem of inconsistent time scales of data fusion between PMU and SCADA data in power distribution network.(3)Based on the specific requirements of PMU data query by external applications of power distribution network and the characteristics of PMU high-frequency streaming data,a pre-access caching method based on distributed storage system is proposed,and a prototype of pre-access caching mechanism based on distributed storage system is developed by using MSQL database and Mycat database middleware,which solves the problem that the performance of providing external data services cannot meet the real-time performance due to the influx of massive data for multi-source heterogeneous data fusion and processing platform oriented to intelligent power distribution network.A large number of simulation tests show that the configurable correction mechanism proposed in this paper can accurately process abnormal data in distribution network according to user-configured correction rules.The PMU-SCADA data time proofreadingand and fusion method proposed in this paper can better realize the time proofreadingand and fusion of PMU-SCADA at the same measurement point,and the error of time proofreadingand is less than 100 milliseconds.This paper proposes a pre-access cache method based on distributed storage system,which can query PMU data under the condition of time range query,and can meet the performance index(query 34500 PMU data)of querying PMU data for 1 second in 100 milliseconds.