Parallel Processing Of Power Monitoring Big Data Under Spark Platform And Cross-platform Migration Of Models

With the rapid advancement of smart grid construction,more and more monitoring data of intelligent monitoring equipment are sent to the monitoring center.Especially in extreme weather,some power equipment frequently send alarm data to the monitoring center,causing the monitoring data blowout phenomenon Appeared.Aiming at the problems that traditional single-machine processing methods can no longer perform efficient,fast and real-time processing of massive power equipment monitoring data,this paper studies the Spark-based parallel processing method of power equipment big data,and at the same time studies the fault diagnosis model from Spark to Storm platform.Migration method.A parallel method of partial discharge pattern recognition based on VPMCD under the Spark computing framework is designed.In this paper,we extract the PRPD features of the φ-q-n spectrum from the original discharge signal to form the relevant feature vector as the experimental input.Then use the parallelized VPMCD algorithm to classify the discharge types.Experimental results and analysis show that the recognition accuracy of the variable prediction model pattern recognition method is higher than that of the SVM and BP neural network models.The calculation efficiency under the Spark calculation framework is better than that under the traditional stand-alone environment,which can meet the requirements of big data in the smart grid.Fast processing requirements.A parallel relevance vector machine multi-classifier for partial discharge data classification under the Spark computing framework is designed.Because the partial discharge data is a variety of categories of data,and the correlation vector machine is a two-classification algorithm,so the multi-classification problem of the correlation vector machine is studied.In the multi-classification design,the one-to-one method with the best classification accuracy is selected and the particles are used.The group algorithm optimizes the relevant parameters.Then based on the Spark platform,the one-to-one multi-classification model of the correlation vector machine is implemented,and the data set is compared with the single machine and the parallel environment.The experimental results show that the parallel processing improves the data processing efficiency.The migration method of the VPMCD model from Spark to Storm platform is proposed,and the model markup language(PMML)file is designed.With this file,the VPMCD model trained on the Spark platform can be exported to Alluxio,and then be tested on the Storm platform.Classification of the sample.