Design And Implementation Of Load Forecasting System For Power Dispatching Based On Spark

The recent development and construction of the technologies concerning smart grid and Ubiquitous Electric Internet of Things(UEIOT)have transformed the field of power systems and big data into a brand-new configuration.Therefore,it is of utter importance now for workers in the industry to contrive more efficient ways to deal with high-dimensional data.The current post-graduate project aimed to improve the method for mining,reading,and analysing the data collected and stored by SCADA and EMS systems by filing such data in the Hadoop distributed data storage system and presenting it on HDFS clusters using Spark.The current design intended to use various power dispatching big data analytics software to achieve the following two functions:1.In response to the ever more complex,diverse,and massive power system operating data,the current design proposed to advance the use of “regular expressions”to facilitate searches by keywords.Furthermore,the design used the “MapReduce”filing technique to further filter and categorise the multisource heterogeneous data into more coherent and well-regulated formats.Both measures should considerably expedite the check-up and maintenance process of power system machinery.2.In order to produce high-precision,low-latency forecasts of power load in the current power systems and big data fabric,the design proposed to enhance the calculation method by using Spark Random Forest Regression(SP-RFR)and three Resilient Distributed Datasets(RDD)transformations,before laying the data on Spark distributed clusters.The current design was evaluated in an experimental trial using real-life power system big data from Meishan with comprehensive software testing,model training,and regression prediction.The empirical evidence suggested that the design was able to sift,categorise,and count different data types such as grid operation monitoring telemetry,remote indicator transmissions,and protection alerts,and present them on a user-friendly interface,which considerably assisted the workers with their routine check-ups and maintenance of the machinery.Regarding the same category datasets,the calculation precision using SP-RFR was significantly higher than the calculation precision using regular stand-alone power load forecast algorisms.Moreover,compared to stand-alone algorisms,SP-PFR was more robust to outliers,and showed significantly higher time-efficiency,especially when larger datasets were involved.