Research On Fast Query And Compression Processing Of Monitoring Information Of Electrified Railway Power Supply System Based On Columnar Storage

With the rapid development of electrified railway and high-speed rail in recent years,the level of automation and intelligence has been continuously improved.The dispatching and monitoring system also has been upgrading the comprehensive automation system to ensure the stable operation of railway power supply system.This makes the capacity and scale of power supply monitoring system expands unceasingly,especially to the monitoring system acquisition terminal.Moreover,the departure frequency of the high speed EMU increases and the running speed is faster.Normal operation of traction power monitoring equipment become more strict.At the same time,the sampling frequency of power supply monitoring system is further improved,make monitoring information explode.With the fault recording device as an example,the data acquisition rate is 2.008MB/s,take 2MB/s for calculation,1181.25 G data will be collected weekly,the amount of data will be about 60.15 TB annually.What's more,there are a large number of information acquisition equipment in railway power supply dispatching and monitoring system,and the service period is very long.Facing the mass monitoring information,the existing processing model of monitoring information generally use the disk array and the relational database.These ways have weakness such as the limited storage capacity,the poor scalability and the low process efficiency,etc.Without effective treatment of mass monitoring information,may lead to the interactive difficulty of monitoring information,communication congestion.If there are delay,omission or misstatement of some key information,even lead to scheduling errors and cause an accident.Because the processing bottleneck of row-oriented relational database for big data,academia carry out the research about data processing technology based on columnar storage.What is more,the compression performance and query efficiency of columnar processing are verified in theory and some application fields,HBase cluster based on columnar storage has been applied in some key areas particularly.Therefore,aiming at the efficient processing of mass monitoring information in railway power supply dispatching and monitoring system,focusing on the research of optimization techniques based on HBase columnar cluster in this paper.Then the performance of compression and query for monitoring information based on Brighthouse are analyzed and verified by experiments.Then,a large compression ratio of monitoring information is realized based on the Brighthouse column storage engine.Based on theoretical research and engineering practice,this paper first integratesmonitoring system by design pattern,for solving the difficult of information integration and monitoring equipment compatibility.Then the power supply dispatching and monitoring information processing platform based on HBase cloud cluster is built successfully,and integrates Phoenix into the cluster for parallel query.Implementation secondary indexes based on the coprocessor for query optimization is realized,through analysis and experiment,the query speed of monitoring information can be improved greatly by secondary index.Take the monitoring information of the electrified railway as object,to research on the distributed lossless data compression based on HBase cluster.Combination of compression and secondary index is proposed further,comprehensive optimization of monitoring information access is verified by using the optimization model.The large compression ratio storage model of monitoring information based on Brighthouse is accomplished,and the processing properties are verified by example data of engineering.Research results have provided a new scheme for efficient compression and fast query processing of mass monitoring information in railway power supply monitoring system.