An Auxiliary Detection System For Long-distance Pipeline Leakage Based On Working Condition Analysis

The pipeline leakage monitoring at the front end of the system mainly adopts the acoustic wave method and the negative pressure wave method,etc.The main disadvantage of these methods is that they are unable to identify various complicated working conditions,resulting in many false positives.In addition,due to the rapid growth of data volume of long-distance pipeline,the traditional way of building a system based on Oracle,SQL Server,MySQL and other relational databases on a single Server will soon lead to a sharp decline in system performance.Therefore,the construction of an auxiliary detection system for working condition analysis and identification of leakage based on big data can effectively solve the problems of the original system.Based on CDH and the rapid growth of data volume in long-distance pipelines.A big data platform is built under the existing resources of the laboratory.The purpose is to solve the problem that the system performance cannot be satisfied by the system optimization and expansion for the relational database due to the continuous increase of the system runninig time and data.Reasonable allocation and management of components in the existing big data architecture ensures the security,maintainability and scalability of real-time detection system performance in the future.Based on Spark Streaming,Flume,Kafka and HBase build a real-time data processing framework.From the analysis and preprocessing of source data,the process of simulating real-time data transmission,the real-time data collection by Flume and the data buffering of Kafka,the data processing of Spark Streaming and the storage of data in HBase,the whole process complete real-time processing of data under the big data platformIn this paper,through the research and processing of industrial field data collection,the transformation of time series characteristics to frequency characteristics of stress is completed.Through the research and experiment of existing machine learning classification algorithm,combined with the characteristics of Spark distributed computing,we find the right present.There is a classification algorithm with good training results in offline datasets.The online and offline learning methods are combined to load the offline learning model,and the real-time data is classified and predicted.Improve the accuracy of model predictions by regularly updating offline datasets and reach the goal of online learningThis article establishes the Spring Boot web project through IDEA,and builds a front-end pressure warning interface based on ECharts' rich front-end model library.The results predicted by Spark Streaming in HBase are reflected in the front end in real time.It is used to provide leak detection for the production environment to distinguish between false alarms caused by working conditions.