Research On The Real-time Data Management Of The IOT For The Production Of Oil And Gas

With the deep application of the Internet of things technology in oil and gas production, the data obtained after inspection, measurement, analysis and calculation through the meter fixed at the well site is uploaded from RTU to well control SCADA system so as to realize data display, remote monitoring and management in the oil and gas production process. In order to manage and organize the real-time data stored in real-time database as well as historical database, the labeled storage industrial real-time data should be converted to the business data suitable for analysis, which can be used to realize the alarm and warning decisions to support the query and the second decision. An open architecture between Application service layer and production data acquisition——Real-time data processing system is designed in this paper for the classification of the real-time data processing and management process.Firstly, Using the Organization, distribution, production units classification of real-time data in the internet of things for the production of oil and gas as a starting point, the Real-time data business functions and the division of system module are studied based on in-depth consideration of the data acquisition, transmission, storage and storage standards, meanwhile, the deficiencies in the process of real-time data process including sensitive data volume and high cost of storage in the process of storage, relatively fixed frequency in data acquisition and application, as well as the data’s difficult to recovery after lost are all analyzed.Secondly, the properties including benchmarks, load and durative for real-time data processing of present multithread system are all tested. The health degree, maximum response time, minimum response time, average response time and response error code of single well test page on main functional areas of load test are selectively analyzed. The experimental results show that the response speed of the system reduces with the increase of users on account of the complexity of the collection and processing as well as the real-time data processing method in traditional multithreading system. To solve this problem,a set of real-time data processing and conversion system based on Hadoop platform is designed.Finally, on the basis of comparing several algorithms for ensuring data integrity,the MD5 algorithm is adopted for real-time data grouping in consideration of data integrality requirements in the Iot system of oil and gas, the processing speed of real time data is speed up and the integrity of historical data ensured through this method,thus,a complete,quick and accurate data transmission from designed system to the upper layer can be realized.