Research On The Application Of Hadoop Technology In The IOT About Oil And Gas Production

In the oil exploration industry,with the constant application of Internet of Things(Io T)technology,a large amount of data is generated in the oil and gas development and business environment.With the decrease of collection cost of data and the increase of collection consciousness of data,the oil field has accumulated massive and multi-dimensional oil and gas production and management data.But the current "massive data" and relative "poor awareness of oilfield" has become a bottleneck for the development of oil industry.It is enormous of volume and numerous of types that the unstructured data of oil and gas production,mainly contains a variety basic business data without fixed format,documents of production results,production reports and the "four construction"(modular construction,standardized design,promoted informatization,standardized procurement,)and other data,which size is from 6TB to 8TB.The current amount of oil and gas production data is growing,beyond the traditional database storage range,the regular database(such as: My SQL,SQL Server,DB2,etc.)storage and the methods of data processing encountered bottlenecks.Hadoop,as an emerging distributed processing framework,has the characteristics of high reliability,scalability,efficiency and fault tolerance,which provides a new idea for the storage and processing of massive oil and gas production data.Therefore,Hadoop technology is applied to Internet of Things in the production of oil and gas in this paper,and the oil and gas production data storage platform based on Hadoop is designed and deployed.Based on the historical production data,the forecasting model of oil and gas production is improved.The specific works is as follows:First,the research status and technological superiority of Hadoop are analyzed and concluded,the difficulties in the storage of oil and gas production data are clarified,so that the application of Hadoop technology to oil and gas production is determined for reliable storage,efficient query and data mining analysis.Then,aiming at the characteristics of data structure in oil production site,such as complicated data structure,large-scale data and data interconnection,an oil production data storage platform based on Hadoop technology is designed in this paper,different kinds of data sharing in Hadoop and traditional database is realized,which is conducive to the production of data in the future to do in-depth analysis and mining.For the design of unstructured data storage,Hbase based on HDFS is used as storage database.For the design of structured data storage,Oracle is used as an off-line data warehouse,providing historical analysis of off-line data.While for the production data needed real-time query,the Redis memory database is implemented.On the basis of theoretical design,an oil and gas production data storage platform based on Hadoop is realized,and the performance of the platform is tested,which shows that it is efficient and feasible that applying Hadoop to oil and gas networking.Finally,for the problem of many domestic oil fields entering the stage of decline in production,based on the historical production data stored in Hadoop,an optimum weighted composition prediction model is proposed based on the hyperbolic decreasing model and exponential decreasing model.The prediction results of the three model are compared with the actual yield,and the prediction result of the new model is the closest to the actual production,and the prediction is effective,which can be applied in practice in most of the oil field.