Research On Optimization Of Big Data Query For Oilfield Technological Project Management

Daqing Oilfield Technological Project Management Platform was put into service in2009,this platform realizes a total life cycle management of technological project audit.With the changes of the times,the data of oilfield technological project increase year by year,the huge amount of data add a lot of complicated work.In order to avoid duplication of project and manage technological project efficiently,how to distinguish huge amounts of similar data information quickly and accurately is the primary problem to be solved.Therefore,this thesis intends to introduce the related concepts of big data,build a big data runtime environment and research on the big data query optimization of oilfield technological project by applying relevant big data techniques,in order to improve the efficiency of oilfield project management at last.This thesis is based on the research background and practical demand.First of all,construct the big data environment of Hadoop,aim at the data types and data sources of oilfield technological project,deploy the research and construct on storage structure,provide a strategy for loading and storing data from different data sources,as well as research on the structured data of various types of data,design a NoSQL database to support ETL processing to the data after storage.Secondly,on this basis,establish ETL architecture based on the Hadoop platform to implement a more rational data ETL workflow and provide assurance for the next query optimization.Finally,for the query of oilfield technological project,Combined with the drawbacks of the current big data query method,analysis and research the underlying operating mechanism and process,carry out the optimization design to improve the efficiency of the big data query system,the management efficiency problems of oilfield technological projects on mass data is solved better.This thesis aims at the actual needs and development of oilfield technological project platform,import the storage structure and query methods based on the big data environment to the oilfield technological project,and develop a corresponding system to verify its feasibility,provide theoretical support and example verification to the realization of thesis,to a certain extent,this thesis has certain theoretical significance and reference value to the oilfield technological project management with growing quantity of the data.