Research And Application Of Adaptive Decision Of Directional Drilling System Based On Reinforcement Learning

Directional drilling technology is a comprehensive technology to adjust the well trajectory in real time according to the downhole real-time measurement data.It is widely used in the petroleum industry.The traditional directional drilling tool face control method is labor-intensive and time-consuming work,and largely depends on the experience of drilling personnel.Therefore,the research on the intellectualization of directional drilling is of great significance,but most of the existing technical routes depend on the establishment of the model and the accuracy of the correlation coefficient parameters,which has poor adaptability in the complex downhole environment of directional drilling system.Aiming at the problem that traditional methods can not realize cross well adaptive decision-making,thesis studies a reinforcement learning adaptive decision-making algorithm suitable for directional drilling platform from the perspective of data-driven modeling of directional drilling system,and carries out field directional experiment and analysis.The main work of thesis includes:(1)Aiming at the problem that the traditional drilling dynamic model is greatly affected by geology and well location and depends on the adjustment of model parameters,thesis adopts the data-driven modeling method to build the directional drilling model.Firstly,combined with the structural characteristics of the directional system,the directional drilling model based on the multi head attention mechanism LSTM is designed,and then the model is trained,optimized and evaluated based on the historical directional data to verify the effectiveness of the model.Finally,on the basis of it,a complete reinforcement learning environment is developed to facilitate the subsequent algorithm research;(2)Aiming at the problem that different downhole environments lead to the change of orientation strategy in the process of directional drilling,an adaptive decision algorithm based on reinforcement learning is designed in thesis.Firstly,according to the analysis of reinforcement learning algorithm,the benchmark algorithm suitable for orientation problem is selected,and then the state action space is designed and optimized.On this basis,the reward mechanism embedded with expert operation experience and the experience playback algorithm based on sample priority are designed,to improve the online learning efficiency of decision algorithm;Finally,the effectiveness of the improved algorithm is verified by simulation experiments;(3)Aiming at the problem that it is difficult to combine the theory and practical application of decision algorithm,thesis integrates and encapsulates the decision algorithm,realizes the rapid deployment of the algorithm,and is convenient for field application.Firstly,through the migration of the model,the algorithm has basic directional decision-making intelligence,and then build a data remote real-time interaction module.Through the complete integration and packaging of the algorithm,the functions of multi-platform rapid deployment and online learning are realized.Finally,the field automatic orientation experiment is carried out to verify the effectiveness and practicability of the algorithm proposed in thesis.The results show that the decision algorithm proposed in thesis not only achieves better results in simulation and comparison experiments,but also completes the goal of directional decision-making in multiple tests based on real drilling platform,and has the ability of adaptive decision-making of directional drilling.