Intelligent optimization of horizontal wellbore trajectory based on reinforcement learning

Ultra-long horizontal wells are crucial for improving the production and developmental benefits of shale oil and gas wells. Currently, advanced downhole semi-closed-loop steering drilling technology depends on a dual communication mechanism between the surface and downhole, as well as the empirical decisions of human experts. However, it is difficult to acutely control the trajectory of ultra-long horizontal open hole drilling in a reservoir owing to geological uncertainties related to shale oil and gas, the uncertainty of the drilling tool's building capacity, and the lag of measurement information while drilling. This report proposes an adaptive target detection and design method for ultra-long horizontal well trajectories based on reinforcement learning. The developed approach enables dynamic identification of reservoir sequences according to logging-while-drilling data and prediction of horizontal targets in real-time, with a recognition accuracy of 90.1%. Additionally, measurement-while-drilling data are used to accurately characterize the depth, inclination, and azimuth of the target-entering process. The dynamic interaction mechanism between the bit and the target environment is simulated by defining the bit action, reward function, and an update mechanism. The drilling engineering conditions restrict the interactions, such that the bit automatically decides the direction in which to drill the targets, demonstrating 100% accuracy for the wellbore trajectory toward the target. The experimental application results indicate that the developed method can be applied to determine the dynamic target area of a horizontal well in real time and make intelligent downhole decisions regarding ultra-long horizontal section borehole trajectory adjustments while drilling and measuring. The drilling rate of high-quality reservoirs is therefore significantly improved. The results discussed herein provide insights to support further developments of downhole full-closed-loop intelligent autonomous decision-making borehole trajectory control.