A Study On 3D Curve Navigation Control Algorithm For Fixed-wing UAV Based On Reinforcement Learning

With the wide application of fixed-wing UAVs,autonomous flight tasks are becoming more and more complex and diversified.The existing navigation control algorithms are mostly adapted to the navigation control of two-dimensional routes,which is more difficult to meet the complex flight tasks.In order to make the fixed-wing UAV have accurate 3D spatial curve navigation control performance and adaptive control capability,a reinforcement learning based3 D spatial curve path tracking navigation control method for fixed-wing UAV is proposed and implemented in this paper,and the main research work is as follows.Firstly,based on the mathematical expression of the curve in the 3D space in the Cartesian coordinate system(in this paper,the conical helix is taken as an example),the expression of the conical helix defined by the Frenet-Serret framework is derived,and the parameters of the helix radius,flight height,rotation matrix,elevation angle,azimuth angle and polar angle of the conical helix are defined in the sphere space.The L1 flight navigation controller is designed on the basis of the set of force equations,the set of moment equations and the set of motion equations of the UAV.Secondly,base on L1 flight navigation control,take the tapered spiral as an example to design the 3D Space Curve based on L1-Navigation,3DSCL1 algorithm: derive the lateral velocity from the lateral plane projection of the space curve,Curvature,lateral plane trajectory error and other parameters,propose and derive the conical spiral acceleration navigation control formula,calculate the UAV acceleration according to the formula,and use the acceleration value as the input of the 3DSCL1 navigation controller to realize the heading and roll control;Then use the projection of the position vector of the closest point of the drone to the curve on the longitudinal plane to calculate the height of the drone’s flying target,and use this height value as the input of the height controller,according to the drone’s flying target height and height measurement value Calculate the pitch angle and throttle control commands with the climb rate to output the elevator control value to realize the height control of the UAV.Finally,a Software-In-The-Loop flight test base on the open source Ardupilot platform is performed to verify the effectiveness of the proposed algorithm.Finally,to address the problem of insufficient accuracy of model parameters of micro and small fixed-wing UAVs,a 3DSCL1 based on Q-learning algorithm,Q-3DSCL1 is proposed.Firstly,offline training is conducted based on UAV flight log data and Q-table with flight decision information is generated;secondly,based on the offline trained Q-table,position information of real-time UAV flight,acceleration gain is output optimal control value to achieve the adaptivity of UAV navigation control.In order to verify the effectiveness of the method in this paper,the 3D spatial curve navigation control algorithm based on BP neural network and fuzzy control is designed again for comparison,and the comparison experiment shows that the Q-3DSCL1 proposed in this paper has better control accuracy and strong adaptiveness.