Research On Flight Control Of Small Coaxial Twin-rotor UAV Based On Deep Reinforcement Learning

Small coaxial twin-rotor unmanned aerial vehicle(UAV)has the advantages of low cost,small size,compact structure and strong load capacity.It has broad application prospects in military fields such as reconnaissance,surveillance and key target attack,as well as civil fields such as electric power inspection and remote sensing shooting.However,due to the strong coupling,underactuation and strong nonlinear characteristics of the small coaxial dual-rotor UAV system,the classic methods of control have been difficult to fulfill the control requirements,which poses a great challenge to the dynamics modeling and flight control of the small coaxial dual-rotor UAV.Based on small coaxial double rotor UAV as the research object,basic stability control in the implementation,on the basis of the combination of deep learning feature extraction technology and the environment interaction strategy of reinforcement learning,reinforcement learning method based on depth design with good command tracking performance,and the actuator fault with fault tolerance flight controllers.The main work of the thesis includes:Firstly,a small coaxial twin-rotor UAV system was built,the dynamics characteristics of the small coaxial twin-rotor UAV were studied,and the nonlinear dynamics model was established.Through the open-loop simulation experiment,the influences of the dynamics uncertainty and external interference factors on the dynamics model were analyzed.Furthermore,a stability controller based on PID is designed to realize the stability control of small coaxial twin-rotor UAV.The effectiveness of the controller is verified by simulation experiment and hovering flight test.Secondly,according to the characteristics of the small coaxial twin-rotor UAV system,the reinforcement learning controller is proposed and optimized.The random selection point of the strategy is changed to the scheme of selecting according to the change of probability density,and the layered reinforcement learning algorithm is designed to improve the training efficiency of the reinforcement learning algorithm.Furthermore,the Actor-Critic(AC)frame in traditional reinforcement learning is extended to the strategy gradient method.Aiming at the control problem of small coaxial twin-rotor UAV,A training framework of Deep reinforcement learning controller based on Deep Deterministic Policy Gradient(DDPG)algorithm was constructed.Simulation results show that the controller based on DDPG has better robustness and adaptive ability than traditional PID controller when the target state and initial state are different.Finally,a passive fault-tolerant control scheme based on multi-dimensional vector field is proposed to solve the fault tolerant control problem of small coaxial twin-rotor UAV under environmental disturbance and fault conditions.The multi-dimensional vector field determines the direction of each dimension through the classification of support vector machine(SVM),and generates compensation instructions through the prediction and calculation of small coaxial twin-rotor UAV’s attitude and position and other flight states,so as to realize the passive fault-tolerant control of uav under fault conditions.Through simulation experiments,it is verified that the passive fault-tolerant control scheme makes the small coaxial twin-rotor UAV system have good fault-tolerant performance when the actuator is faulty.