Research On Autonomous Navigation Of Quadrotor Based On Reinforcement Learning

In recent years,with the wide application of quadrotor in aerial photography,resource exploration,crop protection and other fields,the industry has higher requirement on its control performance and intelligence.At the same time,the machine learning(such as neural network and reinforcement learning)is gaining advantages in robot control and navigation.In this paper,the neural network and reinforcement learning are used to improve the control performance and intelligence of quadrotor,and the problems in modeling,control and autonomous navigation of quadrotor are studied.First of all,in order to solve the problems of strong nonlinearity,vulnerable to interference and inaccurate modeling of quadrotor,this paper proposes a new modeling method based on BP neural network and a controller designing method based on Backstepping.According to the kinematics mechanism of quadrotor,the Dominant model of the quadrotor is established.And the parts that are difficult to model are identified by BP neural network and the Compensatory model is constructed.According to the Integrated model(including the Dominant model and Compensatory model),four nonlinear sub-controllers(including the position,altitude,heading,attitude sub-controllers)are designed respectively using the method of PID and Backstepping.The ways of modeling and controller designing improve the control performance,which is verified through a maze scene in Unreal Engine 4 platform to track the way-points.Secondly,this paper considers the problem of collaborative maze navigation using quadrotor and vehicle.The environment information is obtained from the three-dimensional perspective of the quadrotor,and the maze is reconstructed by image processing.In the reconstructed maze,q-learning-based reinforcement learning method is used to search the optimal path from a given start point to the end point.Later,this optimal path is sent to the unmanned vehicle enabling it to track the maze path.When searching the shortest path,this paper proposes a new q-learning-based search policy(named improved ?-greedy policy),which reduces the search time significantly.In this paper,the convergence is analyzed from the perspective of dynamic programming,and the effectiveness is demonstrated from several maze scenarios.Finally,this paper considers the autonomous navigation problem of quadrotor in a complex environment.The traditional navigation method usually discretizes the environment space first and then makes planning according to the discrete grid space.However,different discretization methods will lead to different planning results.In this paper,an autonomous navigation method based on policy gradient reinforcement learning is proposed,which enables the automatic planning and navigation for quadrotor directly in continuous space.During the flight of quadrotor,a set of neural network is used to record and learn the optimal path autonomously,which enables the end-to-end collision-free autonomous navigation for quadrotor.This paper gives the principle and improvement analysis of the policy gradient method through theoretical derivation,and verifies the effectiveness of autonomous navigation in a complex environment built in Unreal Engine 4.