Research On Control Method For Micro Unmanned Helicopter Via Reinforcement Learning

The micro unmanned helicopter presents a complicated non-linear dynamics system with high-dimensional and strong-coupling, and it's always a challenging problem in building model and designing stable flight controller. In this paper, which is based on Raptor30 remote model helicopter, the state-space model is achieved by system identification, and the unmanned helicopter's attitude controller in hovering is designed via the reinforcement learning method.Based on the model of micro helicopter and reinforcement learning algorithm, the following issues are researched:First, parameters of 6-DOF helicopter state-space model are identified by frequency domain system identification and the hovering state-space model is achieved. In the process of identification, a search method is presented that the K-mean theory in pattern recognition is used in the cost function, which can promote the efficiency of identification. Meanwhile, the identified parameters are analyzed by the theory of Cramer-Rao inequality and insensitivity, and compared with the actual flight data, the identification result is excellent.Second, on the base of value function of the reinforcement learning algorithm, the attitude controller is designed by tabular method and Gaussian Softmax Basis Function (GSBF) neural network method. And the simulation result shows that when the system state is a continuing high-dimensional space, the GSBF algorithm has great promotion in study efficiency and controller characteristic than the tabular one.Last, in order to overcome the disadvantages of value function of the reinforcement learning algorithm, the policy search reinforcement learning is introduced. A new gradient search algorithm, which is based on Pegasus ideal, is presented, and according to the algorithm, the attitude controller of the unmanned helicopter is designed. The simulation result shows that the controller can stabilize the helicopter that hovers in place, and the conclusion of this algorithm is coincident with the actual flight more than the value function algorithm does.