Study And Experiment Of Reinforcement Learning Algorithm

In recent years, the design of aperture radio telescope is much bigger, the observation band is much wider than before. The requirements of the tracking precision and pointing accuracy become higher either. Therefore, how to restrain the vibration problem of the antenna, becoming very important. The purpose of this paper is to design the controller makes the large aperture reflector antenna a higher pointing accuracy and tracking accuracy.Through in-depth study on reinforcement learning, build a kind of flexible structure controller based on Q-learning algorithm, the controller is a good solution to the problem of computing the value function when the reward and the state transition function can not know exactly.Secondly, based on the Q-learning algorithm for flexible structure controller, there will be a problem that a continuous state conversion to a discrete state, this problem will lead to the dimension disaster problem etc... Furthermore, considering the reinforcement learning is learning through interaction with the environment, agent can obtain less information. Therefore, this paper improve the flexible structure controller which based on Q-learning algorithm, the design of a PD+Q-learning type flexible structure controller which running the PD control first can provide effective prior knowledge, and making the Agent algorithm to speed up the convergence.Finally, through the MATLAB software, according to the result of the simulation and experiment, respectively to demonstrate the effectiveness of flexible structure controller with Q-learning algorithm and PD+Q-learning algorithm, the simulation results show that the PD+Q-learning controller can improve the pointing accuracy and the tracking precision obviously.