Application Research Of Deep Reinforcement Learning SAC Algorithm In Motion Control

Deep Reinforcement Learning is the most similar learning paradigm of Machine Learning to the learning style to natural animal,regarded by many scholars as the most feasible route to achieve Artificial General Intelligence.As it combine the decision making of Reinforcement Learning and the perception of Deep Learning,Deep Reinforcement Learning realize the end-to-end learning style from direct input to output.There is wide applicatory prospect and critical theoretical study value.There are some urgent problems to be solved at the current stage of study.To be first mentioned about is that the agent of Deep Reinforcement Learning must explore in the environment,gain experience by trial and error,and then upgrade the strategy.This problem leads to the situation where there is few landing product in the real environment when the exploring cost is high,while when the cost is low,project like AlphaGo dazzles.To solve the problem that the time and equipment is costly when realizing the Deep Reinforcement Learning in real environment,in this paper,the application of Deep Reinforcement Learning Soft actor-critic(SAC)algorithm in motion control is studied based on the linear first-stage inverted pendulum hardware test platform and the four-rotor UAV simulation test environment.SAC algorithm is the latest model-free deep reinforcement learning algorithm proposed by OpenAI team,which has the advantages of strong robustness and insensitive hyperparameters.However,there are few applications of SAC algorithm at present,so it has high practical value.The main research results of this paper are as follows:(1)A four-rotor UAV simulation test environment is established based on Python,to exercise the motion control experiment based on Deep Reinforcement Learning SAC algorithm.By comparing the design of different reward functions and the setting of hyper-parameter in the algorithm,the experience of deep reinforcement learning algorithm was summarized,and some problems and solutions encountered in the experiment were demonstrated,which provided a reference for solving the motion control problem with deep reinforcement learning.(2)Set the straight line based on PLC level inverted pendulum hardware test platform,and the simulation test platform was built based on the Python language simulation training environment.Use the SAC algorithm in the simulation environment for the inverted pendulum swing-up and stabilization and control training.The result was applied to inverted pendulum hardware test platform,and achieved good effect,providing new thoughts to the practical engineering application.