Position Control For Hydraulic Drive Unit Based On Deep Reinforcement Learning

Robots are increasingly commonly used in our daily life,from production line,housework to earthquake ruins,fire scenes,which makes stronger demands for robots.Hydraulic robots are designed for heavy load and complex environments.The hydraulic drive unit is the core actuator of robots and the basis of robot motion performance.The traditional PID controller is the most commonly used method in hydraulic systems,which performs well usually.However,hydraulic robots often work in wild environments,traditional methods can't satisfy the desired performance.Machine learning is a method to learn valid information from big data.Applying machine learning to controller design can equip the controller with the ability of learning the environment's characters and improving the performance,which has become a popular research orientation.In this paper,machine learning methods are adopted to hydraulic drive unit to equip the controller with self-learning ability to improve its performance.The following works are done:(1)The mathematical model of the hydraulic drive unit position control system is built using MATLAB/Simulink.Then the constant-value PID method and the varied-value PID method is studied based on the simulation model.From the analysis of the simulation results,the following things are got.Constant-value PID method performs poor in both control accuracy and adaptivity.Varied-value PID method performs well in control accuracy but also poor in adaptivity.(2)BP neural network is introduced to improve the performance of PID method.The learning sample is firstly collected.The first network is used to match the relationship between among condition,control parameters and system performance.Then a lot of samples with different control parameters and working conditions can be generated by network 1.For each working condition,an appropriate control parameter can be calculated using network 1 under certain performance requirement.The second network is used to learn the relationship between working conditions and the selected parameters.The finished network is adopted to the control system to change the PID control parameters automatically.The performance of the network improved PID control method is studied in simulation and analyzed in comparison with traditional PID method.(3)To improve the controller's adaptivity,an improved DDPG method is designed for hydraulic drive unit position control system.A fuzzy exploration strategy is designed to improve the exploring efficiency of the intelligent agent;Armijo-Goldstein Criterion based BFGS training method is used to the learning of networks to enhance the fitting ability;the new control variable to the servo-valve is generated by multiplying the policy network value and the system position error to make the algorithm more steady.Then,the improved DDPG method is research in simulation and analyzed in comparison with the former three methods.(4)The four control methods in this paper is studied on the hydraulic drive unit performance test platform and analyzed together.