Research On Robot Path Planning Algorithm Based On Deep Reinforcement Learning

In recent years,with the increase of human-robot collaboration scenarios,robotic tasks have gradually developed towards complexity and diversification,such as robots opening and closing doors,robots cooking,robots sorting out scattered objects and so on.When traditional control methods deal with such unstructured scenarios,they often cannot find the optimal strategy due to the inability to establish an accurate model.Deep reinforcement learning provides a new solution.Relying on the powerful nonlinear fitting capabilities of neural networks,robots can explore and learn autonomously in unstructured scenes without the need for artificial design strategies based on environmental dynamic models.In theory,as long as the exploration time is long enough,the optimal strategy can be found.This article focuses on the trajectory planning algorithm of the 6-Dof robot arm,starting from the traditional model predictive control to the application of deep reinforcement learning,the following studies are carried out:(1)In order to meet the requirements for safety and smoothness of the 6-Dof robot’s online trajectory planning algorithm,a model predictive control based online trajectory planning algorithm is studied.and adaptive weight parameters,a combination of soft and hard constraints,etc.are proposed.This method can reduce the jitter of the trajectory while ensuring safety.An UR10 robot is used to verify the algorithm in simulation and real scene.(2)Based on the OpenAI＿ROS framework,a reinforcement learning training environment for the UR10 robot is developed,and an obstacle avoidance planning solution based on deep reinforcement learning is given.The current design framework based on the Cartesian coordinate system has insufficient convergence performance and generalization performance.Therefore,an obstacle avoidance learning framework based on the Frenet coordinate system is proposed,and the current mainstream three deep reinforcement learning algorithms are used for comparison in a simulation environment.Experimental results show that the new framework has a significant improvement in convergence performance and generalization performance compared with those based on Cartesian coordinate system.(3)Traditional control methods are more efficient in solving the task of controlling the robot body.However,it is difficult to design control strategies for unstructured tasks that require contact with the external environment of the robot.Based on the task of robot sorting out scattered objects,this paper uses a hierarchical reinforcement learning method of manual division,independent training,and state machine integration to train robot to push arbitrarily placed objects into target positions.The algorithm is verified in virtual environment and was transplanted to a real UR10 robot.