Industrial Robot Training And Sim-to-real Transfer Method Based On Deep Reinforcement Learning

Deep Reinforcement Learning(DRL)enables agents to learn policies in different domains,which has attracted widespread attention in academia and industry.In manufacturing,dexterous manipulation skills can be learned from raw pixels by training industrial robots using DRL algorithms.However,directly training physical robots is a time-consuming and costly process.A common solution is to first train the robot in a virtual environment and then transfer the algorithm to the physical robot.How to realize sim-to-real transfer is a challenging problem faced by robot training.The ability of digital twins to create a dynamic,real-time representation of a physical robotic grasping system provides an efficient approach to address the above problems.In this thesis,we focus on industrial robot grasping and assembly task scenarios based on deep reinforcement learning algorithm,and propose a sim-to-real transfer method based on digital twin for transferring the DRL algorithm trained in virtual environment to physical robots,in order to realize the successful grasping and assembly of parts by physical robots.This thesis establishes two sets of parallel training systems with virtual and real images as input,and uses the output of the virtual system to correct the grasping coordinate points of the real scene,so as to achieve more accurate and efficient grasping.The experimental results verify the effectiveness of the deep reinforcement learning algorithm in the grasping action of the robot and the feasibility of the sim-to-real transfer method based on the digital twin system.Firstly,this thsis designs a set of industrial robot grasping and assembling system by analyzing the characteristics of industrial robot grasping and assembling action,and analyzes the forward and inverse kinematics of the industrial robot.Based on the parameters of the industrial robot and the parts to be assembled,the construction of the virtual entity model in the simulation environment is completed,and the construction of the basic functions of the digital twin system is realized based on the information acquisition and control of the industrial robot in the actual task scene.Then,this thsis proposes an industrial robot grasping action training method based on the Prioritized DQN algorithm,and a sim-to-real transfer method based on a digital twin system,and introduces the concept of reliability in it.The algorithm that has been trained in sim scene is used as the benchmark,and the output coordinate value of the algorithm is used to correct the output coordinate results of the algorithm for sim-to-real transfer in the actual task scene,so as to improve the training efficiency of the sim-to-real transfer process.Finally,this thsis preliminarily verifies the grasping effect of the industrial robot based on the Prioritized DQN algorithm by training the industrial robot in the simulation scene.Then,through the digital twin system established above,the image information in the sim and real environments is obtained respectively.And respectively used in the training process of the corresponding neural network model.By setting up a simulation experiment platform and a physical grasping experiment platform,the sim-to-real transfer method combined with the digital twin is verified,and by comparison with the grasping experimental results of the simto-real transfer process without the combination of the digital twin system,it is proved that the method is effective in feasibility and reliability during sim-to-real transfer.