Research On Pap Operation Skill Learning Of Manipulator Based On Reinforcement Learning Algorithm

With the rapid development of artificial intelligence,reinforcement learning has gradually become one of the key algorithms for robot operation skill learning research.Aiming at the problems of long cycle,high cost and low efficiency in the development stage of robot operation skills,it is the most important to design a robot operation skill learning system with certain autonomous decision-making and learning ability.This paper focuses on the pick-and-place(PAP)operation skill learning problem.On the basis of comprehensive analysis of the research status at home and abroad,and combined with the relevant knowledge of reinforcement learning algorithm,a more effective PAP operation skill learning method is designed.Specific research contents are as follows:Firstly,aiming at the common problem of low efficiency and low success rate of PAP operation skill learning based on reinforcement learning algorithm,this paper proposes a robot PAP operation skill learning algorithm,which combines Actor-Critic(AC)and Q-Learning algorithm.Based on the equivalence of Actor-Critic and Action-Value,using the fusion learning of strategy algorithm,the ACQL algorithm combining AC and Q-Learning is deduced and designed;Secondly,during the process of PAP operation skill learning,the ACQL algorithm updates the strategy through the Actor-Critic algorithm,and stores the learned data in a replay buffer,then the Q-Learning algorithm is used to learn the data in the replay buffer and update the Q value,which solved the convergence problem of Q-Learning.Finally,a comparison experiment is designed to verify the effectiveness of ACQL algorithm to solve the problem.Secondly,in order to solve the problem of high data correlation and not very satisfactory learning efficiency in ACQL algorithm,a PAP operation skill learning method based on A3 CQL algorithm is proposed.Based on the Asynchronous Advantage Actor-Critic(A3C)algorithm,the ACQL algorithm is improved,and the A3 CQL algorithm is designed.The A3 CQL algorithm is used in the process of PAP operation skill learning,the advantage function is used to optimize the Critic network,so as to maximize the reward value of the PAP operation skill learning of the robotic arm;Secondly,when the robotic arm explores,it combines a value function,allowing agents to perform asynchronous learning;Finally,the efficiency of A3 CQL algorithm to solve PAP operation skill learning problem is verified through experiments.