Research On Robot Manipulation Skills Learning Based On Deep Reinforcement Learning

Robot manipulation skills learning is an important branch in the field of robotics research.With the rapid development of artificial intelligence technology,deep reinforcement learning algorithms have been initially applied to solve this problem.However,due to the sparse reward characteristics of robot manipulation skills learning tasks,existing algorithms have training problems such as low efficiency and non-convergence of training.Aiming at the problem of sparse reward,this paper improves the existing deep reinforcement learning algorithm and designs a more efficient robot manipulation skills learning algorithm to improve the robot’s perception ability and autonomous decisionmaking ability in complex environments.The specific research contents are as follows.Firstly,in view of the low training efficiency of deep reinforcement learning in the learning task of robot manipulation skills in a sparse reward environment,an Adaptive Temperature Hindsight Experience Replay(ATHER)algorithm is proposed.First,an adaptive temperature parameter is introduced,which can dynamically adjust the exploratory nature of the agent according to the environment to adapt to different tasks and different stages of the task,which improves the performance and robustness of the algorithm.Then,a simplified value function calculation is proposed.The method uses the action value function to approximate the state value function,which simplifies the algorithm model and improves the training efficiency of the algorithm.Finally,the target network mechanism and the dual network partial inheritance mechanism improve the stability of the algorithm training and solve the problem of overestimation.the problem of deviation.Secondly,aiming at the low utilization rate of data samples in the application of the post-experience replay method,a Meta-Learning Double Experience Replay Buffer Hindsight Experience Replay(MDHER)algorithm is proposed.First,without changing the overall size of the experience pool,the original experience pool is divided into two parts:the sampling experience pool and the virtual experience pool.Then,batches of data are extracted from the two experience pools in proportion to train the algorithm model,so as to ensure that both kinds of data exist in each training.Finally,based on the meta-learning method,the proportional hyperparameter is learned by using the knowledge learned in the past,and the ratio of the two data is dynamically adjusted according to the value of the two data,so as to achieve the purpose of improving the value of the training samples.Finally,the ATHER algorithm and the MDHER algorithm are implemented in the Fetch manipulator environment and the Hand manipulator environment using the Mu Jo Co simulation environment,and compared with the existing algorithms in 8 multi-goal and sparse reward robot manipulation skills learning tasks respectively,which verifies the effectiveness of the method proposed in this paper.