Research On Master-slave Heterogeneous Manipulators Motion Algorithm Based On Reinforcement Learning

Teleoperated robotic systems are generally used to perform operations in special environments such as dangerous and unknown environment.In the face of complex and changing task requirements,the master-slave devices of teleoperation systems are often heterogeneous in structure,while the systems should have flexible,transparent and accurate teleoperation control.This paper takes master-slave heterogeneous remote operation robot system as the research object,and improves the control performance and human-computer interaction experience of remote operation system by carrying out research on two key technologies of master-slave space mapping and remote operation control.The main research work consists of the following areas:Firstly,a master-slave heterogeneous robot teleoperation system platform is built,and the kinematic modeling of the master-slave robot arm is carried out by DH method and exponential product method respectively,and the inverse kinematic solution process of both is derived based on geometric method;the communication between the master-slave robot arm and the upper computer is established by using ROS(Robot Operating System)framework and TCP/IP protocol respectively,and realized the data interaction.The reliability of the platform is verified by conducting constant scale mapping experiments on the built platform.Secondly,in order to achieve a wide range of mapping space coverage and positional mapping,a hybrid position-joint mapping method based on boundary drift is proposed.The method divides the teleoperation task into two phases: in the task approach phase,the robot workspace is covered in a wide range by the boundary drift algorithm,and the rate mapping is applied for control,so as to quickly control the slave robot’s motion to the target position;in the task operation phase,the position and pose of the slave robot are mapped with high accuracy simultaneously by a hybrid mapping algorithm combining the Cartesian space and joint space composition to achieve accurate following of the master control commands.And the pose following experiments are carried out on the teleoperation experimental platform to verify the effectiveness of the proposed algorithm.Then,a slave-side force-aware feedback control method based on improved Soft Actor-Critic(SAC)is proposed in order to improve the environment perception capability of the slave-side robot of the teleoperated system.The method introduces the Prioritized Experience Replay(PER)algorithm to improve the sparse reward problem in the traditional SAC algorithm and accelerate the convergence speed and stability of the SAC algorithm;fuses the force sensing information and trains the end-end force sensing information through the network to improve the robot’s end-end perception of the environment.And the six-axis robot arm model is built in Coppelia Sim simulation environment to complete the training of the improved SAC algorithm network,and the simulation results verify that the proposed method can better accomplish teleoperation control.Finally,a master-slave heterogeneous remote operation system contact task experiment is designed to verify the impact of the proposed algorithm on the control performance and human-computer interaction of the remote operation system.In the experiments,the trained improved SAC network is deployed on the built master-slave heterogeneous remote operation system experimental platform,and the proposed master-slave spatial mapping method is applied to perform the contact task experiments.According to the experimental results,the performance indexes of the whole system are analyzed and evaluated.The experiments show that the proposed master-slave spatial mapping method and the improved SAC control algorithm can make the system more transparent,the operator can operate more intuitively,the contact force of the slave is more stable,and the coverage of a large working space can be realized.The research results of this paper have some theoretical support and practical value for teleoperation control.