Activity-aiding Human-robot Collaborative Handover Based On ResNet 6D Pose Recognition

In daily practice,there is a significant demand for diverse object handover between humans.For example,in the automobile production line,workers need to pick up parts and deliver them to colleagues,or get parts from colleagues and put them in the appropriate position.Similarly,in households,children assist bedridden elderly people by passing them a water cup,and in medical surgeries,assistants take over surgical tools used by doctors.These tasks require a considerable amount of time and manpower.In these scenarios,it is not only necessary to deliver the target object efficiently and quickly,but also to put the safety of the interacting object in the first place.Collaborative robots can serve as human colleagues to perform these simple,time-consuming and laborious tasks.We expect humans and robots to handover objects seamlessly in a natural and efficient way,just as humans naturally handover objects to one another.This paper proposes a 6D pose recognition-based human-robot collaborative handover system to address the problem of inaccurate object grasping caused by imprecise recognition of object poses during the human-robot collaborative handover process.A 6D pose recognition network based on the ResNet algorithm is introduced to achieve accurate recognition of the object posture and to achieve a precise grasp.Further,an improved method for making dataset of objects to be delivered is proposed to realize precise recognition of arbitrary objects.Finally,the precise pose positioning and accurate grasping of objects are realized by the vision system calibration,coordinate transformation,and improved grasping scheme.The main contents are as follows:(1)To solve the 6D pose recognition problem,the residual network is introduced to conduct semantic segmentation and key-point vector field prediction on the image,and the RANSAC voting is used to predict key-point coordinates.Then an improved EPnP algorithm is used to predict the object pose,which can improve the accuracy.(2)By analyzing the advantages and disadvantages of the existing dataset,and based on the existing 3D reconstruction technology,an improved dataset production method is proposed to realize the accurate recognition of arbitrary objects,which can reduce the time required for dataset production.(3)Through the internal parameter calibration and hand-eye calibration methods of the camera,the transformation relationship(from the object coordinate system to the camera coordinate system and then to the robot base coordinate system)is obtained.In this way,the pose of the target object in the robot base coordinate system is determined.A grasping method,separating position and orientation calculation,is proposed to realize precise object pose localization and accurate grasping.(4)To validate the effectiveness of the proposed human-robot collaborative handover system,a handover experiment platform was set up.with four volunteers conducting 80 handover experiments.The experimental results show that the average deviation distance of the proposed human-robot handover system is 1.97 cm,the average handover success rate is 76%,and the average handover time is 30 s.Without considering the grasping posture,the handover success rate can reach 89%.This demonstrates the proposed human-robot collaborative handover system has good robustness and can be applied to different scenarios and interactive objects,with promising application prospect.