Research On Human-to-Robot Object Handover Based On Action Recognition

Due to the involvement of multiple objects such as robots,humans,and objects,the human-to-robot object handover covers numerous research fields such as intelligent perception,decision planning,and motion control.Existing research on human-to-robot object handover has focused more on object pose prediction and grasping path planning,while not enough attention has been paid to the prediction of human handover intention and grasping detection in handover scenarios.Based on this,this thesis explores the improvement scheme of relevant algorithms from two meaningful perspectives,so as to further enhance the intelligence and smoothness of human-to-robot object handover process,mainly including the following:In the handover pose keypoint detection and action recognition task,firstly,the prediction ideas and regression methods of the pose estimation network are comprehensively analyzed.Based on the above analysis,the YOLO-Pose model is selected as the keypoint detection network.Secondly,considering the limitation of the robot’s field of view within the effective handover range and the specificity of the handover action,a set of upper body key points is redefined to enhance the characterization ability of the skeleton information.Then,the Bone-OKS(Bone Object Keypoint Similarity)loss function is used to enable the network to learn the length information of each part of the skeleton in order to reduce unreasonable postures and accelerate the convergence of the model for the odd postures that appear during the inferencing process.Then,to further improve the output stability of the model in scenes with fast hand movement speed,an exponential sliding average filtering algorithm is introduced in the model post-processing process to alleviate the output jitter.The experimental results show that the Bone-OKS loss function improves the YOLO-Pose accuracy by 0.8% on the COCO-Whole Body upper body key point dataset after label extraction.The exponential sliding average filtering post-processing algorithm can effectively alleviate the keypoint output jitter under correlation conditions.Finally,in order to predict the handover intention based on the obtained key point information,Spatial Temporal Graph Convolutional Networks(ST-GCN)was selected to train on a dataset containing four types of handover actions,and achieved 94% recognition accuracy.In the handover capture detection task,Generative Residual Convolutional Neural Network(GR-Conv Net)is selected as the grasping detection model for the handover system.To address the problem of inaccurate grasp frame localization in GR-Conv Net under handover conditions,the grasp quality label generation method based on elliptical Gaussian distribution is used to fit the importance differences of pixel points in different regions by analyzing the distribution of suboptimal grasp frames in grasping detection and inspired by the localization method of Anchor-free class method in the object detection task.At the same time,the Focal Loss idea is borrowed to add weight focus design to the GR-Conv Net loss function,which makes the network more focused on learning the grasping of the center region of the elliptical Gaussian kernel.The experimental results show that the accuracy of the optimized GR-Conv Net model increases from 82.5% to 87.6% on the handover grasp detection dataset when the grasp quality label encoding method is changed from binary to ordinary Gaussian distribution,and achieves another small improvement of 1.1% when the grasp quality label generation method based on elliptical Gaussian distribution is further used.For the GRConv Net model equipped with both the elliptic Gaussian distribution grasping quality label generation approach and the FS-L1(Focal Smooth L1)loss function,the accuracy reaches 91.3%,an 8.8% improvement over that before any improvement is applied.In the final handover experiment session,the improved models for each of the above segments were systematically integrated.The models are tested on an experimental platform for human-machine object handover,which is built on the Jaco2 robotic arm and Real Sense D435 i depth camera.Based on the completion of the robot arm hand-eye calibration,four students were arranged to conduct a total of 240 humanto-robot handover experiments for six types of common household objects.The experimental results show that the overall handover success rate of the system is 77%,thus providing some reference for research in the field of human-to-robot object handover.