A Human-to-robot Object Handover System Based On Human Behavior Patterns

With the development of artificial intelligence technology,human-centered and unrestricted human-robot interaction is becoming a research hotspot.Allowing robots to understand humans and perform operations in accordance with human behavior patterns is gradually becoming a new direction of human-robot interaction technology development.This paper starts with one of the basic tasks in human-robot interaction: human-to-robot handover behavior,and proposes a human-to-robot handover system based on human behavior patterns.This paper mainly conducts research from the following aspects:Firstly,a method of recognizing the human-to-robot handover intention is proposed by fusing RGB-D images and human bone joint information.The color images and the depth images are captured and aligned from RealSense D415 camera.And through the OpenPose skeleton recognition method combined with the mapping method of depth image,the threedimensional skeleton joints of human body can be obtained in real time;The center-joint-point feature representation method is used to describe the upper limb posture,and the support vector machine classifier is used to perform Real-time classification of postures;The regional growth method combined with HSV color threshold segmentation is used to detect the presence of hand-held objects,and the human handover intention is judged together with posture detection.Secondly,an object-handover-point prediction method based on human comfort model in three-dimensional space is proposed.Construct a human arm model to solve the forward and inverse kinematics and working space of the arm;Build a binary comfort model that combines the arm joint torque model and the medium joint angle model,and convert the object-handoverpoint prediction problem into a comfort optimization problem to predict the location of humanhuman object handover point.And carry out human-human object-handover-point verification experiment to verify the feasibility and accuracy of the model;Use the model in the process of human-to-robot object handover and based on real-time skeleton joint detection to solve the human body height information,and estimate the input parameters of the handover point prediction model.And realize the real-time solution of the object-handover-point in the humanrobot object handover process.Thirdly,build a robot platform and a human-to-robot object handover system.Introduce the ontology hardware and sensor hardware parameters of the robot platform,construct an interface layer API program to control the robot ontology hardware.And write a ROS(Robot Operating System)driver based on the interface layer,integrate the robot ontology hardware and various sensors to build a complete ROS layer ecology.Decoupling the upper and lower control of the robot to improve system stability;Design the control logic of the human-to-robot object handover system based on the process of the human behavior model,and build the framework of the human-to-robot object handover system and integrate the key modules.From hardware to software,build a set of experimental platform for human-to-robot object handover system.Finally,carry out the human-to-robot object handover system experiment.The data set of handover intention information are set up,and the classification training of handover intention are carried out by support vector machine.and use cross-validation to optimize hyperparameters to obtain 99.6% gesture recognition accuracy;Optimize the presence detection parameters of hand-held objects to increase the recognition accuracy to 97.3%;Carry out the overall experiment of human-to-robot object handover,let multiple users experience the object handover system and conduct RoSAS evaluation,proving that the human-to-robot object handover system based on the human behavior model proposed in this paper can effectively reduce the handover time and bring users a more natural and smooth handover experience.