Research On Cooperative Robot Teaching System Based On Visual Gesture Interaction

Robot programming by demonstration is a recent trend in robotics,employed to transfer new skills to robots from observations of tasks demonstrated by humans.Solving the problem of robot teaching and programming in a changeable environment is the key to restrict the further development of robots.Aiming at the problems of long programming deployment time of traditional industrial robots,low programming efficiency,inability to adapt to new environments,poor flexibility,high programming skills required for teaching operators,and long robot program development cycles,this paper designs a collaborative robot teaching system based on visual gesture interaction.The system uses a Kinect2 depth camera to interact with the robot by recognizing fingertip moving trajectories and static gesture signals in the workspace.The trajectory teaching task was completed by the marker pen at the end of the UR5 mechanical arm,which verified the flexibility and efficiency of the system.The main research contents include:(1)This paper takes the UR5 collaborative robot as the research object,the hardware composition and software algorithm framework of the whole robot teaching system are summarized.performs internal calibration on the depth and color cameras of Kinect2,and the hand-eye calibration relation of the robot is released through TF coordinate conversion function package.(2)The hand-foreground images extracted by the hybrid Gaussian background subtraction algorithm and the frame difference method are compared by simulation.The basic principles of image smoothing filtering,threshold segmentation and morphological algorithms are summarized.This paper studies the image processing algorithm of fingertip detection by gravity center distance method,and uses K curvature algorithm to eliminate non-fingertip points.(3)Based on the YCbCr skin color model and convolutional neural network,complete the task of static gesture recognition.After 200 epochs of neural network training,the gesture recognition rate reached 99.43%.Operators can control the teaching process through gestures,which further improves the efficiency of human-robot interaction and enhances the user-friendliness of the teaching system.(4)Based on ROS Moveit motion planning library and Qt Creator GUI development tool,a human-robot interaction is designed.Under the ROS system,a Kinect2 depth camera is used to capture the movement trajectory of the fingertip,and the computeCartesianPath path planning function is used to complete the task of robot teaching and reproduction.The experimental research on the four types of curve trajectories has verified the feasibility and effectiveness of the teaching method in this paper.By comparing the traditional teaching methods(teaching pendant teaching and drag teaching)to complete the teaching task time,The time spent on trajectory teaching based on visual human-robot interaction is much less than the traditional teaching method,and it takes 7.275 s on average.Further verified the efficiency and ease of use of this system.