Research On The Application Of Pose Estimation In The Vision System Of Table Tennis Robot

Table tennis is the "national ball" in China,which is deeply loved by the people.The research work of table tennis robot has a very broad application scene in our country.The table tennis robot system consists of vision system,decision-making system,control system and other systems.The vision system is the "eye" of the table tennis robot.Only through the visual system to timely capture and process the external information,such as the flight path and rotation type of table tennis,the batter’s swing motion,the racket’s motion path,etc.,can the decision-making system integrate these information,then retrieve the optimal return scheme,and send the program command to the robot’s control system.After that,the control system drives the robot to stroke.Therefore,the quality of table tennis robot stroking the ball is closely related to the quality of its vision system design.However,most of the table tennis robots only focus on the flight path and state of the table tennis itself in the design of the vision system,and ignore the research on the movement of the table tennis player and the changes of his racket.Therefore,this thesis proposes to apply the research of human body and racket pose estimation to the design of table tennis vision system.Through the analysis of the player’s and the racket’s pose estimation results,we can get the rotation category and rotation speed of table tennis,and help the decision-making system make the optimal return decision.In this thesis,the human pose estimation is applied to the design of the vision system of the table tennis robot.Owing to a complete set of hitting actions of a table tennis player is a group of video sequences,and most current pose estimation algorithms take a long time to process,this thesis proposes a new pose estimation algorithm based on high-resolution network.YOLO v5 target detection network with the Ghost module is used as the human detector of the pose estimation algorithm,and the high-resolution network is used as the human key point detection network.Accelerate the speed of human body detection and ensure the detection accuracy of the model.Then,a data set is made for the specific scene of the table tennis robot vision system to train the proposed pose estimation model.Finally,the experiment shows that the improved attitude estimation algorithm can quickly and accurately locate the key positions of the batter.Then the position and pose estimation of the racket is applied to the vision system design of the table tennis robot.This thesis calculates the rotation vector and displacement distance of the racket plane in space based on the PnP algorithm,and uses the obtained racket rotation and displacement information for the experimental data of the classification of table tennis rotation types by the later neural network.The improved Hough transform is used to extract the image coordinates of the feature points of the racket without marking,and the Canny edge detection algorithm is improved to improve the accuracy of the feature point location.Then the extracted feature point image coordinates are used for the coordinate transformation of PnP algorithm to calculate the racket position and posture.The experimental results of fitting the motion trajectory of the center point of the racket show that the fitted motion trajectory is basically consistent with the real motion trajectory of the racket,which verifies the accuracy of the method proposed in this thesis for estimating the position and posture of the table tennis racket during movement.Finally,this thesis classifies the rotation type of table tennis ball based on the position and pose estimation results of human body and racket.Key frames are extracted from the collected video sequence to obtain effective shot sequences.YOLO v5 network integrating Ghost module is used to detect the positions of rackets and players,and then GA-BP neural network is established according to the racket pose estimation results to classify five kinds of table tennis rotation categories.At last,the player’s gesture estimation results are used to analyze his action,and DTW algorithm is used to calculate the similarity distance of the swing action.The rotation intensity of the table tennis ball is classified according to the speed of the action.The final experiment shows that the human and racket pose estimation results can better classify the rotation types of the table tennis ball,which is effective and feasible in the design of the table tennis visual system.