Application Research Of The Shuttlecock Detection,Tracking And Trajectory Prediction Algorithms Based On Video Streams

In recent years,with the rapid rise of sports industry in China,the number of people engaged in various sports is increasing.Among them,badminton has become one of the most popular sports in China due to its advantages of the less restriction in fields and easiness to practice.At the same time,the rapid development of science and technology has advanced the development of intelligent badminton robot.With the promotion of the 14 th Robocon National Robot Competition "Shuttlecock Dual Males",more and more university researchers and enterprise teams at home and abroad focus on the research of full-automatic badminton robot.Vision system of high performance is the core of full-automatic badminton robots.For the market-oriented badminton serve machines,only players can be trained to hit all kinds of serves repeatedly.The badminton serve machines lack an effective visual feedback system and have a low level intelligence.Designing an intelligent badminton robot with low cost which can realize man-machine interaction can not only alleviate the demand for badminton trainers,but also improve the overall technical level of badminton enthusiasts and the watching experience of badminton competitions.In this dissertation,aiming at the vision system of badminton robot,we completed the research of shuttlecock side detection,tracking and trajectory prediction algorithms in video streams(two-dimensional image plane).Through the ZED camera,the video streams of flying shuttlecock were collected in multiple contexts with invariant backgrounds or slow changes form the side,and the experimental dataset was made.Previously,a fast tracking of object centers algorithm based on detection(hereinafter referred to as "FTOC")was proposed to realize shuttlecock tracking by combining the traditional three-frame difference,machine learning AdaBoost and inherent characteristics of shuttlecock flight process,such as the size,circumference and Euclidean distance between two consecutive frames of the flying shuttlecock(priori information).In the shuttlecock flight video stream of laboratory(complex context),the results showed that the shuttlecock tracking recall of FTOC increased from 78.87%to 94.52%.The tracking recall and average location precision of FTOC achieved the best in comparison with several classical tracking algorithms.In addition,the averageprecision,average recall and average frame rate obtained by FTOC in four simple and complex contexts of the flight shuttlecock video streams were 93.92%,85.92% and15.13 frames/second,respectively.Overall,FTOC algorithm has certain advantages in precision,recall,real-time and strong context robustness for shuttlecock tracking.However,FTOC algorithm operated in CPU mode,and the period of model training and parameters tuning is long.There is still room for improvement in recall and real-time performance.In order to further improve the computational efficiency and obtain more accurate shuttlecock coordinates.In the latter stage,aiming at solving the detection problem of flying shuttlecock as a small target in video streams,we improved the deep learning one-stage detection network Tiny YOLOv2 from two aspects,namely the loss function and network structure,and obtained the proposed networks M-YOLOv2 and YOLOBR for shuttlecock detection.For the flight shuttlecock video streams in four simple and complex contexts,the average precision,average recall and average frame rate of YOLOBR were 96.7%,95.7% and 29.2 frames/second,respectively.The results showed that compared with the Tiny YOLOv2 and M-YOLOv2 detection networks,YOLOBR has improved the precision,recall and real-time performance for shuttlecock detection,and has strong context robustness.Finally,according to the shuttlecock coordinates obtained by the FTOC algorithm,combined with the Least Squares method and Kalman Filter,we implemented the shuttlecock trajectory prediction in the video stream.The results showed that while maintaining real-time performance,the Least Squares method has certain predicted performance on shuttlecock flight trajectory,and Kalman Filter can optimize the shuttlecock predicted points.