Design And Implementation Of Gesture Recognition System Based On Machine Vision

With the rapid development of information technology and the widespread use of intelligent terminal equipment,gesture recognition and gesture interaction are widely used in a variety of interaction scenarios due to their natural and intuitive characteristics.Relevant researches mainly focus on the usability theory and method of gesture recognition and the technical application of gesture interaction,which also has important research value for the systematic design of natural gesture recognition.The recognition of gesture information through machine vision can reduce labor cost and is of great significance to realize intelligent life.In this paper,a gesture recognition algorithm is designed based on YOLOv5 and Media Pipe Hands models to realize real-time and accurate gesture detection in a natural and unconstrained state,and the system is built on the embedded platform of Jetson Xavier NX.The main works are as follows:(1)To solve the problems of large size of deep learning model and poor real-time detection in embedded devices.Firstly,the lightweight model Shuffle Netv2 was used to replace the backbone network in YOLOv5 to reduce the number of model parameters.Then the attention mechanism is added to the backbone network to improve the detection accuracy.Finally,CIOU loss function is introduced to improve the convergence rate of model training process.The improved model compresses the number of parameters to 55.41% of the original one,and the floating point operations to 49.71% of the original one.The detection speed is increased by10.1FPS while the detection accuracy is guaranteed.(2)Combined with Media Pipe Hands,a YOLOv5＿Media Pipe gesture recognition method was proposed.Based on the above improved YOLOv5 detection model,the method first determined the target of the gesture area,and then detected and analyzed key points of the hand through Media Pipe Hands.According to the calculation of key point vector Angle of the finger,the finger bending situation was determined,and the gesture information was obtained.It effectively solves the problems of rotation occlusion in natural and unconstrained environment,low recognition rate and poor generalization of the existing methods when the lighting background is different.At the same time,12,000 gesture data sets associated with the proposed method were made,including 6000 self-built gesture images and 6000 public data sets.Different angles,different shooting distances,different occlusion degrees and different lighting backgrounds were fully considered,which greatly enhanced the robustness of model recognition.Finally,the effectiveness of the proposed method is verified by experiments on the self-made data set and the public data set.(3)To avoid the risk of cross infection caused by the use of public facilities in view of the current situation of COVID-19 and influenza A repeated uncertainty.A contactless gesture control system in elevator is designed based on Jetson Xavier NX embedded platform.The system reads the video stream through the camera,decodes the video stream and inputs the proposed YOLOv5＿Media Pipe model for reasoning.The system is implemented by multithreading,which can simultaneously read the video stream and conduct model reasoning.In addition,Tensor RT is used to optimize the system detection performance.By reducing the calculation accuracy and simplifying the model structure,it greatly improves the detection speed of the model on the embedded platform,so as to achieve the purpose of real-time detection.After testing,the detection speed of the system designed in this paper on the embedded platform of Jetson Xavier NX reaches 30.2FPS,and the average detection accuracy m AP@0.5 is 98.3%,which realizes the accurate and real-time detection of the gesture recognition scene in the elevator,and has great practical application value.