Gesture Recognition And Application Based On Lightweight Convolutional Neural Network

Because gestures are simple,easy to understand,and non-contact,gesture recognition and its applications are one of the hot research directions in the field of human-computer interaction.Compared to early costly wearable devices such as data gloves,vision-based gesture recognition has the advantage of non-contact interaction,especially in human-computer interaction systems such as smart robots,which use images collected by cameras to quickly detect and locate the gesture.Areas,by identifying the type of gestures and then determining the meaning of the gestures to control the corresponding functional modules,have the characteristics of flexible and simple control.However,due to the diversity of gestures and the complexity of detecting the background and environment,the accuracy of previous gesture recognition has a certain impact.With the continuous development of deep learning and deep neural network models,new opportunities have been brought to improve the accuracy of gesture recognition.Compared to traditional image recognition methods,deep learning takes images as input,and uses layer-by-layer convolution of the structured convolutional neural network to extract features,avoiding the singularity of artificially designed feature extraction algorithms,and using inverse Algorithms such as propagation continuously optimize the parameters of the entire network,thereby improving the accuracy and robustness of target object recognition.However,in recent years,in order to pursue higher recognition requirements,the deepening of the network structure has also generated more parameters,increasing the calculation cost,reducing the recognition speed of the entire model and reducing the opportunity for portability.This thesis mainly uses the lightweight MobileNet_SSD model obtained by combining SSD and MobileNet networks to detect 10 types of gestures in complex environments.Since the height and width ratio of each type of gesture is relatively fixed,the default anchor box width and height in the prediction layer The ratio is adjusted,which effectively improves the detection efficiency.The gesture image to be recognized is segmented according to the gesture prediction frame detected in the image,which reduces the interference influence of the surrounding environment on gesture recognition.In terms of gesture type recognition,it is also necessary to pre-process the segmented gesture images to further reduce the impact of gesture background on the accuracy of gesture recognition.The transfer learning InceptionV3 model is used to retrain on the self-built 10-type gesture dataset.The accuracy rate of recognition of 10 gestures on the validation and test sets is as high as 98.4%.In order to verify the real-time nature of the model,the model is imported into a human-computer interaction platform for testing.the experimental results show that this method can effectively improve the accuracy of gesture recognition and has strong real-time performance.