Application Of Vision-based Gesture Recognition Technology In Aeromedical Simulation Training

With the rise of artificial intelligence,gesture recognition technology has become more prevalent in human life.Static gesture recognition and dynamic gesture recognition are collectively referred to as gesture recognition technology.Compared to static gesture recognition,the technical complexity of dynamic gesture recognition is much greater.Although there are various ways to implement dynamic gesture recognition,most of the technical difficulties have not been completely solved.In this paper,we start from several steps such as hand detection,feature extraction,and dynamic gesture classification of dynamic gesture recognition,and combine with deep learning and other methods to implement a medical dynamic gesture recognition system for the first time in dynamic scenes applied in an aeromedical rescue flight simulator.The main work and innovations in this paper are as follows:(1)Hand detection: In response to problems such as complex backgrounds can cause a large burden on hand gesture recognition,this system gives priority to hand detection technology.Due to the small percentage of hands in the image and problems such as accuracy and real-time of model recognition,this system uses the improved YOLOv5 s for hand detection,which improves the detection accuracy by 2.0%.(2)Hand pose estimation: This paper proposes a top-down hand pose estimation algorithm for flight simulators that generate vibrations during flight simulation.The acquired hand images are input to the hand key point extraction module using direct regression method.Experiments confirm that the latest lightweight network Re XNet works best as the backbone network.Later,the effect of removing high frequency and low amplitude vibration interference in dynamic environment is achieved by filtering and other operations.(3)Action classification: For the problem that dynamic gesture recognition focuses on temporal information and this system contains multiple network models,this part uses a multi-layer stacked Gated Recurrent Unit(GRU)network model for experiments.For the problem that flight simulator simulation generates large bumps when encountering bad weather,this paper introduces attention mechanism in each layer of the stacked GRU network,which makes the classification network focus more attention on the gesture itself and ignore the interference caused to the user by sudden large bumps in the dynamic environment.The recognition accuracy reaches89.0%.In this paper,the dynamic gesture recognition algorithm for dynamic environments is proposed for the first time.The hand detection module is responsible for acquiring the hand position in the image,then performing hand key point extraction to obtain hand features,and finally performing dynamic gesture classification for medical surgery,using filtering and attention mechanism to minimize the interference in dynamic environment,so as to obtain a dynamic gesture recognition system applicable to dynamic environment.In addition,this paper creates three large datasets of dynamic gesture recognition system for aviation medical emergency rescue for the three modules of the system.