Experimental Research And Application Of Mobile Augmented Reality Based On Gesture Interaction In Middle School

In the middle school experiments,the learning of experiment operation and the demonstration of experimental phenomena depend on various experiment equipment,which hampers experiment teaching or autonomous learning under the limited conditions.Combining the experiments with augmented reality technology,using virtual experiment equipment instead of physical one,and demonstrating experimental phenomena with experimental principles can facilitate the teaching and independent learning of the experiments.In this thesis,the mobile augmented reality technology and gesture interaction are investigated.The research objective is accomplished,including the lightweight of gesture detection algorithm feature extraction network,the acceleration of mobile terminal gesture detection inference and the gesture semantic interaction in augmented reality experimental scenarios.First of all,for the problem of complex structure of the feature extraction network and large amount of network calculation in the SSD(Single Shot Multi Box Detector)detection algorithm,this thesis combines the lightweight of deep separable convolution,and replaces the VGG feature extraction network in the SSD detection algorithm with Mobilenet network.Meanwhile,the multi-size convolution kernel groups are used to replace the single convolution kernel of the pooling layer of the Mobilenet network,which lightens the network structure while taking into account the characteristics of different receptive fields.The experiments show that this method can effectively improve the average accuracy of the target detection model.Secondly,to handle the problem of the slow inference speed of gesture detection in mobile devices with limited computing resources,the study adopts the network structure pruning and GPU inference acceleration on mobile terminals.Considering that the size of the gesture target in the actual scene is relatively fixed,the study removes the detection layer in the SSD detection algorithm that has a large amount of calculation but little effect on the detection result to reduce unnecessary reasoning processes.At the same time,with the help of Tensorflow Lite inference acceleration engine,the GPU is used to accelerate inference operations on smart phones.The experiments show that this method can greatly improve the speed of gesture detection when ensuring the accuracy of gesture target detection.Finally,this thesis implements the demonstration of the middle school experiments and application examples through the mobile augmented reality middle school experiment system.The system uses the mobile terminal augmented reality development tool to display virtual experimental equipment.Combined with the results of gesture detection,the interactive semantics are mapped to the semantic objects in the experimental operator's interest area.By analyzing the interaction semantics,this study completes the interaction operation and demonstrates the experimental phenomenon.