Research And Application Of Multi-device Test Scene Interaction Technology Based On Augmented Reality

In recent years,the rapid development of Chinese manufacturing industry,hardware equipment in various fields constantly updated and iterated,in this process,testing is the key to ensure the quality and performance of equipment.The testing process helps device manufacturers and developers identify and correct problems in product design and development to ensure high quality and reliability before the device goes to market.At the same time,testing can also help device manufacturers and developers optimize device performance and functions,improve user experience and satisfaction,and enhance the market competitiveness of devices.Therefore,in order to improve the efficiency of the test process and the experience of the testers,this paper developed an interactive system of multi-device test scene based on augmented reality technology and human-computer interaction technology under the premise of multi-device test scene based on augmented reality.The main work of this thesis is as follows:(1)A 3D registration algorithm based on ORB-Jaccard is proposed to improve the accuracy of tracking registration in 3D.By introducing the Jaccard similarity coefficient into the ORB algorithm,the Orb-Jaccard algorithm is proposed to further improve the matching of feature points,reduce the problem that the traditional ORB algorithm through HAMMING distance matching feature points are prone to mismatching and errors,and improve the accuracy of feature point detection and matching.A 3D registration technique suitable for multi-device scenario is obtained.(2)The GRU-Bi GRU-Attention model was designed and proposed for dynamic gesture recognition.By stacking GRU model and Bi GRU model,attention mechanism is introduced to improve the precision and accuracy of dynamic gesture recognition.Through the comparison experiment with the traditional cyclic neural network,it is proved that the model has high accuracy and real-time performance in dynamic gesture recognition.(3)The interactive system of multi-device test scene based on augmented reality is designed and implemented.Through Unity3 D and Vuforia tools,combined with the 3D registration technology based on ORB-Jaccard,3D registration tracking of multiple devices was realized,as well as the combination of virtual-real.In order to make the human-computer interaction function not limited to the camera hardware requirements,the interaction through dynamic gesture and mouse operation and the interaction with the model in the augmented reality scene through the mouse are respectively realized.Through the interaction of gesture recognition and mouse operation,it is developed by the trained GRU-Bi GRU-Attention model,Open CV and Media Pipe tools.The interaction between the mouse and the model is realized by mounting the script to the model of the device in Unity3 D.The whole system is developed using Python and Vue,which is realized on the basis of Django-Vue framework.Augmented reality and human-computer interaction are integrated into the system,and an interactive system of multi-device test scene based on augmented reality is realized.