Application And Research Of Natural Interaction Technology In Virtual Simulation Teaching

At present,the country is committed to promoting the integration of information technology and education in order to improve the level and quality of education.The application of virtual reality and augmented reality technology in simulation teaching is one of the important manifestations of this integration.Traditional simulation teaching interaction methods mainly rely on mouse,keyboard,VR handle and other equipment,which lack immersion and natural interaction,and are not suitable for complex experimental environments.Natural interaction is a way of simulating human-human interaction,which is simpler and more efficient than traditional human-computer interaction.This paper aims to introduce natural interaction technology into virtual simulation teaching,realize multiplatform collaborative interaction,and provide a new idea,method and practical example for virtual simulation teaching.The premise of natural interaction is the formal expression of the user’s cognitive behavior,and the state switching mode of natural gesture semantic design and interaction is constructed based on cognitive factors.In order to solve the diversity of natural interaction terminal platforms in the current virtual simulation teaching,two gesture interaction methods are designed in this paper.Method 1: Record gesture samples,use VGB and Adaboost algorithms for optimization and training,complete gesture recognition,and combine HTC Vive to realize gesture interaction on the PC side;Method 2: Self-made gesture data set,use Yolo V5 algorithm to complete gesture recognition,and realize UWP Terminal gesture interaction.Through the gesture semantic design and the analysis of experimental results,both methods have a recognition rate of more than 90%,which can meet the teaching requirements of daily virtual simulation.Aiming at the problem that the current virtual simulation teaching platform is poor in scalability and unable to cooperate and interact,a technical design model based on C/S mode and IOCP is constructed to realize the asynchronous network communication between PC,mobile and Hololens multi-device.Firstly,the spatial coordinate calibration technology is used to unify the spatial coordinate systems of different platforms,and the state synchronization technology is used to ensure the consistency of object status and operation between different platforms;The one-person perspective solves the problem that only the wearer can see the Hololens screen;finally,the application of space anchor point technology solves the problems of virtual object drift,user misoperation and repeated positioning.Aiming at the diversity of virtual simulation teaching content and forms,the system technical route is constructed,and the natural interaction of typical 3D virtual simulation teaching scenes is designed and realized,mainly including: interactive simulation of Kinect and HTC Vive based on virtual reality technology,Virtual simulation based on augmented reality technology and Hololens2 virtual simulation based on mixed reality technology are adapted to a variety of common devices.Evaluation is an important link in the teaching design of natural interactive virtual simulation.This paper takes the chemical iron product corrosion simulation experiment as an evaluation case,constructs an evaluation model,designs the evaluation experiment process,and completes the cognitive and Interactive assessment.This experiment effectively demonstrates the scientificity and usability of the method,technology and visual evaluation proposed in this paper.