Research On The Model And Algorithm Of Multimodal Intention Understanding For Virtual Teaching

Virtual reality technology can bring a strong sense of immersion and reality.It uses computer functions and devices reasonably to generate a simulation environment,so that users can experience things that can be seen and cannot be seen in the real world.Based on this characteristic and the development demand of education field,it is an inevitable trend to apply virtual reality technology to education field.In the "opinions on accelerating the development of virtual reality industry" issued by the Ministry of industry and information technology of the people's Republic of China,a number of objectives for accelerating the development of virtual reality industry in China are clearly put forward,among which "VR + Education" is put forward in the application of key industries.This means that at the level of national strategic development needs,the application of virtual reality technology in the field of education is also of great significance.The combination of the two can be summed up as "virtual teaching".The convenience,economy and safety of virtual teaching are complementary to the difficulties in the implementation of scientific experiment teaching,the shortage of experimental materials and the potential danger existing in current scientific education,which is the main driving force for the continuous development and innovation of virtual teaching.In addition,the rich user information can make the system better understand the user's intention,and also make the interaction of virtual teaching more natural and approachable.For example,multi-modal information such as gesture,voice and touch make it possible for more abundant virtual interaction behaviors and methods.The development of virtual teaching can not only solve the stubborn problems in current science education,but also stimulate students' enthusiasm for exploring the scientific world.This paper takes the virtual experiment teaching system for middle school education as the background,and takes "understanding users' complex intention,improving users' learning effect and learning experience" as the ultimate goal.In view of the problems of high interaction load,poor presentation effect,and inadequate understanding of user intention in the current virtual teaching experiment system,this paper proposes a multi-modal intention understanding method based on virtual interaction,establishes a virtual experiment teaching system based on virtual reality,and solves some interaction defects in the current virtual interaction.The main research contents and innovations of this paper are summarized as follows:(1)This paper proposes interactive tools and methods for virtual teaching,and proposes new fusion features for common click-gestures to improve the real-time and stability of fingertip tracking.We analyze the characteristics of user interaction in virtual interaction based on virtual hand,and propose a correction algorithm to solve the problem of virtual and real human hand dislocation in virtual interaction(2)This paper proposes the intention understanding algorithm and navigation interaction method based on multimodal fusion.Aiming at the complex user's intention under virtual interaction,the intention understanding model is constructed by using multimodal information,which helps the system to infer the user's current operation intention effectively.On this basis,the intelligent navigation algorithm is proposed to help the user to carry out virtual experiment efficiently and conveniently.(3)The intelligent experimental equipment of virtual reality fusion is proposed.In order to solve the cognitive errors,unclear functions and unclear meanings caused by fully virtualized experimental equipment,the interaction channel between users and virtual equipment is built by using micro hardware equipment to realize the function recurrence of real experimental equipment,so as to help users better understand the actual role and significance of experimental equipment.(4)This paper puts forward the interactive mode of inquiry experiment in virtual teaching.The current virtual teaching mostly realizes standardized experiment teaching,that is,users are required to complete specific experiments according to standard processes and steps,without considering the adjustable factors and influence conditions of experiments,which limits users' deep understanding of experiments and their enthusiasm for scientific experiments.The inquiry interaction mode combined with real-time interaction provides users with richer interactive content,increases the user's operable space and inquiry space,and greatly stimulates the user's enthusiasm and thirst for knowledge.