The Method Of Intention Perception Based On Multi-mode Fusion And Its Application In VR

Experimental teaching plays a very important role in middle school teaching.However,there are some problems in the teaching process,such as dangerous experiments(such as the dilution of concentrated sulfuric acid),expensive instruments(such as the high-precision microscope),and difficult to observe the three-dimensional structure(such as the heart model,the molecular model of substances and the mathematical geometry model),which make the experimental teaching difficult to carry out or be replaced by video and PTT.In order to solve the problems in experimental teaching,people use the technology of human-computer interaction and virtual reality in experimental teaching to build a virtual experimental teaching system.At present,most systems used in experimental teaching have problems of insufficient intelligence,unnatural interaction and insufficient experience of experimental operation.Therefore,this paper proposes to use the multi-mode fusion method to perceive the user's intention.Through the multi-mode intention perception method,not only can make up for the lack of intelligence of virtual experiment,but also can get rid of the bondage of mouse,keyboard and handle through natural interaction to restore the powerful function of human hands.In addition,the integration of tactile modal information based on experimental simulation equipment can greatly enhance the user's experimental operation feeling and cultivate the user's hands-on ability.Innovative points of this paper:Firstly,a k-means clustering dimension reduction euclidean metric template matching algorithm(KEM algorithm for short)based on data glove is proposed.The gesture recognition rate of this algorithm is 99.42%.Second,a multi-modal intent perception method based on monitoring sequences(MFGVM algorithm for short)is proposed.It combines gestures and voice,can perceive the user's relatively simple interaction intention,and has good intelligence and naturalness.Third,a multi-modal fusion intention perception method based on SVM and set theory(MSSN algorithm for short)is proposed,which combines multi-modal information of visual,auditory,tactile and contextual scenes.It describes more complex user intentions by means of intention behavior nodes,so that the system can more fully understand user intentions.Fourth,a navigation experimental system based on MSSN algorithm using experimental simulation equipment is proposed.Through the experimental simulation equipment designed in this paper,the user's operating experience is improved.According to the MSSN algorithm,the system senses the user's intention and conducts navigation,thus reducing the user's load and improving the experimental efficiency.