Teaching System Research And Design Based On The Multimodal Interaction

Received a good education can make us understand oneself should shoulder social responsibility and family responsibility and enhances the national overall quality unceasingly. Therefore, let the teaching level of citizen's influence improve. In addition to improve teachers' quality and improve the teaching method, the system is method to provide an effective of teaching level and a novel way of teaching. It not only can improve the enthusiasm of teachers' teaching, but also can improve the subjective initiative of students, improve teaching efficiency. So the overall teaching effect is greatly improved.With the popularity and depth of computers and a variety of interactive devices, intelligent, humanized natural human-computer interaction way is more easily accepted. Multimodal way of human-computer interaction is a combination of a variety of models of information interaction, and then each mode can play their own advantages. The comprehensive utilization of a variety of behaviors complete the acquisition of information and express the user's intention to finish interaction process of the persons involved with the computer better. Multimodal interaction theory applies to in the teaching to introduce the new teaching method for the traditional classroom and improve the students learning interest and subjective initiative.This system is mainly to study and design a teaching system based on multimodal interaction, use of gesture interaction, voice interaction, handwritten multimodal interaction patterns such as interaction, in a kind of brand-new teaching way to complete the whole process of teaching. The system obtains human bones node data and voice data by using the device body feeling detector Kinect and combines human body postures, gestures, voice with the mouse as a system of input methods. The skeleton of node data obtained by Kinect mainly applies to gesture recognition of interface elements operation and the flow chart of drawing the tablets two aspects. Using dynamic hand gestures method completes the gesture recognition process through the relative position of bone node and the range of setting threshold to determine, including one hand and two hands a variety of gesture recognition. If the recognized gestures match predefined gestures, the system will complete the corresponding operation. Tablets mainly use the right hand bone node to simulate the brush to finish drawing track, during the process of drawing combined with the gesture of left mouse button press to complete drawing process. Through the acquisition of speech information recognition for command of grammar and matching with predefined syntax command in the system. If the true value is greater than a certain threshold matching success, the system will execute the corresponding operation with the grammar command.The system makes use of the multimodal interaction, for example gesture interaction, voice interaction, handwritten interaction and so on, to complete the teacher's teaching process. At the same time, it provides a new way of teaching, then teachers and students complete knowledge and learning knowledge of the process in a unique way efficiently.