Study On The Multimodal Interaction And The Information Intergration

The development of the computer science and the transformation of the computing pattern have influenced our society's development deeply. In today and the future, the network and mobile equipment are so popular that they can not be adapted by the desktop computing any more. The Ubiquitous Computing, totally a new computing pattern, which emphasizes embedding computers into people's daily life and working environment, is just brought out in this situation. What we need is a disappearing computing environment without the limitation of time or location.In this environment, users can employ the services and resource freely, through the handset, the wearableset, and other computing equipments. The innovation from the desktop computing to the ubiquitous computing tells us that the harmonious and natural human-computer interaction is an important sign of ubiquitous computing. This thesis mainly analyses a new human-computer interaction: Multimodal Interaction(MMI for short), and studies the concept, the framwork, key techniques of the MMI and the status of the MMI in the ubiquitous computing.There are four main parts in this thesis: putting forward the thesis, expounding the concept of the multimodal interaction and its framwork, studying the information integration technique and developing the method of the multimodal information intergration. The main work has been listed as follows:(1) Analyses the development of multimodal interaction technique and its application in ubiquitous computing, and it is based on the single-channel interaction technique. This thesis introduces some popular single-channel interaction techniques, and analyses the functions of the context aware in Ubiquitous computing, which are the precondition of the study of multimodal interaction here.(2) Analyses the protocol of the multimodal interaction and the system requirement of the context aware computing, then the context awared model of the interaction management component and the system environment component are designed.(3) Information integration is a key technique to achieve the multimodal interaction. So, the strategy of the information integration is a keystone in this thesis. The structure description of the task, algorithms, the system's error admition, andother contents are studied for this problem.(4) Offer a current templet of the multimodal information integration, the common semantic describing modal is an essential factor. It is an idea of the thesis and also the goal of application software system. In the multimodal interaction techneque, it has more important meaning.(5) In the last part of this thesis, the research direction and foreground of the multimodal interaction technque are listed.