Research On Gesture Interaction Based On Cognitive Information

With the development of various aspects of computer technology, the traditional way ofhuman-computer interaction is facing great challenges, and natural and harmonious interactionis the trend in the development of human-computer interaction. Gestures as a widely used wayof communication in daily life, not only contains the powerful semantics, but also directlyexpress interactive intention, therefore, gesture interaction is one of the important way ofhuman-computer interaction. The vision-based gesture interaction with nature-friendlyinteractive way is the most accord with human interaction habits, so the vision-based gestureinteraction becomes human-computer interaction research focus.This thesis is supported by the National Natural Science Foundation (No.61173079, No.60973093), and Shandong Provincial Natural Science Foundation Project (ZR2011FZ003).This thesis makes a new gesture interaction studies in order to reduce the cognitive burden.How to reduce the cognitive burden of gesture interaction is the focus of this thesis. Cognitiveinformation can help the user to complete the operation as hints, metaphor, or explicit way, socognitive information can help reduce user’s cognition burden. This paper is focused oncognitive information and how to apply the cognitive information to specific gestureinteraction.This paper analyzes the cognitive information of gesture interaction in virtual assemblyplatform, and applies cognitive information in the process of specific gestures interaction. Thefirst study is the concept, classification and visualization of cognitive information, and theanalysis and mining of all kinds of specific cognitive information. Then cognitive informationis applied to the object selection, collision detection and correction process according to thecharacteristic of all kinds of cognitive information.In this paper, the main work is as follows:1. The gesture segmentation and gesture tracking in complex background. It establishesthe skin color model based on brightness index and characteristic vector, and makes the gesturesegmentation based on the skin color model, which has good robustness for indoor stabilitylight. The improved particle filter is used to make gesture tracking. The improved particle filteruses Gaussian sampling instead of random sampling and optimizes the weight to improve theaccuracy of gesture tracking. 2. The analysis of cognitive information of gesture interactive in virtual assembly platform.First, cognitive information is classified according to the role of those, secondly, it analyzes andsummarizes the characteristics and the role of various kinds of cognitive information, and givepart of cognitive information acquisition method, finally, it gives the method of finding the bestcognitive information visualization.3. It puts forward the navigation information model for virtual assembly platform, thenavigation information is got through the scene information and3D gesture state information,and cognitive information is visualized in this model,, which guide the gesture interaction. Thenit puts forward the new object selection and the collision detection algorithm based on thenavigation information model, which not only makes less time when performing objectselection and collision detection, enhances real-time interaction, but also reduces the user’sgesture interaction cognitive burden.4. This paper presents a self feedback correction algorithm, firstly it divides the process ofgesture interaction into sub processes, and sets up the scene information database and gesturestate information database, secondly, the gesture state information and the scene information ofthe start time of the sub process are stored in the corresponding information base, and itestablishes the gesture state information timing table and the scene information timing table,finally it achieves the fallback operation by visiting the gesture state timing information tableand the scene information timing table. This algorithm creates the scene information databaseand gesture state information database to avoid redundant information in the gesture interactionprocess, and divides the interaction process into sub-processes to ensure the integrity of theoperation. The experimental results show that, the self feedback correction algorithm is notonly fast error correction, but also reduce the user’s cognitive load. The algorithm is realizedon the error correction of the operator’s wrong operation, now the study about this aspect is less.