Research And Implementation Of Augmented Reality Electronic Sand Table Cooperative Interaction Technology

With the progress of the times and the rapid development of science and technology,the military electronic sand table is developing in the direction of diversification,intelligence and individualization.Under the conditions of modern information-based joint operations,commanders need to be able to conduct efficient discussions and collaborative work,which puts higher requirements for the intuitiveness of the electronic sand table,the convenience of interaction and the efficiency of collaborative.The traditional electronic sand table realizes the control of the sand table by means of touch screen technology and remote control technology.It has the disadvantages of high cost and inconvenient transfer and single interaction mode.So there is a big gap with the above requirements.In recent years,AR technology has become a hot topic of research.AR technology has brought a three-dimensional visual experience beyond reality and convenient human-computer interaction.This paper designs and implements a collaborative interactive system of electronic sand table based on AR technology.The main innovative achievements of this paper include:(1)A multi-mode natural human-computer interaction system for AR electronic sand table is studied and implemented.This paper analyses the interactive process and elements of electronic sand table and designs a multi-modal natural human-computer interaction framework,then establishes the mapping relationship between the semantics of natural human-computer interaction and sand table operation;The Chinese speech instruction library is defined according to the application requirements,And the Chinese speech recognition function is implemented based on the Speech SDK of Microsoft Corporation;The command gesture recognition is realized based on the surface myoelectric sensor;Experiments show that the natural interaction modes of gaze,gesture and voice can not only meet the interactive operation requirements of traditional electronic sand table,but also enhance the input efficiency and improve the naturalness and efficiency of interaction.(2)A local multi-person collaborative interaction technology for AR electronic sand table is studied and implemented.Based on virtual scene sharing technology,users wearing different AR devices can see the same virtual sand table placed in real space.The effective operation of sand table by any one person will be broadcast to other users synchronously,thus realizing the function of multi-person discussion.A concurrent control strategy is designed to solve the conflicts problem caused by multi-person simultaneous operation of the same target.(3)A three-dimensional expression simulation technology for virtual avatar is proposed to realize the function of visual collaborative discussion in different places.Firstly,the expressions of remote worker is tracked using a RGB camera,the face feature point data is extracted based on Constrained Local Models and the data is sent to local;The interpolation algorithm based on Radial Basis Function is used to calculate the coordinates of vertices of the virtual avatar face mesh,the model is driven to simulate the same facial expression with the other offsite worker;Finally,in order to improve the accuracy and efficiency of the deformation algorithm,a selection method of interpolation control points and a sub-region interpolation method based on the distribution of the human muscle group and the greedy algorithm are proposed,The experimental results show that the algorithm can meet the real-time and realistic requirements of applications.This paper mainly studies the natural human-computer interaction technology,and realizes the local multi-person collaboration function and the remote multi-person collaboration function of AR electronic sand table It enhances the intuitiveness of the electronic sand table,the convenience of human-computer interaction and the efficiency of multi-person research.