The Research On The Application Of Gesture Interaction Technology In Engine Room Virtual Reality

In the traditional virtual reality technology,the Human-Computer Interaction is based on the mouse and the keyboard,the same to the engine room virtual reality,they are lack of immersion experience.The Human-Computer Interaction based on somatosensory has gradually replaced the traditional one in some field.Whether the somatosensory base on computer vision or inertial sensor,it has made a lot progress.And the gesture interaction technology as a way of somatosensory interaction,it been filly utilized whether in the field of scientific research or in the production industry have.The paper is based on inertial sensor type somatosensory technology to connect the gesture interaction technology with engine room virtual reality for completing the interaction in human and computer.Firstly,three kinds of gesture representation methods and their differential equations are introduced.By comparing their features and combining requirements of the gesture device,the quaternion is selected as the gesture attitude representation method.The somatosensory data were collected by micro-electromechanical six-axis sensor,as the accelerometer and gyroscope exist their own errors,so the complementary filtering was used to fuse accelerometer and gyroscope data,in order to reduce the error.The fused data were used to update quaternions by the Runge-Kutta method.The attitude transformation matrix and Euler angles are calculated according to the relations between quaternion and direction cosine matrix and Euler angle.Secondly,the virtual hand model was established based on the real human hand in 3D modeling software.Thirdly,a wireless data glove was designed to collect the hand gesture information of the human hand,and calculate attitude transformation angles of the palm and fingers.Finally,transplant the embedded system on the data glove and design the embedded program by the C language.The virtual hand model and the virtual engine room model were imported into the Unity3D,and the model scripts were edited by C#language to receive the information from the data glove,and the virtual hand attitude transformation was completed in real time.During the measurement phase,the Human-Computer Interaction is realized through the data glove and the engine room virtual reality to realize real-time gesture display,and carry out specific instruction operation in engine room virtual reality.