Research On The Predictive Simulation Technology Based On Augmented Reality Of Teleoperation System

The transmission delay of the remote operation will not only affect the stability of the teleoperation system, but also make operator do the misoperation easily. Therefore, in the teleoperation system, the predictive simulation module usually is used to solve the transparency problem of the system. The traditional predictive simulation module is based on virtual reality, which cannot reflect the reliable situation with the modeling accuracy problem and the unknown remote environment. While the predictive simulation module based on augmented reality can solve all the problems the traditional module has through the image fusion of the virtual model and the remote environment image, which make itself become the focus of the recent study of the predictive simulation module. In the teleoperation system, the motion state of the remote manipulator after the time delay can be achieved and corrected in real-time by the prediction algorithm based on Kalman filtering, and the virtual model with the same motion-state of the predicted result can be fused with the remote environment image by augmented reality technology. But, how to show the positional relationship of the virtual object and the real object correctly and how to predict the motion-state of the remote manipulator accurately still need further research. About the occlusion and prediction algorithm problems, on the basis of previous studies, the following tasks are completed in this paper:Firstly, the predict simulation platform of the teleoperation system based on augmented reality is built. According to the experiment requirements, the hardware devices are selected, the software platform and the communication system are designed, and the virtual model is built at the same time, which can complete a simple teleoperation task. After all these things, the methods proposed in this paper about the occlusion and prediction algorithm problems can be verified directly in the following sections.Secondly, an approach for occlusion handling based on the pretreatment of virtual object is proposed after the analysis of the current occlusion methods. The depth of the real object which occludes the virtual model can be extracted with the registration position of the virtual model, and the contour of the real object can be extracted from the depth image at the same time. In the OSGART scene where the virtual model is rendered, the contour of the real object is registered and rendered as a3D model. The virtual object is occluded by the rendered contour and only the part of the virtual object that was not occluded can be displayed. When the color image and the virtual object after the pretreatment were combined into a single composite image, the effect of occlusion handling can be achieved.Then, the prediction algorithm based on Kalman filter is decided to be used after the analysis of the current algorithms and the experimental conditions, and the basic Kalman filter algorithm and its corresponding model equations are introduced. On the basis of the current improved Kalman filter algorithms, the adaptive Kalman filter algorithm is proposed in this paper and the improved model equations are deduced at the same time. And through MATLAB simulation experiment, the method proposed in this paper is proved feasible and effective.Finally, the predict simulation platform of the teleoperation system based on augmented reality is built. The task of controlling the remote manipulator to move the test tube will be completed to verify the occlusion handling method and the adaptive prediction algorithm based on Kalman filter proposed in this paper are feasible and effective.