Research On Mixed Reality Fish Tank System And Interaction Technology

Fish tanks and fish have a positive effect on human physical and mental health The fish tank has a soothing effect,and watching fish produces a relaxing effect and attracts people's attention.In addition,the fish tank is also a common ornamental equipment for Chinese families,with a relatively simple structure and low cost.This article developed a Mixed Reality(MR)system based on the real fish tank,which effectively integrated the real fish tank environment with the virtual underwater world,to provide an interesting mixed reality environment.The MR system can be applied for children's English learning,cognitive rehabilitation training for children with autism spectrum disorders(ASD)and so onThe MR fish tank system developed in this article has a liquid crystal dimming film cured in the middle of the glass behind the real fish tank,which is used as a projection screen.The projected image can be projected on it and merged with the scene in the real fish tank.On the glass in front of the fish tank,there is an infrared touch frame,which turns the surface of the fish tank into a touch interactive panel.In the practical application of MR fish tank system,such as cognitive rehabilitation training for ASD children,we found there was a problem.Due to the refraction of glass and water,the content projected on the liquid crystal dimming film arrived at human eyes,produced a position "drift" It leads to the phenomenon of "touch is not what you see" when different users interact in different positions,that is,the problem of inaccurate touch interaction.In addition,the back and forth movement of real fish in the fish tank causes a certain occlusion of the virtual scene,so that users cannot effectively obtain the information of the virtual scene.For this reason,this article studies the mixed reality fish tank system and interaction technology.In summary,the main work and contributions of this article are as follows1.This article develops a mixed reality system based on real fish tank.It combines the rich ocean world with the real fish tank environment through projection display technology,providing users with a realistic and interesting mixed reality environment.The system integrates multi-touch and voice interaction technologies,which can provide users with a natural and simple way to interact2.This article proposes a precise touch interaction technology for MR fish tank in non-stereoscopic display environment.It aims at solving the problem of touch offset generated when users use the MR fish tank interface.On the basis of giving two offset compensation methods using light refraction and clustering method,a compensation method based on light refraction and clustering is proposed.The experimental results show that this method can effectively compensate the user's touch interaction on the fish tank interface and improve the user's touch accuracy3.In view of the real environment of fish tank,this article proposes a MR fish tank interface generation technology in stereoscopic display environment,considering the problems such as occlusion and light refraction deviation.It realizes the virtual-real fusion of the contents of the MR fish tank,which has more vivid and real stereoscopic images,to provide a realistic mixed reality experience.The real objects such as fish in the fish tank can be detected in real time to obtain its coordinate position,and then the light refraction method is used to compensate the influence of refraction on its position in real time.Based on this position information,a stereoscopic rendering method is adopted to display real fish tags in real fish tank,and realize collision detection and avoidance between virtual fish and real fish.In addition,this article solves the occlusion problem of real fish on the virtual scene and enables users to effectively obtain the scene information.