Blended-Keyframes For Mobile Mediated Reality Application

With the recent rapid development of mobile digital products,not only are they used for industrial scientific research,many consumer-level mobile devices currently have strong computing capabilities,and can quickly process some simple 3D scan data.The rapid development of hardware devices has promoted the development of AR,MR and other mobile applications.The computing power of current handheld devices can support more complex calculations,and the resulting MR applications have become an integral part of today's society.These applications can support the user to place virtual objects in the environment where the user is currently located.However,based on the current status of relevant research,the method of simple editing of the surrounding environment through MR technology still faces various difficulties and problems.It is these problems that limit the current popularity of MR applications in consumer-grade handheld devices.In order to solve the problem of insufficient computing power of the mobile terminal equipment and difficulty in the calculation of 3D scene reconstruction,the author proposes(1)a 3D mobile terminal reconstruction method based on key frames.By using several key frames and replacing the storage of the three-dimensional model with two-dimensional images,the memory usage of the mobile terminal device is improved,and the demand for the computing power of the device is reduced.(2)Compared with the existing mixed reality method of using preset models,this paper proposes a mobile mixed reality method for non-preset model environments.There is no need to pre-process the scene in advance,the image is obtained through a consumer-grade RGB-D camera,and finally a two-dimensional perspective image with a modified real surrounding environment is output.The method proposed by the author can create key frame images based on 3D reconstruction of real scenes in real time,and use existing image segmentation techniques to segment objects from the environment.After the above operations,the method proposed in this paper can remove the target object from the current perspective image,and then use the 3D reconstructed virtual object of the target object to place it in the environment.The method proposed in this article can be applied to the creation of mixed reality related applications on the mobile terminal.