Merging Virtual And Real Worlds In Mobile Augmented Realty Assembly System

Mobile Augmented Reality Assembly is a technology that apply mobile augmented reality into the industrial assembly area, aiming at improving efficiency and reducing costs of production. Merging virtual and real worlds is one of essential technology in augmented reality. By means that real world is overlaid with additional from a computer generated display. Illumination consistency and geometric consistency are two evaluation criterions in fusion, geometric consistency is most focused in this paper, and occlusion is primarily considered in geometric consistency. Occlusion is based on the correct space positional relationship between virtual objects and real scene to overlay virtual scene. Sometimes the virtual objects occlude the real object and sometimes on the contrary. Fusion scenes will more realistic by occlusion, thereby improving the user experience. There are three approaches for occlusion, including model-based, depth-based and image-based. The paper focuses on occlusion by model-based, and running on Android system.First of all, the feature points based marker-less recognition and model registered has been studied and implemented. Internal parameter matrix is acquired by camera calibration. 3D scene is recognized by feature points extracting and training, and false match is optimized. Virtual model is registered with three real target, which has different volume, structure and surface material. Moreover stability and robustness were verified with registered result.Secondly, virtual models were rendered on mobile terminal. The structure of OBJ model was studied. Then models were parsed by application layer on mobile. OpenGL ES rendering pipeline principle and coordinate transformation was studied. OpenGL ES light model management was implemented, made virtual models more realistic.Thirdly, occlusion was studied and implemented. Traditional fusion scenes just render virtual models at topmost and occlude real scene, then design a merged method with occlusion. Layer of application was designed, and laying superimposed scenes with different types. Further 2D aided information was oveilay, and 3D models was rendered efficiently. Depth map of virtual scene was acquired with OpenGL ES rendering engine, then depth map of real scene was acquired by virtual avatar. Occlusion was implemented with model-based, virtual models were occluded by three targets, merged scenes with correct geometric consistency were acquired.Finally, a completed mobile augmented reality guided assembly system had been established, and running assembly instance. Structure of Android system was studied, then the structure of overall system was designed. Functional modules of system architecture was designed and divided. Software and hardware was analyzed, then the environment of running was introduced. Instance was running in assembly system, and occlusion implemented in guided assembly process, aiming at getting a more realistic merged scene.