Research On Real-time 3D Reconstruction Technology Based On Depth Camera And Tracking Camera

3D reconstruction refers to the process of establishing a mathematical model suitable for computer representation and processing of objects or scenes.Whether it is the traditional digital media film and television production,cultural relics restoration and protection,medical image processing,or the rapid development of human-computer interaction,augmented reality,virtual reality and other fields in recent years,3D reconstruction plays an important role.However,limited by the scanning equipment,the texture information of the scanned object and the camera pose estimation method,there are still many problems in the data acquisition and processing of real-time 3D reconstruction based on depth camera,which affect the robustness of the reconstruction and the accuracy of the model.This thesis studies the data acquisition and processing of depth camera and tracking camera in real-time 3D reconstruction,connects a variety of heterogeneous depth cameras,establishes a unified data interface,takes the image data of depth camera and the pose data of tracking camera as the input source,and takes the construction of 3D models of different scenes as the research goal to produce 3D geometric models with high precision and clear texture.This thesis focuses on the following aspects of data acquisition and processing of real-time 3D reconstruction:(1)In the acquisition stage of depth and color image data,aiming at the problems of various types of depth cameras and inconsistent data acquisition methods,a unified data interface is established for a variety of heterogeneous consumer depth cameras.This thesis studies the acquisition methods of depth image data and color image data of three consumer level depth cameras realsense d435,structure core and Astra plus;The main parameters of consumer depth camera are compared,the acquisition process of depth and color image data is analyzed,a unified base class of depth camera image data acquisition is defined,and three subclasses of depth camera are derived.Experiments show that the depth camera image data acquisition method implemented in this thesis can better use different kinds of depth cameras to obtain color image data and depth image data in the reconstruction process,and reconstruct and generate a three-dimensional model with high accuracy.(2)In the stage of pose data acquisition and processing,aiming at the poor effect of weak texture scene reconstruction,the optimization method of camera pose estimation based on pose data is studied.In this thesis,the data source of real-time 3D reconstruction is added,and the depth camera and tracking camera are used for scanning reconstruction.The pose data obtained by the tracking camera are preprocessed to complete the data conversion of different coordinate systems.The pose data is used to optimize the camera pose estimation in the modules of filtering and matching,local optimization and global optimization,so as to improve the robustness of reconstruction.Experiments show that the camera pose estimation optimization method based on pose data can effectively improve the accuracy of weak texture scene reconstruction model.(3)In the stage of image data feature detection and matching,aiming at the problem that feature matching is easy to make mistakes in the process of reconstruction,an image matching filtering method based on color and area is studied.In this thesis,three image feature detection methods sift,surf and orb in the process of real-time 3D reconstruction are compared,analyzed and implemented,and the matching filtering method based on color difference and support area is studied and implemented.Experiments show that the matching filtering method based on color and area can effectively reduce the number of false matches and improve the accuracy of pose estimation and the accuracy of reconstruction model.(4)Using the above improved image data and pose number acquisition method,camera pose estimation method based on pose data,image feature detection and matching filtering method,and 3D model generation method in traditional 3D reconstruction,a real-time 3D reconstruction system based on depth camera and tracking camera is built in this thesis,which can robustly realize the real-time 3D reconstruction of indoor objects and generate 3D models with high accuracy.