Research On Real-time 3D Reconstruction Model Quality Improvement Technology Based On Depth Camera

Real-time 3D reconstruction technology based on consumer-grade depth camera includes frame-by-frame capture data of depth camera,real-time estimation of camera attitude,extraction of surface model from fusion model,foreground segmentation and other steps,and in the reconstruction process,noise,redundancy and frame-to-frame mismatch often occur.This thesis describes in detail three ways to improve the accuracy and quality of the final 3D model from the data captured by the depth camera to the final establishment of the 3D model,which are adaptive processing of frame block,refinement processing of tsdf and foreground segmentation based on plane detection.In the shooting process of the depth camera,due to the severe jitter of the camera or the change of light,the current frame and the previous frames are often unable to match in the camera pose estimation stage,resulting in the invalid pose of the current frame.These frames are often abandoned and do not participate in the fusion and extraction,resulting in the hole or unclear of the final output model,which affects the quality of the output model.In this thesis,the adaptive frame block processing method is designed,and the local dynamic block method is adopted to reduce the number of mismatched frames,fill the model hole,and improve the quality and clarity of the model.In most 3D reconstruction technologies,the operation of fusion and extraction stage is often realized by parallel operation at the GPU end to ensure its speed.However,due to the limitation of GPU video memory,it is impossible to improve the accuracy of voxel size when scanning large-scale scenes.However,voxel size is the most important index to determine the accuracy of output model.This thesis designs the fine processing method of tsdf,realizes the whole process of fusion and extraction on the CPU,improves the accuracy of voxel size(from 1cm to 0.25cm),and effectively improves the accuracy of the model.Due to the wide viewing angle of the depth camera,invalid data outside the foreground area is often captured in the shooting process,which not only affects the quality of the output model,but also consumes a lot of memory space.This thesis proposes a foreground segmentation method based on plane detection,tracks the boundary plane between the foreground region and the background region,realizes the segmentation of foreground and background,and only retains the model contained in the foreground region.Experiments and tests show that the quality of 3D modeling after improvement is significantly improved compared with that before improvement,and the accuracy and quality of output model are improved.