Research On Depth Estimation Method Based On Multiple RGB-D Imformation Fusion

With the rapid development of naked eye 3D display technology,people’s expectations for 3D display content are rising.In 3D video generation,depth map,as an important way to characterize scene geometry,is the key information for generating high-quality 3D video.In addition,depth estimation has a wide range of applications in many fields,such as 3D reconstruction,video frame interpolation,and autonomous driving.However,vision-based depth estimation methods rely on image features that make dense matching between images difficult in regions such as weak textures,repetitive textures,and Non Lambertion surfaces,leading to degradation in the quality of depth estimation.And structured light-based depth measurement methods usually produce large measurement errors in dark-colored regions,highly reflective regions of surfaces and the edge regions of objects to be measured.Therefore,in order to cope with the depth estimation problem in the above challenging regions,this thesis proposes a depth estimation method in which structured light depth measurement information and multi-view geometry information are guided by each other.The research contents and innovations of this thesis are as follows:(1)A high-precision joint calibration method for multi-camera arrays is proposed.The calibration method first decouples the complex multicamera array system into multiple sub-camera array systems for separate calibration,and then uses reference system transformation and global optimization to obtain the calibration parameters of high-precision multicamera arrays.The calibration method is proved to be effective in solving the calibration problems of 40-way ring camera array and 5-way depth camera array,in which the root mean square error of reprojection is reduced to 0.35 mm and 0.20 mm,respectively.The multi-camera array joint calibration method proposed in this thesis has the characteristics of wide applicability and high accuracy,and can provide high-precision camera parameters for the subsequent depth estimation task of multi-view RGB-D information fusion.(2)To address the shortcomings of the vision-based depth estimation method and the structured light-based depth estimation method,a depth estimation network based on multiple RGB-D information fusion is proposed and a relevant dataset is constructed for network training.The network uses the depth information acquired by the structured light depth camera and the multi-view geometry information for joint depth estimation,which can not only accurately estimate the depth of the whole scene,but also generate more accurate and complete depth images for challenging areas.The experimental results show that the root mean square error of depth estimation is reduced from 16.52 cm to 3.47 cm at input,and the depth error rate is reduced from 17.8%to 15.5%.The depth map generated by the depth estimation network proposed in this paper has clear edges and correct depth in areas such as weak textures and nonLambertian bodies.