| Vision-based 3D reconstruction task is a classic computer vision task,which plays a vital role in digital twins and automatic driving.Vision-based 3D reconstruction takes multi-view images of a scene as input.It constructs the 3D geometric structure of the scene through the structure from motion,sparse reconstruction,dense reconstruction,point cloud registration,and other steps.Benefiting from the convenience,cheapness,and richness of image data,3D visionbased reconstruction has made great progress.In recent years,scholars have made detailed explorations for different calculation modules of the 3D reconstruction process,but still inevitably left some problems to be solved.This thesis focuses on dense reconstruction and point cloud registration among them.The challenges of these two parts are as follows.First,The initial depth assumption layer in depth estimation of dense reconstruction is often obtained by dividing the sampling interval gained from the minimum depth and maximum depth in the scene,which can not provide accurate initial depth hypotheses for the algorithm,thus resulting in matching ambiguity and wrong depth estimation.Secondly,the poor significance of features in weak texture and non-textured regions also causes matching ambiguity.Moreover,the occlusion of foreground objects and the camera pose transformation lead to different pixel visibility.How to weigh the matching cost according to the difference in pixel-wise visibility is also the difficulty of depth estimation.Regarding point cloud registration,the scale of a point cloud,which is often millions or even tens of millions,poses a severe challenge to the efficiency of registration algorithms.Different interference conditions,such as noise and outliers,affect the accuracy of point cloud registration.Finally,due to the lack of geometry in the partial-to-partial point cloud registration,the extracted high-dimensional features contain different shape information,which limits the registration accuracy.Given the above challenges,this thesis accomplishes in-depth research on depth estimation and point cloud registration in 3D reconstruction.The main research contents include the depth estimation algorithm based on monocular prior and normal vector constraints(MDN-MVSNet),the Super Edge 4 Points Congruent Sets(Super Edge 4PCS)algorithm for point cloud coarse registration and the Virtual Points Registration Network(VPRNet)algorithm for partial-to-partial point cloud registration.The main contributions of this thesis are as follows.1.A depth estimation algorithm based on monocular prior and normal vector constraints estimation called MDN-MVSNet is proposed.The basic process of the existing popular MultiView Stereo(MVS)depth estimation algorithm is to sample depth hypotheses within the depth range of a scene firstly,then perform high-dimensional feature extraction on the input images,construct the matching cost volume according to the image pose and multiple depth assumption,and finally regress the cost volume to output the depth map.Among them,sampling in the whole depth range of a scene causes matching ambiguity and inaccuracy of depth estimation.Besides,the network’s understanding of the scene structure depends on the accuracy of depth estimation,and there are no other spatial constraints to enhance the comprehension of the three-dimensional structure of the scene.Although some algorithms add the normal vector as the supplement of spatial information,the accuracy of vector estimation is also limited by the selection of neighborhood pixels like traditional algorithms.Moreover,present algorithms do not make corresponding changes to the poor feature significance of textureless and weak texture regions,resulting in similar matching costs for different depth hypotheses in these regions and the inaccuracy of depth estimation.Finally,a few algorithms estimate the visibility difference caused by object occlusion and pose transformation.Even though some algorithms use depth features to regress the confidence,it is also an end-to-end convolution operation,which is poorly interpretable.Based on the above problems,this thesis proposes a depth estimation algorithm based on monocular prior and normal vector constraints called MDN-MVSNet.This proposed algorithm first estimates the monocular depth of each input image and takes the estimated depth as the center to obtain the pixel-wise depth assumption,providing accurate initial depth values for MVS.Then MDN-MVSNet utilizes each image’s pose and monocular depth to calculate the robust overlapping 3D point set.Then the algorithm takes the points set and image features as input to calculate the pixel-wise normal vector through the normal estimation network,avoiding the selection of neighborhood points.After feature extraction,the feature saliency enhancement based on Vision Transformer is performed to improve the difference of matching cost.Finally,the proposed algorithm performs adaptive visibility estimation for each pixel based on feature matching and pixel continuity.It then weights the constructed cost volume with the pixel-wise visibility weight.The experiments on multiple scene data show that the proposed MDN-MVSNet algorithm has advanced accuracy and generalization compared with popular traditional and depth learning-based algorithms in recent years.2.An efficient point cloud registration algorithm,Super Edge 4PCS,is proposed.Although the current excellent traditional algorithm Super 4 Points Congruent Sets(Super 4PCS)reduces the calculation time to a linear level through smart indexing,the algorithm lacks scale reduction for the point cloud.So its efficiency is limited by the size of the input point cloud.Besides,the efficiency and accuracy of the Super 4PCS algorithm are easily affected by noise,outliers and other interference conditions.Especially,the registration efficiency of the algorithm is significantly reduced under strict restrictions such as the outlier ratio of 0.4 and the overlap ratio of 0.2.The Super Edge 4PCS algorithm proposed in this thesis reduces the scale of input point clouds by boundary segmentation and overlapping regions extraction.It restricts the selection of candidate registration base pairs in overlapping regions,thus can improve the registration efficiency.Simultaneously,boundary segmentation and overlapping regions extraction also weaken the impact of noise and other interference conditions like noise and outliers on efficiency and accuracy,thus improving the robustness of the algorithm.Finally,the algorithm includes a candidate registration base calculation module based on the volume information of the point cloud.This module employs the distance constraint of the spatial point pair to obtain the set of candidate registration bases efficiently.Experiments on point cloud data with different scales and overlapping ratios show that the Super Edge 4PCS algorithm proposed in this thesis is significantly more efficient than the current popular Super 4PCS coarse registration algorithm under the premise of accuracy.Besides,it shows superior robustness and adaptability to strict constraints than Super 4PCS.3.A registration algorithm for incomplete point clouds,VPRNet,is proposed.The current popular deep learning-based point cloud registration algorithms,such as Deep Closest Point(DCP)and PointnetLK,extract the features of point clouds through PointNet or Dynamic Graph Convolution Neural Network(DGCNN).Then these methods map the low-dimensional coordinate to the high-level context information and then regress the high-dimensional features to obtain the transformation parameters of point clouds.However,due to the incomplete shape of the input point clouds,the extracted local and global information contains geometric differences,which affects the selection of candidate registration base pairs.The VPRNet algorithm for the partial-to-partial registration proposed in this thesis uses the attention-based Generative Adversarial Network(GAN)to complete the incomplete point clouds,reducing the impact of geometric structure differences in the feature regression and correspondence matrix computation.Then,the deep registration network iteratively calculates the transformation parameters of the point cloud by the fused high-dimensional features of the complete point cloud.Experimental results show that the VPRNet algorithm proposed in this thesis shows leading accuracy and stability compared to popular traditional and deep learning algorithms. |
