Research On Deep-learned 3D Object Reconstruction Of Real-world Images

With the rapid development of artificial intelligence,information technology is leading the revolution in the fields of virtual reality,augmented reality,3D game.Information expresses gradually from 2D images to 3D representation.3D object reconstruction aims to generate the depicted object as faithfully as possible given 2D images.The main 3D representation methods are voxel,point cloud and mesh.3D object reconstruction is an important task to understand the relationship between images and their corresponding 3D representations.There are still many problems to be solved,especially for single view image 3D object reconstruction.Information lost caused by self occlusion of single view image,the change of image illumination and texture,the background interference between synthetic view and real world view,and poor reconstruction of detail parts all limit performance of 3D object reconstruction.Even though multi-view images brings some information,it still can’t solve these problems completely.Therefore,this paper will proceed from the difference between the synthetic view and the real world images,the details parts of the reconstruction results,use the image feature information and the coarse parts of the feature coordinates selection to solve these problems in 3D object reconstruction.The main work is as follows:Firstly,we propose a framework,which use image segmentation to segment the object from real-world images,and then use 3D object reconstruction to generate voxels results.Through the analysis of the transformation relationship between the whole image data distribution and the image feature data distribution on different domains of the real world view and the synthetic view,the experimental results show that the image instance segmentation method can effectively eliminate the background interference of the real world view.Secondly,we propose a 3D object reconstruction method based on convolution kernel parameter fusion,which extracts multi-branch features from the image feature map and fuses the model feature data.The multi branch skeleton convolution kernel is used to fuse the information of the feature map,and the multi branch encoder is constructed to extract the features of the view image.The parameters of the model are unchanged in inference,and changed in training.Experiments show that using multi branch encoder,model can effectively use the extracted view image features to improve the performance accuracy of 3D object reconstruction.Finally,we propose a 3D object reconstruction method based on selection of feature points coordinates,selected from feature map.Using the coarse feature points vectors,a shared multi-layer perceptron is used to generate fine feature vectors.Through the constructed loss function of the generated feature vectors,we achieve comparable performance while reducing the amount of model parameters and the inference time of model.Reducing the model parameters is beneficial to voxel generation when generating high resolution results.