A Research Of Multi-view 3D Object Reconstruction Based On Deep Learning

Reconstructing 3D digital geometric structures from images is a core problem in many fields such as computer vision,computer animation,industrial manufacturing and so on.The ability to reconstruct the complete and accurate 3D geometry of an object is essential,which has been found applicable and useful in many areas.In recent years,with the development of deep learning,3D reconstruction of objects based on deep learning has become a research focus in the field of computer vision.In this thesis,voxel is used as the representation of 3D models.And this thesis aims to search on the method of multi-view 3D object reconstruction based on deep learning.The main work of this article is as follows:1.For the RGB image obtained by the camera only contains the surface information of the three-dimensional model,a method of constructing a dataset by simulating the 3D structure of biological macromolecules through an electron microscope to collect 2D projections is proposed,so as to obtain internal information containing the three-dimensional model dataset.2.3D-ResNet network is proposed to realize multi-view 3D object reconstruction.The neural network can be trained and tested without inputting any additional information such as image annotation information,pose information,and object category labels.And it is verified that the 3D-ResNet network can learn the mapping from 2D images to 3D structures by changing the method of dividing the dataset and constructing a new dataset.3.A scheme of using LovaszSoftmax loss function for model training is proposed,which can improve the effect of 3D reconstruction,and the evaluation index IoU value is higher than training the model using only the cross entropy loss function.The method proposed in this thesis has been evaluated quantitatively and qualitatively through experiments.The results show that these methods can effectively improve the performance of multi-view 3D object reconstruction based on deep learning.