Research On High-precision 3D Reconstruction Method Of Irregular Objects Based On Computer Vision

With the rapid development of robotics,autonomous driving,virtual reality,and augmented reality in recent years,the demand for computer perception and interaction with three-dimensional environments is increasing.Three-dimensional reconstruction is an important means for computers to understand the three-dimensional world.However,traditional three-dimensional reconstruction relies on hardware devices such as laser scanners and binocular cameras,with laser scanners often being expensive and binocular cameras being sensitive to environmental lighting.The structure-from-motion algorithm lacks certain robustness in certain situations.In contrast,computer vision-based 3D reconstruction methods balance economic cost and certain robustness.In this thesis,we study irregular object high-precision 3D reconstruction from three perspectives: 1)Firstly,we propose a 3D-VGT method for reconstructing irregular objects using multi-view reconstruction.The 3D variational autoencoder can embed the 3D model into a continuous latent space and form a 3D model manifold in the latent space.We experimentally and theoretically verify the continuity of the latent space.The2 D image encoder(Vision Transformers)in this method can fuse 2D images of the same irregular object from different angles to form 2D image latent variables.Borrowing the idea of adversarial learning,we accurately embed the 2D image latent variables into the3 D model manifold,establishing a connection between 2D images and 3D models.This method can ensure the geometric accuracy of reconstructed 3D models to a certain extent.We conduct multiple comparative experiments on the virtual dataset Shape Net and experimentally validate that compared with some deep learning-based 3D reconstruction methods,this method has certain advantages in terms of geometric accuracy;2)Secondly,we propose a neural implicit representation-based 3D reconstruction method.This method uses the strong interpolation ability of neural networks to express rich information in the real environment,ensuring geometric accuracy and appearance accuracy(color and light)to a certain extent.In addition,we also explore the impact of the range of volume rendering used in neural implicit representation training on sampling points.Provide a strategy for constraining the sampling range.We conduct multiple comparative experiments on the real dataset LLFF and a self-made dataset and experimentally validate that this method has certain advantages in terms of geometric accuracy and appearance accuracy compared to precision 3D reconstruction methods;3)Finally,we combine the semantic segmentation method with the neural implicit representation-based 3D reconstruction method,enabling the neural implicit representation to perceive the semantic information of the environment and solving the problem of interference caused by the background when reconstructing irregular objects in the real world.At the same time,we experimentally validate that the UAGAN semantic segmentation method proposed in this thesis has high accuracy characteristics.In this thesis,we conduct research on 3D reconstruction methods from different perspectives and conduct experiments in virtual and real datasets.We have discovered and solved the problem of background interference in 3D reconstruction methods in practical applications.