Research On Proactive 3D Reconstruction Of Large Scale Indoor Changing Scenes

Scanning and 3D reconstruction of real-world scenes are crucial in the fields of computer vision and computer graphics,and play a key role in achieving scientific and technological advancements such as digital twins and metaverse.Camera localization that is both convenient and robust serves as the foundation of scene reconstruction,while complete and efficient data acquisition ensures high-quality reconstruction.On the one hand,machine learning techniques have greatly improved the accuracy and inference efficiency of camera localization-However,most of the existing algorithms require a considerable amount of training data and time,and hardly handle changes in the scene.On the other hand,a series of proactive scanning and reconstruction methods have been developed to enhance the quality of reconstruction by using reconstruction outcomes to guide data acquisition.However,the efficiency of current proactive scanning methods in reconstructing large-scale scenes is still low.Regarding the aforementioned problems,this thesis focuses on studying how to conveniently deploy camera localization algorithms,robustly localize cameras in changing scenes,and efficiently scan and reconstruct large-scale scenes.Firstly,this thesis improves the deployment convenience of camera localization algorithms through fast training with few-shot databases.Secondly,this thesis enhances the robustness of camera localization in changing scenes by actively sensing scene changes.Finally,this thesis achieves proactive scanning and reconstruction of large-scale scenes through multi-robot collaboration,with high quality and efficiency.The innovations and contributions of this thesis mainly include the following aspects:(1)Camera localization via fast training on few-shot databasesThis thesis proposes a decoupled learning framework based on scene coordinate regression model,which improves different modules using methods such as pre-training,discretization,and meta-learning,reducing the algorithm’s demand for training data volume,accelerating the algorithm’s training convergence speed,achieving fast training on few-shot databases,and ultimately improving the deployment convenience of the camera localization algorithm.Experimental results show that in small-scale scenes,the algorithm can complete training with less than 50 frames of training images and about three minutes of training time,achieving better localization accuracy than the state-of-the-art algorithms at that time.In addition,the algorithm is also suitable for outdoor scenes.(2)Camera localization with active perception on scene changesThis thesis proposes an outlier-aware neural decision tree,which combines decision trees and neural networks,and designs a unique outlier-perception module to deepen the algorithm’s understanding of the scene context.The algorithm only establishes the correspondence between image pixels and scene points within the area inferred as static,thereby reducing the impact of changes in the scene on camera pose estimation and ultimately enhancing the robustness of camera localization algorithms in changing scenes.Experimental results show that in changing scenes,the localization accuracy of the algorithm is about 30%higher than that of the most advanced algorithm at that time.In static scenes,the algorithm also achieves accuracy comparable to the most advanced algorithm at that time.(3)Proactive scanning for multi-robot collaborative reconstruction of unknown scenesThis thesis proposes a proactive scanning and reconstruction method for large-scale scenes through multi-robot collaboration.The collaborative scanning problem is abstracted as a dynamic task assignment problem,formulated and approximately solved based on the optimal mass transmission model.Combined with path planning,trajectory optimization,and robot motion control,a multi-robot collaborative scanning system is ultimately formed.For the first time,this system leverages multi-robot collaboration to proactively scan and reconstruct large-scale scenes,achieving high-quality and efficient data acquisition.Experimental results show that the system achieves higher efficiency compared to the most advanced scanning strategies at that time,while ensuring equivalent or higher reconstruction quality.In addition,the system is robust to changes in the numbers and initial poses of robots,and it is generalizable to different types and scales of scenes.