Research On Dense Matching Method Of UAV Aerial Stereo Image Based On Siamese Network

The development of photogrammetry technology has become one of the main means for government departments and enterprises to obtain reliable 3D geographic information data.The dense matching method is the key step of photogrammetry to reconstruct 3D space from 2D image.The quality of matching results directly determines the accuracy of the subsequent 3D reconstruction.With the development of Intelligent City,digital twin city and automatic driving technology,people put forward higher requirements for the accuracy and efficiency of point cloud and 4D products.At present,although the traditional dense matching method has achieved good results and applications,there are still problems in dealing with weak texture and repeated texture,and the emergence of deep learning method provides a new way for the development of dense matching.Compared with the traditional methods,the deep learning method takes data as the driving method and neural network model as the support,which has good prediction ability for the matching results,and has faster computing ability with the support of CPU + GPU.Therefore,this paper takes UAV aerial image as the research object,aiming at the many kinds of mismatching problems in the dense matching results,a dense matching method of UAV aerial stereo image based on deep learning is proposed.The main research contents and achievements are as follows:(1)Referring to the existing deep learning algorithm,a deep neural network model suitable for dense matching of UAV aerial stereo images is constructed based on twin neural network and practical application scenarios.The model adopts the idea of pixel by pixel translation matching of image blocks,extracts feature vectors from the left and right images input by two branch networks,calculates the inner product,and finally outputs the matching cost volume within the disparity search range,which is used to calculate the visual difference of each pixel.The existing aerial stereo image pairs and real disparity map are used as training data,and the network model parameters are updated by supervised learning.The idea of pixel by pixel translation matching of image blocks and the strategy of inner product layer instead of full connection layer make the original binary classification problem into a multi classification problem,which has a unique optimal solution,improves the reliability of prediction results and effectively avoids the occurrence of false matching.(2)Aiming at the problem of mismatching repeated/weak texture regions in traditional dense matching methods,as well as the edge expansion phenomenon of the results of deep learning dense matching methods,and the problems of unsmooth parallax in part of the flat area,a method of using the color information and texture feature information of ground objects is proposed.The method of adaptive optimization of the matching results solves the above problems: through the prior knowledge that the same feature in the aerial image has similar color information in the left and right images,the mismatch of the occluded area such as the edge of the building is constrained;According to the image texture feature information,adaptively selecting the penalty parameters for matching cost aggregation can effectively improve the matching results.(3)The public KITTI2015 data set and WHU MVS/Stereo aerial imagery of a certain area in Guizhou Province are used as verification data,and the KITTI2015 accuracy evaluation index is used to quantitatively compare and qualitatively evaluate a variety of traditional and existing deep learning intensive matching methods;then use the method in this article The acquired disparity map is reconstructed in three dimensions,and the point cloud results are compared with commercial real-world modeling software.The results show that the method proposed in this paper not only improves the matching accuracy of each pixel,but also can better deal with the problems of repeated/weak texture regions and edge expansion,and obtain a more excellent and smooth disparity map,and the reconstruction effect of the three-dimensional point cloud is also Has certain advantages(4)Using the Python programming language,combined with the Tensor Flow deep learning framework,Opencv image processing library and Py Qt5 user graphical interface components,a prototype system for dense matching of drone aerial images based on deep learning is realized.