Dense Image Matching And Its Application Based On Optical Flow Field

Generating three-dimensional models from two-dimensional image data is one of the important research contents of photogrammetry and remote sensing,and dense matching is one of the most critical technologies.In recent years,dense image matching methods have been a hot research topic in the field of photogrammetry and remote sensing.Some dense matching algorithms(such as SGM,PMVS,etc.)have been successfully applied to some commercial software.However,with the development of hardware technology,more and more large-format,high-resolution aerial photography and large deflection angles,low-altitude drone aerial photography data with uneven lighting conditions are applied to real 3D mapping works,and these images are given more challenges to traditional dense matching methods.Furthermore,due to the rapid development of the urban economy,there is an increasing demand for the update of existing3 D topographic mapping data.In the face of large-format aerial photography,traditional methods cannot effectively generate pixel-by-pixel dense matching point clouds.They can only generate digital elevation models(DEM)indirectly through digital line graphics(DLG),which collected manually and then generate digital ortho maps(DOM)through patch rectification,which greatly reduces the production efficiency of 3D mapping products.Besides,although the image dense matching algorithm has been widely used in 3D reconstruction scenes,most of them follow the strategy of computer vision,and few people evaluate the quality of dense image matching from the perspective of photogrammetry and remote sensing.To this end,this paper aims to study the current representative dense image matching algorithms systematically and to utilize the advantages of different methods to solve the above problem that occurred in real photogrammetry application.The main work and innovations are summarized as follows:(1)This paper proposes a theoretical framework and overall technical solution of aerial image dense matching based on the optical flow field.The essence of aerial image matching is to reconstruct the geometric relationship of the stereo image pair through the radiation information and obtain the relevant information of the image correspondences.Therefore,how to quickly and effectively find the pixel-by-pixel correspondences in the overlapping area of the stereo pair is the research focus of this article.In this paper,the optical flow can quickly and efficiently track the movement trajectory of each pixel in the overlapping area of the stereo pair.The use of optical flow provides a high-quality initial value for the pixel-wise fine matching algorithm,thereby reducing a large number of redundant search and invalid calculations to improve the efficiency of dense image matching.(2)A multi-level B-spline optical flow field interpolation method based on triangular network constraints is proposed in this paper.Firstly,the high-precision sparse image tie points obtained by feature matching are used as control points to construct a triangulation and B-spline interpolation grid.In order to generate a close to the real-world surface interpolation grid.The weight of each point in the interpolation grid is self-adaptive by measuring the geometric distance between the interpolation grid and the triangulated grid.After the adapted interpolation grid is generated,a coarse-to-fine strategy is utilized for reducing interpolation error.Experiments show that for a pixel-by-pixel dense optical flow field of a true-color aerial stereo pair with a 100-megapixel format and 60% overlap,it takes only 14 s and coarse matching to consume only 20 s of CPU.The pixel-by-pixel matching point cloud can be directly used in the production of fast image puzzles.Its plane accuracy is better than ± 0.266 m on the ground,that is,± 3.8 GSD,which provides new technical means for emergency mapping and aerial photography quality inspection.(3)A fast guided filter baed fine matching algorithm is proposed in this paper.Considering that the matching result generated by the B-spline appears to be excessively smooth at the edge of the ground objects and the disparity discontinuity region,how to preserve the details of the ground objects effectively is the key to the fine matching algorithm.Since the corresponded objects usually show a high degree of similarity in pixel color intensity,in order to improve the dense matching accuracy and reduce unnecessary redundant calculations,the traditional rectangular search window is replaced by a color aware nonregular window.The refined correspondence is determined with the lowest matching cost by calculating the matching cost aggregation of the and the fast guide filter baed census cost.A large number of experiments show that the precision of the densely matched point cloud after the refinement reaches sub-pixel level,and the actual accuracy of the point cloud is better than ± 3.5 GSD.The precision of dense and coarse matching point clouds has been significantly improved.Moreover,the pixel-by-pixel fine-matching CPU time for a true color aerial stereo pair with a 100-megapixel format and 60% overlap requires an average of154.5 s,showing good application feasibility.(4)Based on the above theoretical research results,a complete set of algorithms for dense matching of aerial stereoscopic based on the optical flow field was realized in the environment of Microsoft Visual Studio 2015.Meanwhile,a set of indicators such as matching success rate,matching efficiency,and reprojection error are utilized to evaluate the point cloud quality of the proposed method.A large number of experiments are conducted on three different datasets,namely the Beijing dataset,Vahingen dataset,and Tokyo dataset.The Beijing dataset involves of44 low-altitude aerial photographs taken by the UAV platform,the Vaingen dataset involves of 14 DMC pseudo-color composite aerial photography of 105-megapixel size,and the Tokyo dataset involve of 48 images of 195-megapixel size.The comparison experiment with t SGM and PMVS show the advantage and limitations of the proposed methods and the widely used methods.The dense three-dimensional point cloud generated by the proposed algorithm is directly used in the production of DSM,DOM,and image alignment.The generated results on Beijing datasets achieved ± 0.227 m on DSM elevation accuracy reached,better than ± 3.5 GSD in-ground accuracy,and the DOM mosaicing accuracy achieved ± 0.070 m,better than ± 1.0 GSD.This accuracy fully meets the accuracy requirements of the current Chinese basic geographic information digital achievements for the 1: 500 scale DEM and DOM(the error in the DEM second elevation is better than ± 0.25 m,the error in the DOM plane is better than ± 0.30 m),which reveals that the aerial dense matching method based on optical flow field proposed in this paper has an applicability potential in emergency mapping and basic geospatial information extraction.