Research On Key Techniques Of Fast Stereo Matching For Photogrammetric Images

Stereo matching has always been a hot topic in the field of photogrammetry and computer vision.It aims to find the same image information between images in different views,which can be used as the structural restoration of multi-view images,the 3D surface data acquisition based on image,the making of digital surface model and digital elevation model,the reconstruction of real scene 3D model and so on.It plays an important role in the 3D reconstruction technology based on image.Under the background of explosive growth of data in the new century,the resolution of photogrammetric images increases gradually with the improvement of imaging technology.Among them,airborne images reach the level of millions to billions of pixels and close-range images to hundreds of thousands to megapixels.The improvement of their resolution makes the stereo matching algorithm need to improve its efficiency to meet the needs of high-resolution image applications.Aiming at the different data levels of two kinds of data,this paper studies two different stereo matching efficiency optimization techniques to meet the need of stereo matching efficiency improvement of various photogrammetric images.In the selection of basic stereo matching algorithm,this paper chooses Semi Global Matching(SGM)algorithm proposed by Hirschmuller in 2005 as the basic algorithm,which adopts the framework of global matching based on minimization of energy function.By aggregating the one-dimensional path cost of multiple path directions in the neighborhood to approximate the two-dimensional optimization,the efficiency of pixel by pixel stereo matching is greatly improved in SGM.However,in the face of high resolution images,the speed of SGM algorithm is still not able to meet the high efficiency requirements of applications,so further efficiency optimization based on SGM algorithm is the research direction of this paper.First of all,aiming at high-resolution aerial images,whose characteristics is that the resolution is generally more than ten million levels.The improvement of resolution will bring about the increase of parallax search range at the same time,so the stereo matching algorithm is not only inefficient in processing time,the memory usage is also very high.On the one hand,most of the applications involved in aerial images are offline processing,the stereo matching module does not require real-time performance,but requires the rapid acquisition of 3D dense point cloud,which offering fast and reliable input data for 3D reconstruction,real image mapping and other modules;On the other hand,the resolution of the image is too large and requires huge storage space.If the available storage space is too small,the algorithm needs to deal with the data in blocks,but too many blocks will reduce the accuracy because of few information of single block.Allow for the above two aspects,this paper proposes a systematic and efficient stereo matching algorithm optimization method based on CPU platform.The main work accomplished is as follows:(1)A very efficient parallel computing model based on the data parallel model is designed.The stereo image pair is divided into multiple stripe pairs according to the column number.Each stripe pair uses SGM algorithm to estimate the parallax independently,which is independent of each other and does not need to deal with any thread conflict.Finally,the parallax graphs of each strepe are fused into the whole parallax graph.(2)For the four steps of SGM algorithm:census transformation,cost calculation,cost aggregation and parallax calculation,the SSE(Streaming SIMD extensions)instruction set was used for instruction-level parallelism.By using the characteristics of single instruction and multiple data streams,parallel computation of multiple pixels or parallax is carried out at the same time and the data parallel operation besides multithread parallelism is realized.(3)A hierarchical SGM stereo matching model is proposed,which effectively solves the problem that too large range of parallax search in aerial images leads to serious time consuming and high memory occupancy in stereo matching.First of all,the image pair is sampled at multiple levels,and then matched from low resolution to high resolution.The parallax map of low resolution image pair matching provides the initial parallax for the next image pair and the parallax range of pixels is constrained to a very small value(no more than 64),which reduces the parallax search range of high-resolution image pairs effectively.The matching efficiency is greatly improved(tens of times of speedup),and the memory occupancy of the algorithm is obviously reduced.Secondly,for high resolution close-range images,the resolution is mostly in the order of hundreds of thousands to millions,which is generally smaller than that of aerial images.But the difference between close-range image and aerial image is that most of its applications require real-time,such as real-time vehicle three-dimensional navigation,motion robot three-dimensional navigation,hand-held three-dimensional scanner and other applications that require real-time acquisition of three-dimensional data.As the key algorithm of 3D information acquisition,the real-time efficiency of stereo matching is very important.Therefore a systematic and real-time stereo matching algorithm optimization method based on CUDA is proposed in this paper.The main tasks accomplished are as follows:(1)Aiming at the four most time-consuming steps of SGM,we design an efficient SGM parallel optimization model based on CUDA,making full use of parallel computing of thousands of cores in GPU,making full use of shared memory to optimize the depth storage efficiency of the algorithm.Through the cost spatial data arrangement of parallax main order,the problem of memory merge access in cost aggregation step is solved effectively.Experiments show that the real-time performance of the proposed algorithm is much higher than that of other optimization algorithms,which is 576 times faster than the original SGM algorithm.For 640 x 480 resolution images,the real-time efficiency of 166.7 fps can be achieved on NVIDIA GTX 1070 GPU when the parallax range is 128.(2)A hierarchical matching model based on CUDA is proposed,which combines the pyramid hierarchical matching strategy with the parallel algorithm based on CUDA to effectively reduce the parallax search range and further improve the real-time efficiency of the algorithm.Experiments show that the real-time performance of the algorithm is excellent for high-resolution close-range images.For close-range images with 1800 × 1500 dimensions,the real-time stereo matching efficiency of 26.9 fps is achieved when the parallax range is 256.