Key Technologies For3D Digital City Reconstruction Using Oblique Images

Oblique photogrammetry technology has been a hot area of research for several years. Compared with the traditional vertical aerial photogrammetry, the oblique photogrammetry technology can obtain the the facade information of the objects on the ground more effectively and one aera can be surveyed from different viewpoints many times. Moreover, the oblique images are more suitable for human visual perception. Nowadays, oblique images are widely used in3D digital city reconstruction, city management and planning, and mapping services. And the oblique imagery has become a standard technology for civil applications, thanks to the development of airborne digital cameras and in particular the development of multi-camera systems, as proposed by many companies (Leica RCD30, Pictometry, Midas, BlomOblique, IGI, UltraCam Osprey, etc.).With the increasing requirement of the3D city model, software manufacturers all over the world commit themselves to providing3D models. Recently, large scale efforts from large enterprises like Microsoft, Google, are underway, aiming at building a virtual equivalent of our planet including realistic models of thousands of cities worldside. The enormous scale of such a project, encompassing tens of thousands of the largest cities worldside, makes any manual attempt to solve the arising problems prohibitively expensive. The only way to tacle such an undertaking is by a fully automatic processing pipeline.Oblique photogrammetry technology can obtain the vertical and facade texure of the ground objects at the same time. In orther words, oblique images have the dual advantage of traditional vertical aerial photos and the terrestrial images. Therefore, oblique images are very suitable for3D reconstruction. However, due to the large tilt angle, the oblique images suffer from the occlusion and varying scale.And because of the two key reasons; the automatic image matching tools designed for vertical images fail and produce a lot of blunders, when they are applied on oblique images. Therefore, this paper proposed an automatical processing pipeline to sovle the problem of reconstruction of the3D digital city using oblique images. And a new mapping method is proposed to obtain the vector data and single object models on orthophotos, which uses a new elevation model called Orthoimage Elevation Synchronous Model (OESM), proposed in this paper.The automatic processing pipeline can be divided into two steps. Firstly, in this paper, a fully automatic methodology for simultaneous orientation of oblique and nadir images is described. The method uses the POS data to evaluate a connectivity graph. Then "Virtual horizon photos" are created for solvering the varying scale problem of oblique images. By using the "Virtual horizon photos", tie points between the oblique images and vertical images are successly obtained. Five points based relative orienatation method combining with ransac algorithm is used to remove blunders. To improve the result of aerial triangulation, a hierarchical outlier detection method for point cloud data is applied, which can effectively and automatically detect outliers remaining in the tie poins whitout manual editing. This method is different from the traditional method based on the reliability theory. It can detect2-fold outliers near the epipolar line, which cannot find by using reliability theory. And traditional method is useful to detect the so called small and medium outliers effectivey, but for large outliers it works badly. The outlier detect method described in this paper could handler the larger and small outliers at the same time, and it can also find small ground outliers. Tests in this paper show that, this method can find outliers effectively, improve the AT result. Finally, after using the outlier detect method, the mean square reprojection error is below0.5pixel, which is only1/3of the result without using this method. Two contrast experiments are setted to test the performance of the whole AT method. The first contrast experiment is made between method in this paper and the ASIFT method. The result shows that method in this paper is better than the ASIFT method, the mean squre reprojection error of the ASIFT is3times of my method, while the mean squre epipolar error is2.5times. And the total consume time of my method is only1/12000of the ASIFT. Compared with the VisualSFM (a famous open source Aerial Triangulation software) the number of tie points obtained by my method is3.7times of VisualSFM. Though the mean squre reprojection error of both two methods are close, yet the the mean squre epipolar error of my method is only1/3of VisualSFM.The second step of our pipeline is an automatical image based modeling methods. This paper proposed a highly automatical pipeline to obtain the surface of the objects from the oblique and vertical images. At the beginning, this paper proposed a method to select stereo models automatically; then the AT result is used to estimate the overlap are of the stereo models; after that a modified Semi-Global method is used. The main improvements consist of following aspects. First, AT results are used to estimate the overlap area between the stereo pairs automatically, then the Census matching method is used to obtain a more reliable initial disparity map. Then, the8-direction scanline optimization is integrated into two loopings. To solve huge memory usage problem of the classical SGM method and overcome the shortcoming of tile SGM method when is used in the big oblique images, a new dynamic disparity searching method is used, which is determined from the previous result using the previous disparity of each pixel and adding a buffer. The contrast experiment between the modified matching algorithm and the PMVS method, shows that the completeness and visualizations of result of my method are both better than the PMVS method.Finally, in order to effectively obtain the objects shape information and building models, which are very necessary for GIS system, this paper proposed a new orthophoto based stereo mapping method, which is different from the traditional mapping method. The new method uses a new elevation model defined as orthoimage elevation synchronous model(OESM) to obtain the true elevation of every pixel in an orthoimage derived from digital elevation model(DEM) and digital surface model(DSM), retrieving the real world scene. The OESM makes it possible to directly collect the3D information of non-ground objects without the tedious stereo plotting. Using OESM, we can get the Vector information and the single and simple object modelling at the same time, which is very useful to reconstruct the simple3D city environment quickly. Compared with the model reconstructed from oblique images, models obtained from this new mapping method consist of single models, while the former is just a good3D mesh.The workflow described in this paper is highly automatically, and obtain reliable result. Moreover, the orthophoto based mapping method is irrelevant to platform and image sources. It is a highly promising approach.