Study On The Automatic Construction Method Of Mesh Model Based On The Oblique Images

Oblique aerial photography is a new aerial photography technology which acquires information of urban scene from multiple perspectives using oblique cameras.Oblique photography overcomes the limit of the traditional aerial photography which can only view objects from the vertical view.In this case,oblique photography can obtain more ground information and reflect the more real situation,which makes up the weakness of the vertical photography and intuitively provides a real world according with the human vision.Oblique photography has been widely used in large scale topographical mapping,texture collection of three-dimensional models,and rapid 3D modeling of the digital city,etc.Different from the vertical aerial photography,oblique aerial photography has two types of imaging modes including fixed-angle imaging,such as SWDC-5 camera,and sweeping imaging,such as A3 camera.From the perspective of a single image,the oblique photography has the following characteristics:? it can acquire images with multiple perspectives and get more detailed ground information.? It has relatively high resolution and large field of view.? The multiple images covering the same features have different resolution.? The phenomenon of object occlusion is relatively obvious.From the perspective of imaging model,the oblique photography is characterized by the following features:? it can obtain data rapidly and reflect the dynamic changes of the study area,which benefits real-time data updates.? The overlap of oblique aerial photography is greater than that of the traditional aerial photography.The more lens and larger overlaps lead to a mass of photos.? It can acquired abundant ground information which shows more details on structure and texture features of each building facade,reduces visual blind spots,and provides complete data of 3D modeling.Nowadays there are many foreign companies can provide the software from image aerial triangulation to urban 3D model,Sweden's C3 Technology,France's Smart3D,Switzerland's Pixe14D and France's Street Factory is included.Smart3D has been widely used in China,and can generate a realistic three-dimensional model.But it still has some problems:? the dense matching data are inaccurate and incomplete;? the edge features of mesh models are not prominent;? the building surface is deformed;?and the 3D model have holes etc.Therefore,it stills a difficult task that automatically constructs 3D mesh model based on the oblique images.We transform the reconstruction problem from image based to point cloud based.First,point cloud is generated by image dense matching by overcoming the influence of large perspective deformation.Then a 2-manifold triangular mesh model is generated by point cloud.Finally,the mesh model is refined by original images.The main work of this paper is as follows:1)We discussed the difficulties of 3D reconstruction based on the oblique images,introduced the basic knowledge of oblique image 3D reconstruction,discussed the key technology of oblique image 3D reconstruction,and deeply analyzed the problems of existing 3D reconstruction algorithm.2)We proposed a new epipolar rectification algorithm of minimum perspective deformation for oblique images.On this basis,we achieved a dense matching strategy considering about the object plane,which overcome the effect of perspective distortion to image dense matching.Epipolar rectification is an essential step in 3D reconstruction.In this paper,we study how to generate the epipolar image with minimal distortion,and reduce the influence on dense matching to the maximum extent.There is a large amount of horizontal and vertical planes in urban area,which can be used to generate the horizontal epipolar images,the vertical epipolar images and the epipolar images with minimum deformation relative to the original images.The last step is to fuse the dense matching results of above three kinds of epipolar images to integrate an optimal dense matching depth map.3)we proposed a depth map fusion method of virtual views,which fuses the overlapping depth map into a non-redundant point cloud and improves the precision of the point cloud.The modeling is of computational complexity and memory consumption,because of massive and redundant oblique image data.The surfaces are covered with multi-level point cloud that also have some errors between them,which means inconsistency between the depth maps.The depth map fusion can remove the point cloud noise,enhance the consistency of the depth map,and improve the precision of point cloud.The frequently-used depth map fusion methods are SURE and OpenMVS,taking each piece of the original image as the reference image,which can eliminate the inconsistency between the depth maps,but cannot remove the redundant information and can cause large computation.Therefore,it is an important task to reasonably select the reference image,avoid redundant computation,and ensure the reference image can cover the whole scenes.By defining a limited number of virtual views,we can reduce redundancy as much as possible,fuse the depth maps in the virtual view,and achieve the 3D reconstruction.4)we proposed a mesh refine method with a single parameter constrained in the normal vector direction,which use the original image information to refine the mesh to improve the accuracy.The mesh model constructed by the graph cut method has the advantages of anti-noise ability,suitable for large scale and complex scene.However,the constructed mesh model does not satisfy 2-manifold features.On the basis of the graph cut method,this paper adopts the vertex relabeling algorithm to construct 2-manifold mesh,which is convenient for subsequent simplification and optimization.The mesh model constructed by the point cloud is usually reconstruct an approximate surface,which has some problems of local deformation,weak edge features and inconsistent density.Therefore,the constructed mesh is needed to refine by using the original image information to make up the information loss in the process of reconstruction.According to the principle of minimum matching cost,the original images and the re-projection introduced by mesh are used to refine the reconstructed mesh.We refine the vertices of the mesh,and constraint its evolution direction on thenormal vector to reduce the degree of freedom to 1.The Gauss curvature is as the regularization term,with anisotropic characteristics,and can preserve the edge features in the evolution process.