Image Based 3D Surface Reconstruction Using Gradient Constraints

Imagery is one of the main data source for 3D reconstruction of natural terrain and man-made objects.Related principle and have been studied long and profoundly in photometry and computer vision areas,which yields large amount of practical methods.However,with the development of practical application,classical theories of image-based 3D reconstruction present lack of adaptability faced,since they use lq norm constraint on first-order gradient as regularity constraint.In these cases,shape gradients become close to zero,which reinforces the fronto-parallal property and produces local horizontal planes.Since real objects or terrains poccess various shape properties,existing theories fail to yield accurate and realistic depiction of them.Around the problem above,we build a novel generalized optimization model for image-based 3D reconstruction.The model composes of data constraint and regularity constraint.The dataconstraint unifies practical problems suchlike stereo matching and shape refinement in an identical framework while the novel regularity constraint improves model’s abilities for shape depiction.We proposed a lq norm constraint of second-order gradient in the novel regularity constraint.It merely constrains spatical variation of shape gradient to produce local constancy or smooth variation,which allow more flexible shape variation.In consideration of some auxiliary observation on first-order gradient in some applications,the novel regularity constraint can include extra constraint on the value of first-order gradient,which not only enhance the robustness of model but also improve reality of reconstructed shape.The proposed model serves as the common theoretical foundation for the whole research.We will carry out detailed discussion concerning the property of urban scenes and terrain scenes.Since the optimization of above model relies on reiable initial solution,we explore the robust matching metric function for binocular stereo matching.In practice,image based 3D reconstruction often uses binocular stereo matching alogrithm to conduct fast initialization.We introduce a novel semi-global matching algorithm named mSGM which adopts pyramid strategy and multi-metric fusion.It produces high-precision matching result but requires long optimization time.If optimization can be avoided with little reduction in accuracy,rapid acquisition of initial 3D value or fast generation of 3D overview would be affordable.For this sake,we attempt to improve the robustness of matching metric function.We design a convolutional network model with sequent convolutions to extract robust matching features for matching metric function construction.Joining these works,we use mSGM algorithm to produce high-quality matching result for parameter training and accuracy assessment of convolutional neural network.We find that the new matching metric maintains higher accuracy and robustness in regard of intensity difference and geometric disturbance though no complex optimization is imposed.For city scenes,we instantialize the generalized optimizaiton model and develop practical solution.3D reconstruction of complex urban scenes is mainly achieved by multi-view stereo matching.Suchlike scenes consist of typical piecewise slant planes which are mathematically characterized with local constancy of gradient.To deal with these 3D reconstruction problems,we adopt matching metric function as data constraint and l1 norm of second-order gradient as regularity constraint in our proposed general model,leading to a total generalized variation stereo matching model.In the solving stage,we relax the original model to an approximate form.Intensity adaptive guiding tensor is simultaneously introduced to suppress over-smoothing on edges from regularity.We refer to Taylor expansion and primal-dual convex optimization theory to derive an iterative optimization algorithm.Algorithm efficiency is further improved with semi-global matching initialization.Experiments are carried out on public test dataset of aerial photogrammetric images and our own dataset of aerial oblique images.The proposed model and method can achieve higher accuracy while effectively enhance slant plane structures.For terrain scenes,we also instantialize the generalized optimizaiton model and develop practical solution.Natural terrain has smooth shape and presents shading variation in image.For the need of large-area and low-cost 3D reconstruction,we have exploration on refinement of initial low-resolution terrain shape.In our proposed general model,we adopt elevation deviation as data constraint,and l2 norm of second-order gradient together with radiometric consistency of first-order gradient based on image-embedded shading variation as regularity constraint,leading to a shading-guided shape refinement model.Again,we relax the model to an approximate one for effective solving.First by modeling radiometric transferring process we propose the method for shading estimation based on least square method and gradient descending.Then we decompose the original problem to two subproblems,which are solved by variational Euler-Lagrange function and Taylor expansion followed by iterative large-scale linear equations solving.Experiments are carried out on simulated and realistic datasets.The proposed model and method can achieve higher accuracy,more details and stronger reality.