Non-Rigid Image Alignment Based On Feature Matching

Due to the improvement of the industrial production efficiency,as well as the development of optical imaging and sensor technology in recent years,machine vision has replaced artifical vision gradually and been widely applied in many industrial fields,in which multiple related images are usually required to obtain more comprehensive information.Image alignment is a technique of finding transformation relations among images captured from different views,at different times or by different sensors,and can be applied in image stitching,object detection,pose recognition,video surveillance and so on.Researchers have studied some subproblems for different applications in depth,including feature extraction and matching,transform model estimation,image resampling and warping.However,in many real-world scenarios and practical industrial electronics assembly,it is a challenging task because of some unconstrained factors,such as illumination change,noise,scale change,occlusion,non-rigid deformation,discontinuous motion,blur and low texture.To handle these complex situations and unsolved problems in current researches,in this paper,taking advantages of the sparsity and stability of feature matching,we study image similarity measure,feature selection,transform model estimation,deformation constraint and some other issues from two aspects: template-based scene analysis and multi-view analysis.Then,we propose several feature-based image alignment methods.In real-world scenarios,proposed methods are applied to object localization,image alignment and stitching.In electronics assembly,proposed methods are applied to component positioning,alignment and stitching of printed circuit board images.The main research contents are summarized as follows:To handle complex factors,such as non-rigid deformation,occlusion,blur and background clutter in template matching scenarios,a similarity measure,which is more robust to outliers,is proposed.The appearance-based one-directional nearest neighbor(NN)matching of patches between the template and the candidate window is constructed and the smallest location distance between each point in the template and its matching points is employed to penalize the deformation explicitly.Then,the weights are added to points in the template box to suppress the negative effects to the measure of background pixels.In addition,the statistical analysis is provided to illustrate the proposed measure can capture the similarity of two distributions.To address the problem of scale changes of the object between the template and the target image in complex scenarios,a scale-adaptive similarity measure for robust template matching is proposed.To discover the effect of scale to the one-directional NN matching of patches between the template and the candidate window,the probability of a patch to be chosen as an NN match and the expectation of the Diversity Similarity is derived by probabilistic analysis.Then,an NN-based scale-adaptive measure is provided.Moreover,the measure is extended to be suitable for both rectangular and masked template.In addition,an efficient scheme for multi-scale template matching is given.Furthermore,the statistical analysis is presented to illustrate the proposed measure has the properties to capture the similarity of two distributions and be invariant to scale changes.To preserve the flatness of planes and the straightness of lines in the real-world scene for multi-view image stitching,a natural method with layered warping constraint is proposed.First,feature matches are grouped into layers using the random sample consensus algorithm.Each layer describes a scene plane.Then,an outlier rejection method relied on grouping result and local residual errors is presented to improve the quality of matches for good estimation.Next,according to the epipolar constraint of corresponding points,intrinsic and extrinsic parameters of the camera,as well as the normal vectors of planes are estimated.In addition,a final view for stitching is selected to compromise the warping degrees of all images.Finally,the mesh warping model is solved to achieve accurate alignment and natural-looking stitching result.To handle illumination change,color variation and low texture in images,a parametric chamfer alignment method is proposed.First,edges with different gradient magnitudes are detected through K-means cluster.Then,parameters of the mesh warping model are estimated by optimizing an object function,which measures the chamfer distance of edges and the smoothness of warping,in which weights are attached to different levels of edge points.The optimization can be solved by the gradient descent method.In addition,after the warping model is initialized by feature-based alignment,a growing technique of refining the model is presented.An image alignment method using mixed mesh warping model for discontinuous deformations is proposed.Given a set of corresponding feature points between images,variational Bayesian linear regression framework is adopted to estimate mixture models and determine the number of components.To improve the alignment accuracy,warping model of each component is refined separately by minimizing the photometric error.Then,all pixels in the overlapping region are assigned to one of the components iteratively in terms of the photometric similarity,the distribution of feature points and the consistency of adjacent pixels,which contributes to the final prediction of warping.