Research On Image Stitching Algorithm Based On Grid Alignment And Image Fusion

As the application range of image stitching becomes wider and wider,image stitching in complex scenes has become a research hotspot.The current research direction is mainly divided into the traditional image processing algorithm direction and the deep learning algorithm direction.These algorithms are improved and innovated around the basic framework of image stitching.The basic frame of image stitching includes two stages: image alignment and image fusion.This thesis also proposes improvements to traditional image processing algorithms and deep learning algorithms around image alignment and image fusion.In the direction of traditional image processing,this thesis proposes two improvements.The image alignment model has evolved from the initial global homography matrix to a mesh deformation model subject to certain constraints.However,in scenes with uneven distribution of feature points,grid deformation also brings about the problem of picture distortion.Aiming at this problem,this thesis proposes a mesh deformation algorithm constrained by geometric structure.On the basis of point features,line segment features are introduced.On the one hand,line segment matching is used to achieve more accurate local alignment;on the other hand,the three-point collinear structure is used to maintain the linearity of the line segment structure during the grid deformation process,so as to maintain the original geometric structure of the image as much as possible.In the image fusion stage,the maximum flow and minimum cut solution model of seam has evolved from a smooth term based on the gradient of image pixel values ?to a smooth term based on perception.However,for scenes with large parallax,the seam may still be stitched and misaligned.In order to reduce the misalignment,this thesis proposes an iterative optimization algorithm for stitching.First,the structural similarity is introduced to define the degree of image block alignment.On the one hand,it can quantitatively evaluate the pros and cons of the seam,and on the other hand,it can update the smoothing term in the seam solution model to realize iterative optimization.At the same time,considering the degree of human eyes’ perception of texture strength,this thesis also uses adaptive saliency as the weight of the smoothing term to further improve the seam solution model.This thesis combines the improved mesh deformation algorithm with the improved seam algorithm,and proposes an iterative optimization algorithm for stitching based on mesh deformation alignment,so as to improve the final stitching quality.In the direction of deep learning,based on the video frame interpolation network RIFE(Real-Time Intermediate Flow Estimation for Video Frame Interpolation),this thesis proposes an image mosaic network RIFE-edge based on grid deformation coarse alignment and fusion weight map.This network is suitable for input images under general conditions and can output clear details.Stitched image.Deep learning is still in its infancy in the field of image stitching.Currently,the few representative works are LCAnet(Video Stitching for Linear Camera Arrays)and VFISnet(A view-free image stitching network based on global homography).However,the above two networks are subject to certain constraints and cannot directly process data commonly used in traditional algorithms.In order to build a more applicable network,this thesis first performs grid deformation and coarse alignment on the input image,overlays the aligned images with each other,as the input of the fusion network,and superimposes the two-way light through the fusion weight map output by the fusion network.The flow guides the deformed image to obtain a smooth transition of the stitched image.At the same time,in order to improve the quality of the stitched ?image,this thesis adds edge feature branches to optimize the details of the fused image and improve the definition of the overlapped area of?the fused image.