Research On Image Patching Algorithm Of Sample Block Based On Structural Sparsity

Digital image inpainting technique, which uses the known information to fill in missing region under some rules, aims to achieve a visually plausible result to prevent the inpainted region from easily observed by an ordinary unsuspecting person. The essence of this technique is to reconstruct missing information from the known part, which is not only the aim of computer vision, but also the problem involved in other image processing techniques. For computer vision, the study on image inpainting is of great significance in the aspects of theoretical research and practical application.With the feature of good repair performance on both small damaged region and large degraded region, the exemplar-based image inpainting algorithms obtain more and more attentions. This kind of inpainting methods can be divided into the matching-based algorithms and markov random field-based approaches. Starting with structure sparsity, this paper investigated how to use suitable prior information to guide the inpainting process in order to maintain the structure coherence and neighborhood consistency. Regarding to the two types of approaches, several novel methods are proposed. Moreover, the validity and applicability of the proposed algorithms are analyzed and verified from theoretical and experimental views. The main work and achievements of the paper are as follows.Regarding to matching-based algorithms, to maintain the structure coherence, neighborhood consistence and improve texture clarity, three algorithms are proposed based on the definition of structure sparsity function, matching criterion and local consistence, including:1. An adaptive image inpainting algorithm based on robust structure sparsity is proposed. The number of nonzero similarity between a patch and its neighboring known patches is used to define robust structure function in order to better measure the confidence of a patch located at structures. Then, the size of patch to be filled, neighborhood consistence weighted coefficient and local search window size are adaptively decided according to robust structure sparsity value, and the associated parameters are set via experimental analysis. The experimental results demonstrate the rationality of the defined robust structure sparsity and the proposed method can immensely improve the computational complexity.2. To improve the inpainting performance for structural and textural degraded images, the patch sparsity image inpainting algorithm combined color and gradient information is proposed. The gradient information is introduced to define weighted color-gradient distance to construct more appropriate matching criterion. On this basis, color-gradient structure sparsity function is constructed to maintain the structure coherence. Meanwhile, the local consistence constraint equation is constructed in both color and gradient spaces, and then multiple candidate patches searched according to weighted color-gradient distance are applied to represent the target patch sparsely under this constraint to maintain neighborhood consistence. The weight of color and gradient information and the number of candidate patches are adaptively decided according to experimental results.3. Inspired by superior performance of the super-wavelet transform and the aim to obtain more information to guide inpainting process, the patch sparsity image inpainting algorithm combining multi-direction features is proposed. The multi-direction features of the degraded image are inferred from super-wavelet transform, and then the features are combined with color information to measure the difference of two patches. Afterwards, a color-direction structure sparsity function is defined to measure the confidence of a patch located at structures Finally, the local consistence constraint equations are constructed in color and multi-direction spaces, which aims to make the sparse representation is consistent with target patch in color and direction features.On the other hand, because the matching-based methods can not well repair the degraded image with random textures, and error accumulation phenomenon exists in the approaches. To circumvent the drawbacks, starting with structure sparsity, two algorithms are proposed based on the suitable construction of whole energy optimization equation and appropriate selection of candidate labels in the MRF-based inpainting approaches, and they are:1. To construct the suitable whole energy optimization equation, an image completion algorithm combining multi-direction features is proposed. Multi-direction features are inferred from super-wavelet transform and the features are applied to measure data energy term and smooth energy term in order to construct whole energy constraint equation meeting human eye requirement. Afterwards, the extremum of whole energy is solved by multi-resolution methods. In the meantime, the patches size is adaptively decided on different resolution level, where the patches are applied to compute data term and smooth energy term respectively. Finally, graph cuts algorithm is applied to compute the extremum of whole energy to obtain the inpainted image.2. Starting with human vision, this paper investigated how to determine suitable prior information to guide inpainting process, and proposed the desired direction structure statistics for image completion. The multi-direction features are inferred from super-wavelet transform and the features are applied to extract the edge feature images. To obtain the structure information, the desired direction edge image is adaptively chosen according to the neighboring known characteristics of missing region. Afterwards, the offsets between two similar patches are counted in the selected direction edge image and non-edge image, and only a few dominant offsets are chosen as candidate labels to guide inpainting process. Finally, multi-label graph cuts algorithm is adopted to calculate the extremum of whole energy to obtain the inpainted image.