Research On Mural Inpainting Algorithm Based On Sparse Representation And Deep Learning

Mural painting occupies an important position in the history of Chinese art,with high artistic and cultural value.As the largest existing ancient mural resources in the world,Dunhuang murals are the treasures of China and even the world’s art palaces.However,due to the long history of murals and the interference of natural climate and man-made destruction,most murals have suffered from pathological problems such as breakage,peeling,and cracks,which require urgent protection.Therefore,research on the restoration technology of diseased murals is extremely important.Currently,the main method of repairing damaged murals is manual repair,but manual repair has the problems of long cycle,high risk,and irreversible results,which become a major obstacle to the repair of murals.Therefore,combining mural repair with computer technology and using digital technology to assist in virtual repair of murals can not only provide important and practical reference value for manual repair,but also digitize and long-term preserve the repaired murals.This thesis deeply studies the current research status of image restoration at home and abroad,and through the exploration and research of sparse representation and deep learning related image restoration algorithms,it mainly proposes the following three aspects of mural restoration algorithms: mural inpainting based on RPCA decomposition of block nuclear norm and entropy weighted clustering sparse representation、multi-scale reconstruction of mural inpainting based on generative adversarial enhanced by joint dual encoders and mural inpainting generative adversarial networks based on structural gated and texture joint guidance.The main research and contributions are as follows:(1)In the process of repairing murals using traditional sparse representation algorithms,the separation of structural information and texture information is not thorough,and the dictionary design is single,resulting in incomplete content restoration and boundary effects in the repair results of mural images,a mural inpainting method based on RPCA decomposition of block nuclear norm and entropy weighted clustering sparse representation is proposed.Firstly,the proposed RPCA image decomposition algorithm based on block kernel norm is used to decompose the mural image into a structural layer and a texture layer.Then,an entropy weighted k-means method is proposed to cluster the structure layer image and construct sparse sub-cluster dictionaries,and reconstruct the structure layer image using the class sparse repair method.Then,the texture layer image is repaired by using the bicubic interpolation algorithm.Finally,the repaired structural layer and texture layer are fused to obtain the repair results of the murals.Through digital restoration of real Dunhuang murals,the experimental results show that the proposed algorithm completely repairs damaged areas,and the objective quantitative evaluation is superior to the comparison algorithms.(2)Aiming at the problems of weak feature perception ability and loss of reconstruction details when the existing depth learning algorithms repair damaged mural images,a multi-scale reconstruction of mural inpainting based on generative adversarial enhanced by joint dual encoders is proposed.Firstly,a generation network consisting of a dual-branch joint encoder and a multi-scale decoder is designed.The dual-branch joint encoder is divided into a gated encoder branch and a standard encoder branch.The gated encoder branch utilizes the dynamic feature selection mechanism of gated convolution and cascades empty convolution to improve the semantic feature perception ability of murals.The standard encoder branch utilizes standard convolution and introduces a dense connection structure to obtain richer mural information.Then,a multi-scale decoder is used for image reconstruction to enhance the reconstruction ability of damaged mural texture details.Finally,the purpose of mural restoration is achieved by using spectral-normalized Patch GAN discriminant model to distinguish between true and false.The experimental results of digital restoration of real Dunhuang murals show that the proposed algorithm is superior to the comparison algorithms in quantitative evaluation such as peak signal-to-noise ratio and structural similarity index measurement.(3)Aiming at the lack of joint constraint guidance of prior information such as structure and texture in the process of repairing murals using existing deep learning repair methods,resulting in structural disorder and texture blurring in the repair results,a generative adversarial mural restoration model jointly guided by structure gated fusion and texture is proposed.Firstly,a generation network consisting of a structure guided encoding sub network and texture guided decoding sub network is constructed.Feature encoding is guided by structural information,and edge contour information is enhanced through a gated feature fusion mechanism.Then,a texture guide and directional attention module are designed to extract layered texture features,guide the decoder to reconstruct the mural image,and improve the clarity of the mural texture.Finally,using the spectral-normalized Patch GAN discriminant model for confrontation training,the mural restoration is completed.Through experiments on digital restoration of real Dunhuang murals,the results show that the proposed algorithm improves both structural continuity and texture clarity compared to the comparison algorithms.