Research On Thangka Image Inpainting Based On Generative Adversarial Networks

The Thangka is an important intangible cultural heritage of the Chinese people,yet in the process of preservation and transmission,it often suffers from damage that cannot be easily removed,such as blurring,color changes,cracks and missing parts,which cannot be easily physically repaired,bringing difficulties for telling the Chinese story.Therefore,it is of practical importance to know how to effectively inpainting damaged Thangka images.Nowadays,frontier technologies such as machine learning are rapidly developing,and deep learning presents great advantages in inpainting tasks.Therefore,this thesis research on Thangka image inpainting based on generative adversarial networks.At the same time,inspired by the way painting artists create edges before textures,and taking into account the challenging problem of restoring images aiming to generate meaningful structures for damaged areas,this thesis takes a two-stage approach to inpainting Thangka images,dividing the process into an edge detection complementation stage and a texture inapinting stage.Specifically,we study the existing limitations of traditional and deep learning based techniques for inpainting of Thangka images,improve the edge detection complementation model and texture inpainting model,and design a inpainting approach for Thangka images based on edge structure constraints.The main work of this thesis is as follows:(1)In the absence of publicly available Thangka datasets,and in order to better validate the feasibility of this thesis’s method for inpainting real damage in later experiments.The thesis is based on the images of Thangkas with natural damage,which were collected through various methods,such as internet access,book scanning and field photography.(2)For the first stage of the inpainting task of edge detection and patching,the first step needs to detect the broken image edges as the input of the patching stage,and in order to obtain edges that are closer to the broken tangka,the Canny Gaussian filter is replaced with an adaptive median filter,and the resulting broken edges will be patched,and a multi-scale edge patching network based on generative adversarial is proposed to solve the problems caused by insufficient learning of valid information,such as prediction errors due to insufficient learning of valid information to fully learn the relationship between foreground mask and background edges.(3)For the weak correlation between known and unknown pixels in the encoding stage,dynamic partial convolution is used,which facilitates adaptive adjustment of the mask ratio to differentiate pixels.At the same time,if the broken and unbroken regions are simply considered to be the same,it will lead to the problem of mean-variance shift,so region normalisation is introduced to divide the spatial pixels into different regions according to the mask,which helps the broken regions to continue the valid pixel information.(4)In this thesis,the two-stage inpainting method is used to analyse the needs of user scenarios and experience,and the Python programming language is used to design convenient inpainting functions,mainly covering several modules such as edge detection and texture inpainting,to achieve a simple and beautiful digital online tangka inpainting operating system.In this thesis,by conducting a large number of comparative and ablation experiments on the Thangka dataset,as well as qualitative and quantitative analysis and comparison,we show that the edge constraint plays a positive role in the inpainting of Thangka image textures,and validate the effectiveness of the proposed method.