Chinese Character Inpainting With Contextual Semantic Constraints

At present,the general natural image restoration technology tends to be mature,but it is not suitable for Chinese character image restoration,and it is easy to generate non-characters,pseudo characters or ambiguous characters,The main reasons are as follows:(1)The existing methods are limited to the texture restoration of a single image,and do not consider the font structure and context of Chinese characters;(2)There is a lack of Chinese character image dataset for supervised training and covering a variety of scenes.This thesis carries out research work from the perspective of semantic feature assisted visual feature.Firstly,a Chinese character image data set based on semantic guidance is designed,which generates handwritten and printed text images with different defects through different masks,and uses text labels to associate image context.Secondly,this thesis proposes a Chinese character image restoration model based on multi classification and discrimination.The model is based on the idea of generative adversarial network(GAN),and accurately restores the stroke structure and content semantics by combining the classification and discrimination mechanism to avoid pseudo Chinese characters.Considering that when the image defect is serious,the model may not be able to learn the character features and need additional guidance from context information,this thesis improved and designed a Chinese image repair algorithm based on context semantic constraints.Supervising glyph expression through the introduction of Global Semantic Supervision Module(GSSM).The algorithm takes GAN structure as the repair module and global semantic vector generated by BERT model as the guidance constraint.The damaged image can make up its own content by referring to the context information during reconstruction,so as to make the generated result more realistic and maintain the correct context semantics,avoiding the generation of ambiguous characters.The above methods are evaluated on handwritten and printed Chinese character images.By comparing with the existing methods,it is found that the proposed method can accurately repair the missing content information of Chinese characters and form sentence-level semantics without any mask prior.