Art Portrait Synthesis And Restoration Based On Prior Guidance

Artificial intelligence(AI)-based art generation has shown empirical performance and broad application prospects.However,conditional controlled art generation still faces great difficulties,due to the scarcity of paired data.Existing methods usually suffer from unrealistic texture details and inconsistent contextual structure.In this paper,we propose to use generative prior to guide art generation.Specially we construct an algorithm for art generation based on prior modulation,and an algorithm for art restoration based on prior prediction,respectively.Then,a demonstration system is developed for practical application.The work in this paper consists of three main points as follows:(1)First,a prior modulation-based art generation algorithm is proposed.We transer the Style GAN in the photo domain to the art image domain,and use the fine-tuned Style GAN as the a prior generative network.In the generator,We first modulate the encoded features with the prior generation network to obtain structurally consistent,domain-transformed initial artistic portrait features.Afterwards,we decode the modulated features of the prior generation network to generate the final portrait.Here,the the prior generative network enables the model to fit the distribution of the target domain,thus improving the quality of art generation.We also propose a new distribution loss to constrain the input image and output image to maintain similar neighborhood relationship,which significantly improves generated details.Experimental results show that our method can generate high-quality multi-style art portraits and outperforms existing methods.In addition,our method can still maintain good performance with few-shot training samples,and thus can be extended to more art generation tasks.(2)Second,an algorithm for art restoration based on prior prediction is proposed.This algorithm uses a two-stage from-coarse-to-fine restoration framework.In the first stage,a U-Net is used to predict the coarse restoration results.In the second stage,the coarse restoration image is used as prior information to modulate decoding features,in the manner of spatially adaptive normalization.In the refining network,we propose a multi-path spatial-adaptive denormalization module to enhance the fineness of the a prior modulation.Finally,to alleviate the problem of overfitting under few-shot settings,we propose a contrast learning loss based on contextual information.We apply the method to the restoration tasks of face pen drawings and Dunhuang murals.The experimental results show that the method can achieve fine restoration of artistic images and outperforms existing methods.Ablation experiments also show that our proposed new modules significantly boost the performance of the model,and improve the restoration quality of art images.(3)Finally,we develop a demonstration system for art generation.The system integrates current advanced art generation algorithms,as well as our algorithms.We have encapsulated these algorithms to provide users with a convenient interface for art generation.In addition,to reduce the operation and maintenance pressure of the back-end system,this paper containerizes the system and deploys it to a K8 s cluster,thus providing stable load balancing and elastic scaling capability.In summary,two prior knowledge-based image translation models are proposed in this paper,and applied to art synthesis and restoration tasks,respectively.Both methods significantly improve the quality of art generation.We also develops an art portrait generation system,which provides a rich and stable user experience.This paper is of great significance to the field of art generation.