Research On Lightweight Model For Semantic Image Synthesis

Image synthesis refers to generating a pseudo-real image close to the real one based on the semantic pixel-level labels.This task can be applied in image editing and image rendering.Semantic image synthesis generally uses Generative Adversarial Networks(GAN)for training,which requires a lot of parameters and computing resources.The quality of the synthesized image is considerable when the GAN model structure and parameters are large enough.However,when the hardware conditions are relatively harsh such as the small memory capacity and the limited computing resources,the quality of the synthesized image will be greatly reduced if the amount of model’s parameters is simply reduced.Aiming at this problem,this article has launched research on the lightweight model of semantic image synthesis.This paper proposes a method combining information entropy increasing and knowledge distillation,which greatly reduces the model’s parameters.On the other hand,we try to narrow the gap of the quality of the synthesized image between the compressed model and the original one.The main innovations of this article include:(1)Analyze the process of semantic image synthesis from the perspective of information entropy,and propose a method of increasing information entropy to change the high pixels consistency of the semantic labels to make the input semantic information richer and increase the input information entropy.(2)Combine information entropy increasing and knowledge distillation effectively.With information entropy increasing modules,the quality of the synthesized image of the student network is close to that of the teacher network.The parameters and calculations of the student are reduced significantly.This paper has conducted multiple experiments on multiple data sets to prove the validity and reliability of the information entropy increase idea proposed in this paper.The student network is not only lightweight,but also bridges the gap with the teacher network.