Research On Garment Image Generation Method Based On Deep Learning

With the development of machine learning and artificial intelligence technology,the field of computer vision has gradually become a research hotspot.As an important branch of computer vision,generated content has attracted the attention of many researchers.Different image generation methods using images,texts and other information as input have been widely studied in many fields.The existing models applied to garment image generation will have a certain performance degradation.The lack of garment image datasets in the current research is difficult to support the development of garment image production.This paper focuses on the following research on garment image generation:1.Construct a garment image dataset.Aiming at the lack of garment image generation dataset in the current research,a multi-modal garment image dataset SimpleFashion is established.The dataset contains 29259 sets of garment information.Each set of garment includes an image,a contour sketch,a Gaussian sketch,multi-classification labels and text descriptions.2.Research on garment image generation based on image content,using semantic label map and sketch respectively.Aiming at the problem of unreasonable local image space matching and mismatch between generated images and semantic labels in image generation using semantics,a combination of multi-generator cascade network and instance adaptive normalization method is proposed to make full use of the instance information in the semantic label map.The local generator and the global generator are used to generate instance-level and global images respectively,and the fusion reconstruction is performed at the global scale to improve the instance-level feature generation effect at the global image level.Qualitative and quantitative experiments show that the performance of this method has certain advantages on multiple general datasets.Experiments on DeepFashion show the effectiveness of the proposed method for garment image generation.Aiming at the problem that the color of the image generated using garment sketch cannot be controlled and depends on random noise,this paper proposed to use the CycleGAN,add color control labels,and fuse the RGB color label with the input sketch in the pixel channel to realize the color control of the generated image using sketch.Qualitative and quantitative experiments show that the method in this paper has better color control effect and authenticity for garment image generation on SimpleFashion.3.Research on garment image generation using text content and multimodal data combined with texts and sketches respectively.In the image generation method based on text description,the DAMSM module has poor consistency matching between a small number of sample texts and images due to the imbalance of datasets.This paper proposed to use the CLIP pre-trained model to increase the text-image consistency discrimination.The image generated by the model is matched with the input text,and the pre-trained model is used to enhance the learning effect of the texts with less frequency of datasets,and improve the performance of the corresponding garment feature generation image.Experiments on SimpleFashion show that the garment image generated by this method has a more realistic texture and has a certain effect on the unbalanced garment feature generation control.Aiming at the poor effect of multi-modal generation caused by the relatively complex features of garment images and the poor quality of few-sample image generation caused by the imbalance of samples under the training of a single dataset,this paper proposed to use the ControlNet network to control the Stable Diffusion pretrained model for training.The method adds two zero convolution layer before and after the model,which strengthen the model’s ability to generate multi-modal images of garment sketches outside the text content.The pre-trained model greatly improves the quality of fewsample garment image generation under unbalanced samples.Experiments on SimpleFashion show that the model can generate fine-grained high-quality garment images using sketch and text multimodal information.4.System implementation of multimodal garment image generation.In order to solve the problem of the lack of existing garment image generation tools,a garment image aided design system is proposed to realize object-oriented multi-modal garment image generation.The system encapsulates the multi-modal garment image generation method and provides users with multi-modal information garment design through a friendly interactive interface.