Research On Garment Image Generation Method Based On Hand-drawn Sketches

Fashion design is a skill that applies design aesthetics and natural beauty to garments and their accessories.It pursues practical beauty and is a creative act that combines the human body as an object and materials with various functionalities.The designer must not only have a good understanding of the lifestyle of the public and the needs of the customer,but also be able to express his or her ideas clearly.The garment sketches are the concentrated expression of the designer’s creative intent.First,the garment sketches are drawn by conception,then the garment effects are designed according to the garment sketches,and finally the garments are made on this basis.This traditional design process is time-consuming and labor-intensive,and relies heavily on the subjective will of the designer.In order to reduce the burden of the designer and improve the efficiency of fashion design,computer-aided fashion design can be used to automatically convert hand-drawn sketches into garment images.The main work of this paper is as follows:(1)The hand-drawn sketch-based garment image generation research in this paper requires a large amount of garment images and sketch data for training,and due to the scarcity of garment sketch data,the existing public datasets cannot satisfy the research in this paper,so a scheme is proposed to solve this problem.The scheme starts with the analysis of the required garment sketch dataset,then collect the corresponding garment images,and then use several existing edge detection algorithms to process the collected real garment images to obtain the garment edge map.In this paper,we first analyze the required garment sketch dataset,then collect the corresponding garment images,and then use several existing edge detection algorithms to process the collected real garment images to obtain the garment edge map.After the qualitative and quantitative analysis of the comparative experiments,the edge detection algorithm with the best performance is selected to construct the garment sketch dataset in this paper.(2)Proposed a two-stage sketch generation method for garment images based on CycleGAN.In the first stage,for the deformation problem of hand-drawn sketches,the input sketches are non-rigidly aligned by the spatial alignment module to obtain more reasonable results,and then the garment images are initially generated by the CycleGAN network based on the self-attention mechanism.In the second stage,in response to the problem that the handdrawn sketches are too simple and abstract resulting in very limited information provided,a diverse network guided by the reference image is designed,which uses the VGG16 encoder to extract style features and content features for the reference image and the generated image of the first stage,respectively,and then inserts the style features into the Ada IN layer of each layer of the VGG16 decoder to finally generate a more content-rich garment image.The experimental results show that the garment images generated by the method in this paper are better in visual effect compared with the comparison method,and they are 12.7% lower than CycleGAN in terms of FID index.(3)Proposed a multimodal garment image generation method based on hand-drawn sketches and text.To address the problem of limited information provided by sketches in the proposed method in Chapter 4,which makes it difficult to generate higher quality garment images,it is proposed to combine multimodal inputs into the model to avoid this problem.The first stage trains the visual Tokenizer to learn to compress the input image into a series of discrete representations;the second stage models the correlation between the sequence of tokens of conditional information and the sequence of tokens of clothing images by a bidirectional Transformer.To improve the quality of generated images,five masking strategies are used in the masked visual image modeling task during training,and non-autoregressive generation is used to improve the generation speed during iterative inference.Experimental results show that the method in this paper outperforms the comparison method in both text-toimage generation and sketch-to-image generation tasks.