Video Virtual Try-on Research Based On Transformer

In recent years,image-based virtual try-on technology has played an important role in many fields such as online shopping,movie production,and clothing matching.Video virtual try-on technology has enriched the visual effect based on image virtual try-on and has also been favored by many researchers.The video virtual try-on method aims to match the human body with the clothing in a video in a temporally consistent manner.Existing methods have been able to generate better try-on videos,but in the face of the input clothing patterns and textures are more complex,and the generated try-on videos will have the problem that a large number of clothing details are lost and the clothing is excessively distorted,which can lead to blurred clothing prints.To solve the problems described above,the following work is carried out in this paper,based on domestic and international research,to address how to generate high-quality try-on videos.1、To address the problem that the input character details and clothing details are lost due to the neglect of the interrelationship between the inputs,a stepwise Transformer module is added to the garment distortion network to solve the problem,which is a combination of the regular Transformer encoder and the cross-channel Transformer encoder arrangement.It can simulate the long-distance relationship between the inputs through the character representation and the garment feature;so that a more accurate remote relationship can be obtained and applied to the subsequent try-on network,and it also helps to improve the performance of the subsequent thin-slab interpolation TPS transform.2 、 The TPS transformation in the garment distortion network is improved for the problem of excessive distortion of the target garment.First,a TPS-based spatial transformation network STN module is added,which is based on the principle of predicting the extent of garment distortion using garment resolution maps with human pose points.Secondly,the TPS parameter regularization operation is added to constrain the overfitting,which is performed on the mesh deformation,by controlling the distortion difference between the previous grid and the next mesh gap,to finally achieve the effect of suppressing the excessive clothing distortion.3、To further improve the clarity of garment prints,a step-by-step Transformer module is also added to the try-on network,but in a different combinatorial arrangement.This module is used to establish global dependencies by character representation,distorted garments and their masks,which enhances important regions in the input data and at the same time allows the UNet network with the added self-attention mechanism to generate better rendered images,and finally generates high-quality try-on videos using the optical flow loss method.