The Research On Time-lapse Video Generation Based On Style Transfer

Time-lapse video captures the real scene in a dynamic time lapse,presenting compelling visual effects in a fraction of the time.With the popularity of new social networks,more and more time-lapse videos can be accessed at any time.However,the process of creating time-lapse videos is often cumbersome and costly.Generating video from a single image has always been a hot research problem in the field of computer vision.Time-lapse video can also be regarded as the image style transfer of video frames over time,while the generation of video frame can be regarded as an extension of the image style transfer task.In view of the shortcomings of the current time-lapse video generation task and the problems to be solved by applying image style transfer to the time-lapse video generation task,this paper designs a time-lapse video generation algorithm based on style transfer.The main research work is as follows:(1)In view of the lack of shadow changes in existing time-lapse video generation tasks and the shortcomings of existing shadow generation methods,this paper proposes an instantiated shadow generation method,which avoids the light properties,geometric estimation of objects,and 3D reconstruction of objects.The method combines UNet and Res Net as the network architecture,and consists of two stages of shadow area prediction and shadow fusion.Using the reference object and its shadow as conditions to predict the instantiated object,which not only improves the accuracy of shadow area prediction,but also generates more realistic shadow.(2)In view of the limitations of the existing time-lapse video generation task and the problems such as style overflow and lack of time consistency of adjacent frames when image style transfer is used to generate time-lapse video frames.This paper proposes a style transfer and time-lapse video generation method based on semantic matching,which takes any time-lapse video as a reference style,a single image as input,and performs style transfer for the same semantic category through semantic matching to generate video frames.It designs a loss function to ensure the temporal consistency between adjacent video frames.Finally,a time-lapse video with the structure of the input image and the style of the reference video can be generated.(3)This paper carries out comparative experiments and ablation experiments,which proves the superiority of the above method and the effectiveness of the design of each part.