Design And Implementation Of Style Transfer Platform For Short Video Based On Deep Learning

The popularity of mobile devices and the development of communication technologies have made short video applications,such as Tik Tok and Kwai,popular all over the country.At the same time,deep learning technologies are being maturing.More and more research results are being applied to mobile devices.Style transfer is an image re-creation technology based on deep learning.It can render photorealistic images into artistic style images,which is highly popular amaong people.The image processing software Prisma has gained tens of millions of users with the utilization of style transfer technology.In this paper,we apply style transfer technology to the short video creation process.Aiming at the problems of poor flexibility,poor stability,and large volume,we carried out a targeted study,proposing an arbitrary style transfer model for video and deploying it efficiently in mobile applications.The specific work of this paper is as follows:First of all,we study a arbitrary style transfer model that is based on conditional instance normalization,meeting the requirements of flexible,real-time,and lightweight,in view of the limited computing resources of mobile devices.In order to enable the model to be applied to video,a temporal consistency loss is introduced at the network feature map level and the output level during training process.Experiments prove that this method can greatly improve the model’s temporal stability to generate a coherent video,while maintain real-time arbitrary style transfer ability.In addition,in order to further compress the model and improve inference speed,the style prediction network,part of the above model,was replaced with MobileNetV3 network and trained using knowledge distillation technology;for the image transform network part,plain convolution is replaced with deep separable convolution.Coupled with the model quantization compression,the model volume was reduced by a total of 88.2%.Based on the video style transfer model described above,we design and implement a short video style transfer platform,which provides many functions such as stylized video recording,style transfer for local video files,short video editing,and artistic short video communication communities on mobile devices.In order to solve the problems of low code reuse and poor testability common to system development,we use a functional reactive programming method to implement a signal-driven MVVM architecture pattern through a data binding mechanism.After the unit test,integration test,UI automation test and performance test,all the functions of the platform are proved to achieve the expected effect and meet the users’ needs for creating artistic short videos.