Research And Software Implementation Of Video Colorization Method Based On Generative Adversarial Networ

By colorizing black and white video,the color of classic video works can be restored,which is helpful for the research of history.Most of current video colorization methods predict color by deep neural networks.This method requires huge datasets and the coloring results are unpredictable,which leads to multi-modality.At the same time,it is easy to cause obvious color jumping problems in the video if adjacent frames are not processed individually.Based on this problem,colorization methods assisted with reference frames are adopted to alleviate the color jumping problem of adjacent frames,but increase the cost of manual participation,and the workload is huge.This thesis mainly studies the problem of automatic coloring of black and white videos,and the main work and innovation points are as follows:(1)The enhancement effect of memory module on Generative Adversarial Networks is analyzed.The discriminator of Generative Adversarial Networks tend to forget the samples generated by the generator at previous times during the training process,resulting in abnormal training fluctuations.Based on this,a memory module is introduced to enable the Generative Adversarial Networks remember the sample information in the training datasets.The memory module can also assist the generator to better understand the underlying distribution of the training data and alleviate the forgetting problem.At the same time,Generative Adversarial Networks can successfully extract useful data from the memory module in the testing stage.(2)A video colorization model Multi Color Net based on memory enhancement network and Generative Adversarial Networks is proposed to alleviate the color jumping problems of adjacent frames and the multi-modality problem of coloring results in the video coloring process.In the video colorization task,the memory enhancement network can store the color features and spatial features of the images in training datasets,and retrieve the color feature closest to the input video frame.This color feature is injected into the colorization model as a reference feature to enhance the color consistency of adjacent frames in the same scene.Then,combined with the adversarial loss advantage of Generative Adversarial Networks,a color image closer to the real is obtained.In addition,in order to ensure the coloring effect of the model on long video and optimize the color continuity of multi-scene video,a method is proposed to judge whether scene switching occurs according to the characteristic differences of the frames before and after the video and the set threshold.The experimental results show that the model can effectively perform scene segmentation,realize the coloring of videos of any length,alleviate the color jumping problems of adjacent frames,and produce more realistic and stable coloring effects.(3)A video processing software named SXJL is designed.Based on the Multi Color Net model,the software is implemented by combining deep learning and software engineering knowledge,and designed in the way of MVC layered architecture.The software can colorize black and white videos automatically,and support fine-tuning of colorized videos,such as adjusting the color of the video.Other functions include image colorization,image enhancement,video mute,video editing,video adding subtitles.