Research On Video Repair Algorithm Based On Deep Learning

Video repair is one of the important research topics in computer vision,which aims to repair defects from the video and make the repaired video have a satisfactory visual viewing effect.Video repair technology has a wide range of applications in many scenarios,such as super-resolution,face recognition,motion capture and video surveillance.In recent years,the research on video repair algorithms based on deep learning has been widely studied,and there has been some progress in related issues such as denoising,content repair,deinterlacing and reflection removal.However,for the existing video repair algorithms,especially the research on video deinterlacing and video reflection removal algorithms,there is still a lot of room for improvement in the effects generated by the current algorithms,such as the phenomenon of artifacts.There are two reasons for this: 1)Different from two-dimensional static images,video is spatial-temporal continuous,and some existing methods do not make good use of the continuous information of video in the time dimension.2)In the running process of the algorithm,the position of video frame features may shift after some module operations,which will also have a certain impact on the visualization results of the algorithm.This paper studies the video repair algorithm based on deep learning.The main contents and contributions are as follows:(1)Video deinterlacing algorithm based on self-attention mechanism.Based on the existing algorithm,this paper introduces a self-attention mechanism,which can well capture and utilize the spatial and temporal information of video frames input into the network,so as to improve the network’s ability to extract features.In addition,this paper also refers to the idea of U-Net network and residual dense network,and designs the De-fusion module of video frames,which aims to fuse the feature information output by the Spatial-Temporal Convolutional Self-Attention module to improve the performance of model deinterlacing.Experimental results show that the proposed algorithm can make the final experimental results show the best visual performance,and the PSNR and SSIM indicators obtained by testing on the dataset Vimeo90 K are about 3% higher than the existing algorithms.(2)Video reflection removal algorithm based on temporal feature alignment.The video reflection removal algorithm designed in this paper uses the self-attention mechanism to capture the spatial and temporal information between video frame sequences.Inspired by the deformable convolution,this paper also designs a deformable video frame alignment module,which processes the features of different objects in the video frame by adjusting the sampling position of the convolution kernel,so as to adjust the position of the video frame features at the pixel level to achieve the goal of video frame position alignment.In addition,the single frame image reflection removal module is redesigned in this paper,in order to better adapt to the video reflection removal task,and finally achieve the purpose of video reflection removal.The experimental results show that the PSNR,SSIM,LMSE and NCC of the algorithm on the dataset FVI reached 25.33,0.9283,0.0059 and 0.9961 respectively;while the PSNR,SSIM,LMSE and NCC on the dataset Vimeo90 K reached 28.88,0.9434,0.0046 and 0.9774.In general,the proposed algorithm improves the index by 1 to 2 units compared with the existing video reflection removal model.