Research On Multi-frame Alignment Method For Compressed Video Quality Enhancement

With the increasing popularity of images and videos in the field of computer vision and various multimedia applications,the number of images and videos is exploding and the demand for high quality visual content is increasing.Due to the increasing volume of raw video data,which makes efficient storage and high-speed transmission complex and challenging,video coding techniques have been proposed to solve the problem of excessive video data volume.Video coding techniques take advantage of the fact that video data content is highly correlated and contains a large amount of redundant information to compress the raw video in a lossy manner,using a certain amount of distortion in exchange for significant bitrate savings.As video coding techniques are lossy,they inevitably introduce compression artefacts that can lead to a degradation of the objective quality of the compressed video,resulting in a poor user experience.Therefore,some in-loop filters have been introduced in existing coding standards to remove compression artefacts,such as deblocking filters,sample point adaptive offset and adaptive loop filters,but these traditional filters have limited improvement in compressed video quality.Inspired by the great success of deep learning networks in many image and video tasks,more and more deep learning networks have been proposed for compressed video enhancement,from the initial single-frame enhancement algorithms to multi-frame enhancement algorithms with better enhancement results.The key technique in multi-frame enhancement algorithms lies in aligning adjacent reference frames,which mainly includes optical flow alignment methods and deformable convolutional alignment methods,but both of these methods have their own drawbacks and shortcomings,so this Thesis conducts a series of studies on multi-frame alignment methods for compressed video quality enhancement,and the details are as follows:1.This thesis proposes a two-stage multi-frame cooperative enhancement network based on optical flow alignment scheme.The two-stage enhancement its performed to make full use of the information of adjacent high quality frames.The compressed video is enhanced in one GOP unit.The joint multi-frame cooperative enhancement takes into account the content and structure similarity of multiple low quality frames between adjacent high quality frames.The enhancement effect can be achieved by taking into account the similarity of the content and structure of multiple low-quality frames between adjacent high-quality frames.2.This thesis analyzes the shortcomings and deformable convolution alignment scheme and optical flow alignment scheme,proposes dynamic deformable convolution,which adaptively changes the convolution weights and convolution sampling positions according to the input,and designs a compressed video quality enhancement network based on dynamic deformable convolution to extract multi-scale features from the input adjacent multiple frames,then adaptively learns the dynamic convolution kernel weights and deformable sampling positions for dynamic deformable convolution alignment according to the feature map,and finally fuses the aligned multiple frames by a quality enhancement module to fully explore and utilize the complementary information between adjacent frames to enhance the target frames.