Optimization For Video Compression And Compressed Video Enhancement

Recently,with the development of multimedia technology,since the video is one of the most important manners for transmitting information,the quantity and variety of videos increase significantly.Furthermore,the demand for high-quality videos,such as high-resolution video or high frame rate video,is also increasing,which needs high efficiency video compression and high-quality video reconstruction and brings a huge challenge for video compression and processing system.In this thesis,to optimize the video compression and system,we aim to improve the compression efficiency and reconstruction quality of compressed videos.On the one hand,we propose the first fully end-to-end optimized video compression system,which boosts the compression performance.On the other hand,we focus on the video compression pre-processing and post-processing and propose an efficient video frame rate up conversion algorithm and video artifact reduction algo-rithm by combining the information from video compression.The contents and contributions of this thesis can be summarized as follows,First,we focus on the frame rate up conversion in the pre-processing procedure in video compression.To generate high-quality high frame rate video and reduce the bandwidth in the transmission procedure,we propose the joint optimization model of frame rate up conversion and video compres-sion.Therefore,our model can generate the intermediate frame based on rate-distortion optimization and achieve the trade-off between video quality and bit rates.Specifically,to enhance the accuracy of motion estimation,we build multiple information based motion estimation approach by using the existing partition scheme in video codec,feature matching technique and MAP based motion segmentation.Furthermore,the prediction error is modeled as the Gaussian distribution according to the motion reliability and quantization parameter.Based on the off-line learning technique,we design a rate-distortion method for the intermediate frame for better compression performance.Experimental results demonstrate that the proposed method can generate high-quality videos and reduce the bit rates when compared with traditional cascaded methods.Second,due to the lossy compression from video coding,the compres-sion artifact is unavoidable in the decoder side.To enhance the quality of compressed video frames,it is necessary to perform video artifact reduction.In this thesis,the video artifact reduction is formulated as a Kalman filtering procedure.The corresponding prediction and measurement are obtained by neural networks.Based on the linearization and Kalman gain,we can get the posterior estimation for the reconstructed frames.Besides,we further ex-ploit the prior information,such as quantized prediction error and non-local information to achieve high-quality reconstruction.Our method combines the advantages of both the learning based method and model based method for a better restoration.Third,we propose the first end-to-end optimized video compression system by combining the advantages from both the classical hybrid coding scheme and powerful neural networks.In the proposed end-to-end system,we employ neural networks to implement the motion estimation network,motion compression network,motion compensation network,residual net-work and bit rate estimation network.The network is optimized by using the rate-distortion technique for better performance.The proposed framework is very flexible and can be easily extended by using the lightweight or advanced network for better speed/efficiency performance.Experimental results show that the proposed system outperforms the widely used H.264 codecs in terms of PSNR.Our thesis focuses the video compression and the corresponding pre-processing and post-processing techniques and shows the effectiveness of the proposed methods for improving the compression efficiency and reconstruc-tion quality.Our work also provides an effective solution for the demand for high-quality video in the further.More importantly,the research on the deep Kalman model and deep video compression also provide insights for further study.