Algorithms On The Hybrid Spatio-temporal Resolution Video Enhancement

With the development of digital multimedia processing technology and the im-provement of the multimedia microprocessor process capability, ultra-high definition television is the new direction among high-definition television. Meanwhile, ultra-high definition video become more and more popular with the high expectation of video quality and visual experience of users. However, the brand network bandwidth requirement of ultra-high definition video become the bottleneck for its popularity.The higher resolution and higher frame rate of video playing on the large-size liq-uid crystal display device, the smoother and the better experience we get. But with the enhancement of the resolution and frame rate, the data size of video would be al-so increased rapidly, it is a huge challenge for the small limited network bandwidth. For instance, the resolution of ultra-high definition video generally is 3840×2160 pixel, compared with high-definition video, the resolution has increased three times, and the corresponding data size is also increased for three times nearly; for frame rate, double frame rate would produce almost one time increased in data size. The scalable video coding of H.264/AVC standard could realize adaptive adjusting frame resolution or frame rate to suit the network changing. Hybrid resolution video is based on the scalable video coding by down sampling the spatial or temporal res-olution of the original video to decrease the data size and suit the limit network bandwidth. Then at the decode side, the lost information of the hybrid resolution video is recovered to obtain the original video via video enhancement algorithms.This paper has focus on the hybrid resolution video processing technology, includ-ing the spatial resolution enhancement and frame rate up-conversion of mono-view video and spatiotemporal resolution enhancement for asymmetric stereo video. Hy-brid resolution video has saved statistically two-thirds storage space and network bandwidth; meanwhile, via spatiotemporal resolution enhancement, it could achieve a better visual experience.In general, the main contributions of our work can be summarized as follows:1. A key-frame based adaptive overlapped block motion compensation algorithm for video super-resolution is proposed in this paper. Firstly, the motion vector field is produced by block-matching motion estimation. Then, based on the sum absolute difference, angel between the current block and neighbor blocks, classify the motion vectors to different reliability level and correct the distort one. Thirdly, use different matrix for motion compensation according to the reliability level to improve the whole quality of frame.2. An edge-detection based frame rate up-conversion is proposed in this paper. First, variable block motion estimation has been employed by taking advan-tages of edge information. Then, for the overlapped areas in the unidirectional motion compensation, select the block which has the minimum sum absolute of difference among the blocks as the best matched one; for the hole areas, we fill the holes with surrounding information according to the edge location of the holes. The proposed algorithm has maintain the complete edge information and relieved the error information during filling holes and block artefacts.3. In this dissertation, we propose a spatiotemporal resolution enhancement for asymmetric stereo video. The asymmetric stereo video has one view with high resolution and low frame rate, and other with low resolution and high frame rate. First, an adaptive support weighted based normalized mutual informa-tion stereo matching algorithm has been employed. Via the stereo matching algorithm, we get the disparity between the two synchronized frames and im-prove the low resolution one by disparity compensation. Then, the improved frame in the low resolution view are treated as key frames, and this view applies the key-frames based video super-resolution to enhance the spatial res-olution and obtain a high resolution and high frame rate view. Finally, for the low frame rate view, we employ inter-view and intra-view frame interpolation algorithms respectively to achieve two interpolation frames, and combine the two frames based on the motion property. The fusion method has a high ro-bust effect and the experimental results prove the effect in both objective and subjective qualities.