The Research On Video Coding Incorporating Human Visual Perception

As the part of modern information technology, video coding can slove the problem of transmission and storage due to image and video explosion,and it broadly used in the multimedia fields. HEVC is the latest video coding standard jointly developed by ITU and ISO / IEC again. From the view of the development of video coding technology and the characteristics of the technologies, how to obtain the optimal rate-distortion performance and computational efficiency is the core issue of designing video coding algorithm with constraint of computational complexity and time-delay.With the further study of Human Visual System (HVS) and psychology, it was found that the perception of HVS for the video scene is subjective selectivity, diversity, namely different regions or objects in video scene have distinctive levels of visual importance. Similarly, for different features, including brightness, texture, moving, edge and position features, the human eye can perceives the different results. These findings have been broadly proved on the research of visual neuroscience. However,it does not consider that the perception of HVS for the video scene is selective in traditional video compression. Therefore, the further research of how to utilize the visual perception principles to improve encoding effect and calculation efficiency of video compression algorithm has important theoretical significance and application value. In this dissertation,we focus on the key technologies of video coding based on human visual perception.(1) The time consuming of each functional module in HEVC is analyzed by analysing experimental data, and the optimized process of CU partition and rate-distortion optimization can be utilized to accelerate HEVC encoding. Firstly , with the help of weigthing of visual sensitivity,we utilize the texture complexity and motion intensity of the blocks to construct a decision condition for early termination of CU partition ; Then,we also compute a biggest depth of current CU using texture features,according to the statistical relationship between the current block and the co-located block. Secondly, this thesis regards the CU division as a problem of binary classification, and introduces the Bias classification algorithm, which generates the threshold value compared with features obtained by encoding module to make decision. For the feature region with rare probability we use Bayesian to determine the division, otherwise using rate-distortion cost. The experimental results prove the correctness of the algorithm proposed in this thesis.(2) Firstly, an improved frame-level bit allocation model is proposed.In this model, the computation complexity and continuity of video content has been introduced to allocate bits for frame with consideration of the content continuity between GOP and the content continuity between consecutive frames. Experimental results demonstrate the reasonability of our proposed method. Secondly, we improve the LCU-level bit allocation algorithm in terms of HVS. Each frame should be divided into salient regions and non-salient regions, and then we can implement different strategies to gain the better performance of rate control.(3) We propose a video saliency detection algorithm in compressed domain, which is different from other image saliency detection algorithms and mainly uses DCT coefficients and MV extracted from bitstream.Because these features are selected by video encoders, these can not only accurately represent the trajectory of object and the texture of image, but also can implicitly express the saliency map. We utilize the normalized entropy to calculate the spatial saliency map and the temporal saliency map by using the moving visual window, and finally merge these maps with variance-like fusion method. Additionally, since HVS is more sensitive to motion and the motion in background and foreground is always mixed in the natural video, it brings a number of difficulties to yield the saliency map. In this work, we statistically deduce the motion perception weights using a global motiom estimation algorithm, and utilized these weights to refine the temporal saliency map to efficiently improve the accuracy of saliency detection.