Reseach On Perception-based Three Dimensional Video Coding

As the appearance of3D displays, coding systems and capture devices, new videoapplications such as3D television and free-viewpoint video have drawn wide attention.Researchers get rid of the redundant information of multi-view video signal so as to compress thevideo data, mainly from the perspective of time, space domain of the Multiview Video Plus Depthformat, but they does not take into account the perception characteristics of the Human VisualSystem. Visual psychology and physiology research shows that the HVS’s perception of video andimage is uniform, and any change in the video and picture can not be completely perceived,meanwhile, the study also found that the three-dimensional visual masking effect also exists. Inaddition, interactivity of the client and server side is also a kind of perception. This paper developesthe notion of perceptual coding combined with the perception of the HVS, and devises3D videocoding method based on perception. This paper mainly includes the following three aspects:(1) Combined with the stereo perception characteristics of the HVS, three kinds of stereoperception model are established. Firstly, the Depth-based Just Noticeable Distortion (DJND)model based on depth information is established by different sensitive degree of the human eye tobrightness, texture, time domain and depth. Then, the Depth-based Foveated Just NoticeableDistortion (DFJND) model based on the characteristics of retina is established by the foveated justnoticeable distortion model of retina and the depth features of the stereoscopic video. Finally, theBinocular Stereo Just Noticeable Distortion (BSJND) model based on the binocular stereoperception is established by the Binocular Just Noticeable Distortion model of reflecting binocularspace sensitive characteristics, the time-domain mask of reflecting the video motion regionsensitive characteristics and the parallax information of reflecting depth sensitivity characteristics.This paper presents three stereoscopic video coding algorithms by using these models on theJMVC8.3encoding platform. Experiments show that the distribution of the DJND, DFJND andBSJND model basically accords with the stereo perception characteristics of human eyes.Compared with the original JMVC coding algorithm, to guarantee the subjective quality ofthree-dimensional image essentially unchanged, the bit rate of the right view and the total bit rate instereoscopic video asymmetric coding algorithm based on the DJND model can save1.42%~28.31%and0.65%~13.27%, respectively, and the total bit rate in symmetric codingalgorithm can save2.02%~23.02%. The bit rate of the right view in stereoscopic video asymmetriccoding algorithm based on the DFJND model can save up to34.89%at most, the total bit rate andthe bit rate of the right view in asymmetric region coding algorithm based on DFJND model cansave15.79%~45.37%and54.06%~89%, respectively. The bit rate of the right view saves17.89~83.60%in stereoscopic video asymmetric region coding algorithm based on the BSJND model.(2) This paper proposes a new3D scene data representation format based on depth-image-based rendering, which is a left view plus residual of a right view. This paper designs threestereoscopic video coding algorithms by the format on the JMVC8.3encoding platform. Firstly,this paper proposes a stereoscopic video coding method with based on depth-image-basedrendering. The experimental results show that compared with the original JMVC coding algorithm,the proposed method can highly reduce the bit rate of right view and maintain high quality ofreconstructed right view in the meanwhile, and in particular, the RD performance can be improvedsignificantly in low bit rate. Then, combined with bit rate allocation proportion relationship of colorand depth video and HVS characteristics, this paper proposes a stereoscopic video perceptualcoding method with based on bit rate allocation. The experiments show that rendering quality ofthe middle view increases by0.19~0.85dB under condition that the total bit rate is essentiallyunchanged. Finally, combined with HVS perception characteristics and the effects of depth videoon the quality of the rendering virtual viewpoint video, this paper gets the left and right view depthnoticeable distortion distribution map and proposes a three-dimensional depth video codingmethod based on perception. The experiments show that the bit rate can save up to56.07%at mostfor same rendering quality, rendering quality improve1.22dB at most for the same rate, thesubjective quality of virtual view almost the same for the same Quantization Parameters, ifcompared with the original JMVC coding algorithm.(3) This paper proposes an effective interactive multiview video system. Based on this system,we design an interactive encoding method which proposes a new video data encoding format witha processing viewpoint and a nearest viewpoint from the processing viewpoint, and then putsforward a new virtual view rendering technology on the client. Experimental results show that theencoding bit rate of processing viewpoint can save88.15%~98.58%, and the non-coding block ofthe processing viewpoint which the client requests accounts for more than97%of the wholeviewpoint, thus it reduces the coding complexity on the server side, if compared with the originalJMVC coding algorithm. The encoding bit rate of processing viewpoint can save about75%onaverage on rendering middle virtual viewpoint and reduce the client’s complexity on the client side,if compared with the original MVD coding method. In addition, the virtual view by rendering hasbasically no difference on the subjective quality.