Research On Content-Adaptive Image And Video Coding Algorithms

Along with the constantly growing demand for image/video transmission and storage,image/video coding technologies are facing with higher requirements and greater challenges.For various coding schemes,one of the reasons that their coding performance is increasingly improved is that the content-adaptive ability of those schemes is becoming greater.However,there are still some limitations on the content-adaptive ability of those coding schemes,further constraining coding efficiency improvement.It is observed that for traditional coding scheme,its content-adaptive ability is affected by three factors,which are the coding efficiency of single coding technology towards certain type of content,the abundance of coding technologies to cover various contents and the decision scheme to choose optimal coding technology for the to-be-coded content,respectively.For end-to-end coding scheme,its content-adaptive ability is impacted by the content-adaptive ability of context information area and model parameters.Focusing on improving the content-adaptive ability of the two coding schemes,the thesis proposes several coding technologies from the perspective of impact factors of the content-adaptive ability,which helps to further improve image and video coding efficiency.The content and contribution of this thesis can be summarized as follows,Intra prediction algorithm for spatial rotation and scaling: Based on the impact factors of the content-adaptive ability of traditional coding scheme,we focus on intra prediction module to study possible method to improve its content-adaptive ability.In particular,for content with spatial rotation and scaling which is rarely investigated,a spatial four-parameter affine deformation model is built and affine intra prediction technology is specifically designed.As a supplementary work of VVC coding scheme(standard),the proposed technology receives0.99% coding efficiency improvement.Neural network-based coding algorithm with additional input: To alleviate the constraints of hand-crafted coding technology design and pre-classification of various contents,we study neural network-based coding technology.To further improve its content-adaptive ability,we investigate the neural network-based coding algorithm with additional input.Specifically,we focus on in-loop filtering module and propose a neural network-based in-loop filter using prediction residuals,to additionally utilize the prior information generated from the coding procedure and to improve the neural network structure in terms of feature extraction and feature assistance.Compared to HEVC standard,the proposed technology obtains an average of6.8% and 2.3% coding efficiency improvement under all intra and random access configuration,respectively.End-to-end image coding algorithm with content-adaptive context information area:To flexibly adjust context information area towards various contents,an end-to-end image coding technology based on multi-scale deformable convolution is presented,where multi-scale deformable convolution is introduced to learn context information area adjustment more accurately.Compared to VVC standard,the proposed technology obtains additional 2.77%coding efficiency.End-to-end key frame coding algorithms with content-adaptive model parameters: To flexibly update model parameters towards the to-be-coded content,we study a temporal-spatial prior based model parameter updating technology.By utilizing temporal and spatial correlation within key frames,model parameters are updated according to content and additional rate cost for transmitting updated model parameters is avoided.Compared to the offline-optimized model,the proposed technology receives 15.33% coding efficiency improvement.The thesis systematically researches methods of improving content-adaptive ability for both traditional and end-to-end coding schemes.Specifically,we study to improve the match between the specific content and the corresponding technology,as well as the match between the specific content and the corresponding neural network components.The thesis proposes several innovative technologies and offers new thoughts towards the content-adaptive image/video coding area.