Online Learning Based Decoder-Friendly HEVC Quality Enhancement Network

Most of the Internet data is composed of image and video data,but its data volume is often very large and not suitable for dissemination,so video coding technologies have emerged to compress video,such as the HEVC(High Efficiency Video Coding)standard,which is widely used around the world.However,while compressing the data volume,video coding and decoding techniques often damage the video quality,so how to enhance the decoded image quality has become an important research problem.Among the traditional methods used to enhance the quality of compressed video,the more well-known ones are De-Blocking and Sample Adaptive Offset(SAO),both of which are also included in the HEVC standard.Both methods are highly specialized,where de-blocking is a technique used to reduce square artifacts in compressed video.The problem arises from the fact that compression algorithms usually divide the image into smaller blocks and perform a more independent compression process for each block,which leads to a possible degradation of visual quality due to obvious boundary lines between the adjacent boundaries of the blocks.Adaptive sample point compensation,on the other hand,is mainly used to reduce the ringing effect in the edge regions of the image texture,which is mainly caused by the distortion of pixel values due to the quantization step in the compression algorithm.In recent years,with the rapid development of deep learning,many scholars have proposed neural network-based quality enhancement methods,which are more universally applicable than the traditional methods mentioned above,in other words only one deep model can handle both the de-squared and ringing effect problems,and these models have achieved good performance and improved coding efficiency.However,the shortcomings of these neural network-based methods prevent them from being applied in real-world scenarios.One is the high computational overhead required by the neural network-based methods,and the other is the large size of the models themselves,which increases the storage burden of the devices.The above two problems make the neural network-based approach difficult to implement on devices with limited computational power and storage space,such as cell phones,which are the decoding end devices in most application scenarios.In contrast,this thesis proposes a decoder-friendly quality enhancement method for this problem,which is named DFCE(Decoder-friendly Chrominance Enhancement).The main contributions of this thesis are as follows:1.A novel quality enhancement scheme based on online learning is proposed.In many deep learning-based low-level image tasks,the target high-quality images output by the model are often not available,such as super-resolution tasks,Gaussian noise removal tasks,rain removal tasks,etc.Therefore,the solution for these tasks is often to train a model offline,and the parameters of that model are constant during testing.The proposed deep learning-based video coding quality enhancement methods also tend to continue this idea,so in order to improve the fitting performance of the model,it is often necessary to use a large number of data sets for a long period of training,so that the volume and computational complexity of the model increases accordingly.However,in the video coding task,the original target image that the model needs to output is available at the coding side,and this provides the possibility of online learning.Therefore,in the DFCE method proposed in this paper,the compressed image and the uncompressed original image are used as a pair of training samples at the encoding side to train the model at the encoding side online,and then the new parameters obtained from the training are transmitted to update the model at the decoding side to obtain quality-enhanced performance improvement to achieve the ultimate goal of code rate saving.2.Based on the above online learning-based approach,a very lightweight and effective network named LGCEN(Luminance-guided Chrominance Enhancement Network,LGCEN)is proposed in this paper.Unlike previous work that tends to increase the complexity and size of the network,the LGCEN network in this paper is designed to be more efficient to achieve a lower volume of individual models.Firstly,we design an Adaptive Layer(AL)embedded in the model,which can implement an online learning-based channel attention mechanism that will be updated with the code-side data when it is actually used,and thus adjust the channel attention more efficiently to improve the overall performance of the model.Further,since luminance images tend to have high resolution and clear detailed textures,the model makes use of luminance channel images to assist the quality enhancement of the chrominance channel.Since the former data is available during the video compression process,no additional computation or storage overhead is required to obtain it.A recursive design is also used to further compress the model volume.This design can increase the model fitting ability without increasing the number of parameters,in other words,it can achieve a lower model size with the same network depth.Experimental results on test sequences of HEVC show that the proposed DFCE method in this paper not only achieves better quality enhancement performance than its predecessors,but also effectively reduces the computational complexity and storage burden of the model at the decoding end.