Research On Video Bit-Depth Enhancement Based On Spatio-Temporal Joint Modeling

Nowadays,electronic hardware devices are developing rapidly,and high-bit-depth display devices such as HDR have been popularized both at home and abroad.However,most video sources are still at a low bit depth,and these low bit depth video sources will appear false contours and color distortion when displayed on a high bit depth display screen after a simple conversion.Therefore,bit depth enhancement processing on video is of vital importance to improve human visual experience.However,most bit depth enhancement algorithms are based on images.If the image bit depth enhancement algorithm is directly applied to the video frame by frame,the supplementary information in adjacent frames will be wasted,the quality of video reconstruction will be limited,and problems such as inter-frame flicker will occur.Video bit depth enhancement algorithm will eventually be implemented in practical applications,and the algorithm can only be applied in practice when it achieves real-time performance.However,in order to achieve better performance for most of the current algorithms,the network is developing in a more and more complex direction,and few can balance the performance and real-time performance of the algorithm.Based on the above two points,this article proposes the following two solutions for the video bit depth enhancement algorithm:(1)A video bit depth enhancement algorithm based on the target-guided spatiotemporal attention mechanism is designed.This algorithm is an end-to-end video bit depth enhancement algorithm based on an encoder-decoder architecture.We a spatiotemporal attention alignment module and a target-guided spatiotemporal attention module for inter-frame information alignment and multi-frame feature fusion.The spatio-temporal attention alignment module adopt a self-attention mechanism to find globally relevant spatio-temporal information,capturing long-distance dependency.The target-guided spatio-temporal attention module is oriented by the target frame,guiding the network to explore more detailed information related to the intermediate frame from the spatio-temporal feature information of multiple frames,so that the network can better integrate the spatio-temporal information.Experimental results prove that both objective performance and subjective video frame quality are higher than existing bit depth enhancement algorithms.(2)A real-time video bit depth enhancement algorithm based on multi-level spatio-temporal feature fusion is designed.Considering that the network should have as low complexity as possible,the algorithm avoids applying alignment methods to video frames,and designs a two-stage progressive low-complexity network architecture to reconstruct low-bit-depth video frames.At the same time,the algorithm also proposes a multi-scale spatio-temporal feature information fusion module,which unifies the feature extraction of video frames and multi-frame feature fusion into one module.From the perspective of the entire network structure,feature extraction and multi-frame feature fusion are unified in one module,further reducing the complexity of the network structure.From the inside of the module,video frame feature extraction and multi-frame feature fusion are alternately processed in a multi-level progressive manner,which can better mine spatio-temporal information and efficiently model spatio-temporal information.Experiments have proved that the algorithm can achieve high performance while achieving real-time performance.Both objective performance and subjective video frame quality are higher than existing bit depth enhancement algorithms.