Research On Machine Learning Based Coding Unit Fast Splitting Algorithm For HEVC Intra Coding

Video applications are penetrating people’s work and life more than ever.The need for video compression is increasing urgently due to the limitation of modern storage and communication techniques.As a result,video coding standards have also been greatly developed in recent years.High Efficiency Video Coding(HEVC)has achieved about 50%bitrates saving compared with its predecessor Advanced Video Coding(AVC),while maintaining the same quality of decoded video.HEVC improves the compression ratio greatly by introducing many advanced coding tools,such as more intra directions,more flexible partition rules and more efficient filters.These new-introduced coding tools also cause a dramatic increasing of encoding complexity.Among these tools,the partition rules make the searching for the best partition structure of a coding unit(CU)one of the most time consuming process.The high computational complexity hints the implementation of HEVC in some real-time scenes,such as video conference and sport broadcasting.To reduce the encoding complexity of HEVC intra coding,this paper focuses on the problem of partition structure determination for intra CUs by using basic machine learning and deep learning methods.Firstly,statistics on CU splittings are obtained and analyzed,and a residual network(ResNet)based hierarchical decision algorithm is proposed for the fast partitioning of a coding tree unit(CTU).Secondly,improved from the hierarchical algorithm,a bagged treed based frame-wise before-hand algorithm is proposed.Thirdly,further optimizing the second algorithm,this paper proposes an end-to-end fast CTU partition decision algorithm by combining bagged tree and ResNet jointly.Three fast partition decision algorithms proposed in this paper are described briefly as following:(1)A ResNet based fast algorithm is proposed,which decides the partition structure of CTU depth by depth.Based on a careful study for the quadtree partition structure,this algorithm regards the determination of the partition structure for a CTU as a three-level decision problem,and the decision problem of each level is further viewed as a binary classification problem.This algorithm designs different ResNet structures for CUs of different sizes to make full use of characteristics of ResNet and CTU partition rules.The quantization parameter(QP)is feed into the ResNet together with the original luminance values of a CU,which reduces the number of ResNet required.This algorithm has a balanced performance on videos with different resolutions and contents,because several representative training sets are constructed.Using this algorithm,encoder predicts the splitting of CUs in each depth,so that the original complex searching and calculation is skipped.The high prediction accuracy ensures that the video quality will not decrease greatly while the encoding speed is improved.(2)A frame-wise before-hand prediction algorithm is proposed by using bagged tree techniques.This algorithm regards the determination of the CTU partition structure as a three-level cascaded classification problem.Only a bagged tree model is needed to decide the partition structure of a CTU depth by depth.In addition,and the balance between rate-distortion loss and time saving can be realized adaptively.Firstly,this algorithm designs a number of novel and efficient features.Combining these features with many features that have been widely used,a candidate feature set is constructed for the bagged tree model.Secondly,importance value of each candidate feature is evaluated for different coding parameter configurations,and the corresponding active features for different coding parameter configurations are determined according to the ranking results.Then,to reduce the bit rate loss caused by misclassification,the splitting decision of a coding unit in each depth is regarded as a three-classification problem controlled by a high threshold and a low threshold.Finally,two algorithms of different versions are proposed,one is called BTFA which can adaptively calculate the high and low thresholds according to the coding content,the other is called ABTFA which can adaptively calculate the high and low thresholds according to the coding requirements.In this algorithm,all of feature designing,three-classification strategy and two algorithm versions reduce the rate distortion loss caused by misclassification,and ensure the encoding quality while improving the encoding speed.(3)To reduce the complexity of the coding block partition in HEVC,a new end-to-end fast algorithm is presented to aid the partition structure decisions of the coding tree unit(CTU)in intra coding.In the proposed method,the partition structure decision problem of a CTU is solved by a novel two-stage strategy.In the first stage,a bagged tree model is employed to predict the splitting of a CTU.In the second stage,the partition problem of a 32×32-sized CU is modeled as a 17-output classification task for the first time,so that it can be solved by a single prediction.To achieve a high prediction accuracy,a ResNet with 34 layers is employed.Jointly using bagged tree and ResNet,the proposed fast CTU partition algorithm is able to generate the partition quad-tree structure of a CTU through an end-toend prediction process,which abandons the traditional scheme of making multiple decisions at various depth levels.In addition,several datasets are used in this paper to lay the foundation for high prediction accuracy.