Innovative Hole-filling Method For Depth-image-based Rendering(DIBR) Based On Context Learning

High quality 3D content is the key factor for the long-term sustainable development of stereo display technology.With the increasing resolution of 3D display and the increasing number of view points,stereoscopic display content is facing new challenges in terms of generation speed and processing algorithm.Due to the complexity of 3D content scenes and the difficulty of extracting depth information,it is difficult to generate 3D images directly fr-om 2D images.Traditional 2D-to-3D image requires a lot of manpower and time.The hole generated in the process of 3D image generation from 2D+D seriously affect the viewing experience.How to fill the hole naturally,truly and efficiently is the difficulty of virtual viewpoint generation.In order to improve the processing efficiency and quality of large parallax,super multi-view and high resolution 3D content,new processing methods and algorithms are needed to realize automatic prediction and filling of empty content.Based on the deep neural network,this paper studies the method of filling the hole in the content generation of large disparity virtual viewpoint,improves the quality of the generated viewpoint and improves the efficiency of viewpoint generation.The main innovations and research contents of this paper are as follows:1.An algorithm is proposed to take both global and local information into account in the process of hole filling,and the image pyramid pattem is used to generate hole filling search windows at different scales.Three traditional hole repair algorithms are optimized,which are example-based,block-based and nearest neighbor-based.Compared with the traditional algorithm,the image restoration algorithm proposed in this paper has the advantages of high stability,fast repair speed and high efficiency in repairing large voids.2.A large disparity void repair model based on deep learning is proposed,which realizes the void repair in the process of large disparity virtual viewpoint generation.The proposed neural network realizes the prediction of the whole semantics of the image by self-encoding,and generates the antagonistic network to make the void filling part more clear and real.On this basis,this paper also creatively proposes a style prediction network,which considers that the texture style of the area around the void should be similar to the texture style of the void.Therefore,the style around the void should be extracted and integrated into the void filling area,which greatly improves the authenticity of the restoration.Compared with the current mainstream hole filling algorithm based on deep learning,Mean L1 Loss(L1 loss function)and Maan L2 Loss(L2 loss function)decreased by 0.42 and 0.09 respectively,and PSNR(peak signal-to-noise ratio)increased by 0.38.3.A progressive large parallax cavity image restoration model is proposed,which solves the problem that the hole filling algorithm based on depth learning can only deal with low-resolution images.In this paper,we adopt an iterative image generation mode fr-om small to large.Firstly,a series of high-resolution images to be filled are sampled down to a certain resolution,and then the low-resolution images are filled with holes.Then,after a series of up-sampling,the holes are filled sequentially,and the holes in high-resolution images are filled step by step.According to this model,the hole filling of 1080x1920 resolution image is realized.The average time of a picture is 0.1s,and the signal-to-noise ratio is 17.9db,which is better than other existing models.