Research On Virtual View Synthesis Algorithm Based On DIBR

With the development of computer science and multimedia technology,the transmission speed and quality of images are constantly improved.Traditional two-dimensional display technology is difficult to meet people’s needs in life,work and entertainment.The superior performance of 3D display technology in terms of reality and interactivity makes it gradually become one of the next generation multimedia technologies with great development prospects.At present,3D video has been widely used in many fields such as education,medical,and entertainment to provide users with an immersive visual experience.As the ultimate of 3D video,free viewpoint video allows viewers to freely choose the viewpoint within a certain range while experiencing 3D visual effects,thereby providing a better interaction experience.Free viewpoint video has broad application prospect in the field of broadcast live,interactive courses,military strategy research,etc.,and has become a research hotspot in the 3D display field.With the increase of viewpoints,the amount of data has increased geometrically,which puts great pressure on the collection,storage,and transmission of video data.In order to solve this problem,virtual view synthesis technology is proposed.Among the view synthesis methods,depth-image-based rendering(DIBR)technology has attracted wide attention due to its advantages of low bandwidth requirements and fast rendering speed.However,during the view synthesis process,the occlusion relationship between objects in the scene and the imaging range of the camera are changed,causing holes to appear in the virtual image.In addition,errors in the acquisition of depth information and the calculation of DIBR also leads to artifacts.In order to improve the quality of virtual view,this article analyzes the characteristics of various artifacts generated in the DIBR process,and conducts an in-depth study on the artifact handling and image quality improvement algorithms.The main work and innovations are as follows:1.Based on the causes and distribution characteristics,artifacts generated in the virtual image are divided into multiple types,including cracks,ghosts,pixel overlaps,and holes.The holes can be further subdivided into two types: disocclusions and out-of-field areas.The former represents the holes caused by the change of occlusion relationship between the foreground and background during the viewpoint conversion process.The latter means that the virtual view exceeds the imaging range of the reference view,resulting in holes on the edge of the synthesized image.On this basis,An artifact handling method based on depth image is proposed to treat each artifact type according to their features.Since the depth image reflects the distance between the object and the camera,and can be used to determine the occlusion relationship between objects in the scene,it is introduced to guide the artifact handling.Aiming at the out-of-field areas,a boundary expansion method is proposed to increase the imaging range of the reference view.A depth image preprocessing method is proposed to correct the ghost pixels at the edge of foreground so that they can be projected to the correct position.For cracks and pixel overlaps generated in DIBR,an adaptive one-to-four method is proposed to fill the cracks and maintain the correct occlusion relationship between objects based on the depth information.For disocclusion filling,a local background-first inpainting method is proposed.The priority calculation and the best matching patch searching are improved,and the depth information is introduced as the auxiliary.Experimental results show that compared with the original DIBR algorithm,the proposed method can significantly improve the rendering quality of virtual view.2.A disocclusion-type aware hole filling method is proposed.Disocclusions are divided into two types based on the classification results of their boundary pixels:foreground-background disocclusion and background-background disocclusion.For the former,an adaptive depth image preprocessing method is introduced to adaptively decompose the disocclusions.For the latter,the corresponding foreground objects are removed,so that the disocclusion can be filled with the reliable background texture.By applying targeted processing methods,the disocclusion filling results are improved.3.A local foreground removal method is proposed for disocclusion filling.Based on the disocclusion edge pixels,local foreground regions associated with the disocclusion are removed from the reference image and its depth image.The removed regions are filled with the texture and depth information of the surrounding background.Experimental results show that after the predicted occlusion layer content is warped to the virtual view through the modified DIBR,it can effectively fill the disocclusions in the virtual image.Compared with the previous methods,the proposed method has better subjective visual quality,and has a significant improvement in objective evaluation metrics.4.For multiple view rendering,a virtual view synthesis method based on asymmetric bidirectional DIBR is proposed.Based on the single view rendering result,the region corresponding to the hole is searched in the auxiliary image.For the auxiliary image,only the selected regions are warped,which contain the contents that are not visible in the primary view.The proposed method can effectively reduce the computational complexity while maintaining the hole filling effect.Experimental results show that compared with bidirectional DIBR,the proposed rendering method can reduce about 37% rendering time and achieve 97% hole reduction,which effectively reduces the computational complexity of multiple view rendering.