Research On The Virtual Viewpoint Synthesis Algorithm Based On The DIBR

When the two-dimensional video has been popular, stereoscopic multi-view video has gradually become a new hot spot in the field of video. Stereoscopic multi-view video can give the audience an immersive experience, so that it has been widely used in 3DTV, video conferencing. Considering that the stereoscopic multi-view video data is very large, it brings tremendous challenges to the compression and transmission. One method to solve the problem is to take sparse camera array to capture video, and render the virtual viewpoint before the video display. In the virtual viewpoint synthesis, it will suffer many problems such as mapping duplication, bad pixels, cracks and holes. If these problems are solved well, we will get a high quality virtual viewpoint image. In this paper, the virtual viewpoint algorithm has been researched and improved.Firstly, this paper proposes the spatial smoothing algorithm and the time domain smoothing algorithm for the depth map. In the original depth map, suffering from its low accuracy depth information, there are plenty of pixels with wrong depth information. In the spatial smoothing algorithm, the gradient is adopted as the weights for filtering to correct wrong depth information. In the time domain smoothing algorithm, the consistency between neighbor frames is utilized to smooth the current depth map. By using such two algorithms, the quality of depth map can be effectively improved.Secondly, in this paper, the discontinuity eliminating algorithm is used during the three-dimensional mapping process. In the traditional three-dimensional mapping algorithm, the mapping duplication has been solved. But the bad pixels and the cracks are still unsolved. Different from this, the proposed method firstly use neighbor pixels to judge whether the mapping point is bad or not. If the mapping point is bad, the point will be abandoned, and neighbor pixels are used to recover its pixel information. After that, it can be judged whether there exists a crack between two consecutive mapping points. For the crack, the two consecutive mapping points can be used to fill its missing information, which can get the high quality map.Finally, the reliability-based filling algorithm is used to enhance the merged virtual viewpoint image. Since the occlusion can generate holes, background pixels will be used to fill these holes. In details the distance between each background pixel and the hole will be used to calculate its weight. By filling holes, the quality of virtual viewpoint image will be improved.Extensive experimental results demonstrate that the proposed algorithm can generate better virtual viewpoint rendering results, which is quite useful in multi-view video system.At the end of the paper, the free virtual viewpoint rendering is proposed and implemented.