Study On Hole-filling Algorithms In Depth Image Based Rendering

Comparing with the conventional2D and3D TV, Free-View Television (FTV) isable to allow the user to choose view freely within a certain range. Therefore, it cangive users more powerful visual impact and extraordinary three-dimensional degree ofimmersion. As one of the key technologies of FTV, Depth Image Based Rendering(DIBR) combines the reference image and corresponding depth information effectively,so that the pixels in arbitrary new view image can be calculated through3D mappingequation. The technology of DIBR has become one of the most important researchtopics in the field of stereo video due to its advantages such as fast speed and highquality of rendering, less data storage space and transmission bandwidth.When using DIBR to draw arbitrary virtual view images, the holes appearunexpectedly due to the inaccurate depth maps and the changes of the occlusion whichresulted from the transformation of viewpoints. The existence of holes not only affectsthe image quality of rendering but also brings destructive interference to visualexperience of the user. As mentioned previously, a proper and effective hole-fillingtechnology is the key to obtain virtual view images with high quality. Several attemptshave been made in image inpainting field so far, especially the image inpainting basedon texture synthesis technology, which is able to give attention to both the structure andtexture of image information. All of these types of methods have a significant role inrepairing the complex scene. The classic exemplar-based image inpainting proposed byCriminisi et al. is used commonly in image restoration, video repairing, and film specialeffects production due to its remarkable inpainting effect.The Criminisi algorithm is mainly studied in this thesis. The more reasonableimprovement methods are put forward after a thorough analysis according to itsdeficiency on priority calculation formula and matching cost function.The main content of this paper is as follows:(1) A priority calculation function with depth information being added to improvethe precision of the inpainting order is designed. Considering Criminisi algorithm couldnot fully reflect the changes of the edge structure, especially for images with complexinformation, depth factor is introduced in this thesis as the auxiliary information for the judgment of the image edge structure.(2) The gradient information is used as auxiliary information when searching forthe best matching block. The Sum of Squared Difference (SSD) is the criterion forselecting a best source patch in Criminisi algorithm, which laid particular stress on thecolor difference among pixels. However, it cannot distinguish the structure and textureinformation of the image well. Therefore, the gradient which can reflect structure andtexture of the image to a certain extent is introduced to ensure more accurate searchingfor the best matching block.(3) A fast hole-filling method based on Depth and Gradient information isproposed to reduce the time cost of image inpainting. The improved algorithm canobtain better effect definitely, however, both the complexity of the algorithm itself andthe inpainting time increase significantly on account of the introduction of the depthfactor and gradient factor. To solve this problem, we combine the Criminisi algorithmwith the proposed algorithm in this paper to shorten the inpainting time, which is goodfor real time image inpainting.