| In recent years,with the development of multimedia and 3D technology,free-viewpoint video(FVV)has attracted much attention,as it provides experience to users to observe a video from different viewpoints.To provide such an experience,FVV technology needs to capture a number of camera views from different viewpoints.However,capturing and transmitting the required number of views is not an effective solution.The emergence of virtual viewpoint synthesis technology based on depth image-based rendering supports the need to synthesize more viewpoints with a limited number of viewpoints.Although DIBR technology is a simple method,it has the problems of overlap,cracks and holes,which affects the quality of the synthesized view.The purpose of this thesis is to solve the above problems and improve the quality of synthesized view,so as to promote the widespread application of DIBR technology in FVV.This thesis conducts in-depth research for the above problems.The major research achievements and innovations are as follows:(1)A depth-based epipolar sorting algorithm is proposed to solve the overlap and crack problems in DIBR technology.This method combines the ideas of epipolar sorting algorithm and Z-buffer algorithm to deal with overlap and crack problems in the process of 3D image warping.At the same time,the method also marks and tracks edge pixels of object by detecting depth value jump in depth map to keep edge structure of object during processing.(2)A hole-filling method based on generative adversarial network(GAN)is proposed to solve the hole problem in DIBR technology.In this method,the edge of hole region is predicted by edge completion network,and then holes are filled by hole-filling network which guided by predicted edge.In the hole filling stage,coarse to fine network structure,color difference compensation module and contextual attention layer are also introduced to improve the quality of hole-filling.(3)A position attention mechanism based on image re-projection is proposed to solve the problem of filling background holes with foreground information in hole-filling method based on generative adversarial network.The method calculates the rotation angle and vertical translation distance required for the vertical alignment between the destination image and the reference image by image re-projection,and then calculates the horizontal disparity between the destination image and the reference image according to the 3D image warping relationship,so as to simulate the hole position and shape of destination image during network training,thus further improving the quality of the hole-filling in destination image.In this thesis,subjective and objective evaluations of the proposed method are performed on multiple datasets.Experimental results show that the method proposed in this thesis can generate virtual viewpoint images with good visual quality.The above research achievements can provide a certain reference value for the application of free-viewpoint video. |
