Research On Method Of Synthesizing Multi-views By DIBR And Process Of Hole-filling

Technologies of stereoscopic TV are all based on the basic principle that parallaxproduces stereoscopic effects. When watching3DTV, the left and right eye ofaudience receives different images, which will be synthesized into stereoscopic sceneswith certain depth perceived by human eyes. The capturing, transmission and displayprocess of traditional3DTV system relies on two separate video streams, one for theleft eye and one for the right eye. However, there are some disadvantages in thepractical applications of traditional stereoscopic TV system, the most serious one isthe backward-uncompatibility with2D displays.To conquer these limitations, Christiph Fehn proposed a novel method ofDepth-Image-Based Rendering in2002, namely DIBR in short. This method is basedon a more flexible joint transmission of monoscopic color video and associatedper-pixel depth information. Then one or more virtual views of3D scene will besynthesized from this data representation at the receiver side to help audience enjoythe stereoscopic videos. The key thought of DIBR is3D image warping equation,which proposed by McMillan in1997. In addition, C.Fehn combined3D imagewarping equation with shift-sensor algorithm together to fix the zero-parallax planeon the screen of display. Not only does it eliminate unwanted vertical parallax, butalso simplifies the synthesis of virtual views further.In practical3DTV system, screen parallax is usually set at the sending terminalby experience and utilized to generate corresponding stereoscopic video at thereceiver. It's obvious that the permanent screen parallax couldn't satisfy all requests ofdifferent situations. Preference for screen parallax, viewing position, the size ofdisplay and other external elements will have great influences on the visual effect of 3DTV. As far as the preference of audience is concerned, people from different agegroups have different preference for screen parallax. Older people were much lesssensitive than young adults to perceiving stereoscopic depth. Moreover when theviewing position of audience changes from the near to the far, he/she will also feel thedrastic variation of depth of3D scenes. It's the same to the size of display, which isproportionate to the perceived depth. As a result, to create adaptive screen parallax atthe receiver, the paper proposes an observation model in standardized proceduresbased on DIBR method. This observation model refers to human visual system andutilizes human eyes to substitute for the camera system in shift-sensor algorithm. Inthis way, the screen parallax is fixed at the most comfortable size for human eyes.Only setting the value of one parameter can customers adjust the size of screenparallax easily, without worrying about the degradation of visual effect ofstereoscopic TV.During the generation of a "virtual" view with the method of DIBR, areas, whichare occluded in the original view, might become visible in any of the "virtual" left-and right-eye views. Since these areas are occluded in original view, informationabout these previously occluded areas is neither available in the monoscopic colorvideo nor in the accompanying depth-images. How these disocclusions should betreated during the view synthesis has become a problem crying for solution. Thispaper proposes a new system to generate virtual views based on DIBR method, whichincludes preprocessing and hole-filling process to fill holes efficiently and improvethe quality of images. In preprocessing, Gaussian filter is adopted to smooth areaswith sharp changes in depth image. In the process of hole-filling, it mainly depends onedge oriented interpolation.