Spatial Temporal Refinement On Depth Video Processing

With the development of digital technology, great progress has been made in 3D display technologies. Traditional 2D video and games can no longer satisfy people's entertaining requirement. And 3D technology becomes a key problem in the next generation display technology. Among these, system based on binocular parallax is the simplest and fastest growing stereoscopic display mode. So the geometry of stereoscopic camera and display system is present first. This is followed by the analysis of rules of disparity distribution in different stereoscopic display system.There are lots of commercial applications in 3D display areas such as viewpoint synthesis, stereoscopic video codec and 3DTV, need to use an exact and smooth depth video sequence for post-processing. But estimating a temporal-coherent dense depth video from stereo video sequences has not been extensively studied or considered. Fortunately, a large number of algorithms have been proposed to solve the problem of stereo matching in image. In this thesis, we proposed a novel method based on segmentation and local optimization to estimate the disparity map of two stereoscopic images. Combined with mean-shift segmentation, we use an adaptive cost aggregation method to enhance the robustness of matching cost. Assumed a region with homogenous color will have similar disparity, we utilize histogram analysis to select seed disparity, then use seeds expansion and plane fitting algorithm to estimate the disparity. After histogram optimization and sub-pixel refinement, we can obtain an exact and smooth disparity map. When testing the Middlebury standard stereoscopic image pairs, the results show that our algorithm is very robust and effective.For depth video recovery, a straightforward method is to apply the standard stereo matching algorithm frame by frame. However, obtaining high-quality dense depth map is still a challenging problem due to many inherent difficulties, and the invalidation of disparity smoothness assumption and absence of time consistency may-lead to the depth video has lots of "popping" artifacts, disparity fluctuation or mismatching regions. This thesis presented a new method for estimating temporally consistent disparity maps for stereo video sequences. The proposed method incorporates the color constancy constraint and the geometric coherence constraint in a non-iterative stereo matching process. It uses the disparity smoothness of overlapped segmentation between adjacent frames based on the affine transformation to constrain the temporal consistency. And a similarity checking based on the spatial geometrical coherence is used to refine the disparity result. Experimental result shows the proposed algorithm can reduce the "popping artifacts" and over-segmentation obviously, and yield satisfactory disparity video result.