Research On Interactive Stereo Image Segmentation And Inpainting

Nowadays,3D technology is becoming more and more popular.The rapid increase in the amount of stereo and video camera devices has promoted research on efficient editing of stereo images.As one of the hot research topics,interactive stereo image segmentation and inpainting faces prosperous application prospect in the fields of stereoscopic image editing,3D video post-processing,etc..Interactive stereo image segmentation allows a user to indicate the foreground and background with strokes or other forms in either of the stereo views and achieves intelligent extraction of target regions through algorithms.The user can optimize segmentation results by adding more strokes.The main criterion for evaluating interactive stereo image segmentation algorithms is the amount of interaction required.Stereo image inpainting refers to repairing holes left in the stereoscopic image after removing the foreground.The main criteria for evaluating stereo image inpainting algorithms are the quality of repaired contents and their stereo consistency.Existing stereo image interactive segmentation and inpainting technologies still could be improved in operation intelligence and visual restoration effects.Therefore,this thesis aims to study more intelligent extraction of foreground regions from a pair of stereo images,and better inpainting results on the left incomplete images.Existing interactive stereo image segmentation methods are mainly based on the framework of Graph Cut.These methods guide the segmentation by modeling the color and gradient clues while ignoring the function of depth information in stereo image segmentation.Existing stereo image inpainting algorithms are generally based on the Exemplar-based algorithm,ignoring the semantic information of the overall image and failing in repairing large holes.Inpainting based on convolution neural network(CNN)can consider the global semantic information,but it has only been studied on single images.There is no algorithm for stereo image inpainting based on CNN.The main work and major contributions of this paper are summarized as follows:(1)By analyzing the function of color,disparity and other clues in the segmentation of stereoscopic images,we propose a method for interactive stereo image segmentation with RGB-D hybrid constraints.Based on an existing color cues-based segmentation algorithm,the proposed method linearly integrates the depth/disparity cues to guide the segmentation.Experiments prove that our algorithm has improved accuracy compared with other algorithms with the same amount of interaction.(2)We fully explore the importance of different types of prior clues for different images and regions in interactive stereo image segmentation.We propose and demonstrate the argument that in interactive stereo image segmentation of different images,different regions,and even different pixels in the same image need different priors.According to this argument,an interactive stereo image segmentation algorithm with adaptive prior selection is proposed.This algorithm can select the optimal prior model for each pixel during segmentation.Experiments show that with the same amount of interaction,the algorithm performs best compared with other algorithms.(3)Based on the framework of convolutional neural network for single image inpainting,we fully explore the characteristics of stereo images and formulate them in the inpainting task.The consistency constraints for ensuring left and right views after stereoscopic image inpainting is thoroughly studied.We propose a stereo image inpainting algorithm based on convolutional neural network.This algorithm guarantees the consistency of visual perception and disparity perception after stereoscopic image inpainting,through a specially designed network architecture and loss function.Experiments show that our algorithm performs better than other algorithms in terms of visual perception and stereo consistency.