Research On Key Technologies Of Content Editing For Stereoscopic Images

With the rapid development of multimedia and 3D display technology,stereoscopic image resources are continuously enriched,which have been widely applied in many fields,such as digital video,industrial design,virtual reality,augmented reality,and so on.As one of the effective methods to obtain high-quality stereoscopic images,stereoscopic image content editing has become one of the hot topics in the field of image processing.Stereoscopic image content editing aims to utilize binocular stereoscopic disparity principle to adjust the resolution,aspect ratio,and other attributes of stereoscopic image.Since stereoscopic image contains the disparity and depth information between left and right views,the image content editing faces the problems of content distortion and disparity distortion.Therefore,how to jointly optimize the shape and depth perception of stereoscopic image,and obtain high-quality stereoscopic image content editing result in the process of stereoscopic image content editing,has important research significance.This thesis mainly studies to improve the depth perception quality of stereoscopic image by content editing.Comprehensively considering the stereoscopic visual features of the human eye and stereoscopic image data features,this thesis focuses on three aspects of stereoscopic image content editing,which includes disparity-constrained guided stereoscopic image stitching,deep stereoscopic image retargeting,and unsupervised stereoscopic image retargeting:1.Disparity-constrained guided stereoscopic image stitching.Stereoscopic image stitching aims to obtain a panorama with wide perspectives by stitching multiple stereoscopic images with overlapping areas.In order to obtain high-quality stereoscopic image stitching results,this thesis presents a stereoscopic image stitching method based on disparity-constrained warping and blending.First,a point-line-driven homography based disparity minimization method is designed to pre-align the left and right images and reduce vertical disparity.Afterward,a multi-constraint warping is proposed to further align the left and right images and preserve binocular disparity information.Finally,a disparity consistency seam-cutting and blending method is presented to determine the optimal seam and conduct stitching.Experimental results demonstrate that the proposed method can eliminate ghosts and prevent the disparity distortion of the stereoscopic images,and effectively improve the stereoscopic image stitching performance.2.Deep stereoscopic image retargeting.Stereoscopic image retargeting aims at converting stereoscopic images to the target aspect ratio.Different from plane image retargeting,stereoscopic image retargeting needs to preserve both the shape structure of salient objects and the depth consistency of 3D scenes.Therefore,this thesis presents a stereoscopic image retargeting method based on convolutional neural network.First,a cross-attention extraction module is constructed to generate attention map,which contains the attention features of the left and right images and the common attention features between them.Second,a disparity-assisted 3D significance map generation module is utilized to further preserve the valuable depth information of stereoscopic images.Finally,in order to accurately predict the stereoscopic retargeting images,an image consistency loss is designed to preserve the geometric structure of salient objects and a disparity consistency loss is developed to reduce depth distortions.Experimental results demonstrate that the proposed method can provide favorable stereoscopic image retargeting results.3.Unsupervised stereoscopic image retargeting.Considering the diversity of display devices,it is difficult to construct a stereoscopic image retargeting dataset with different aspect ratios and true labels.In order to meet actual application requirements,this thesis proposes an unsupervised stereoscopic image retargeting method based on depth perception optimization to address the problem of stereoscopic image retargeting without label information.First,considering the inter-view correlation of stereoscopic image,a view synthesis loss is proposed to promote the generation of high-quality stereoscopic images with accurate inter-view relationship.Second,by exploiting the consistency of stereoscopic images before and after the retargeting,a stereo cycle consistency loss,which including a content consistency term and a disparity consistency term,is developed to preserve the object structure and depth information.Experimental results demonstrate that the proposed method can effectively improve the stereoscopic image retargeting performance.