Single Image Deraining Based On Continuous Density Prior And Cycle-Consistency

Image deraining aims to remove the rain streaks in rainy image and recover the clean background,with applications to outdoor surveillance,autonomous driving,etc.Recently,deep learning with neural networks as the core has greatly promoted the development of image deraining.At present,most of deraining algorithms have achieved remarkable progress on synthetic rainy images by designing excellent deraining networks.However,when applied to real-world scenes,there are still some issues:(1)These methods are not robust enough to different types of rain streaks due to ignoring the prior information of real rain;(2)Their deraining results of real rainy images mainly rely on qualitative analysis and cannot be quantitatively evaluated,due to lacking of corresponding ground-truth images;(3)These methods trained on synthetic data often generalize poorly to real-world rainy images.This thesis will present solutions to the above problems and verify their effectiveness and superiority in experiments.The major focus and contributions are as follows:(1)For ’Rain Prior’ issue,considering the continuous state of real rainfall,we propose a COntinuous DEnsity-guided Network(CODE-Net)for image deraining by(i)introducing weight parameters,utilized to adjust the sparsity of sparse representations and meanwhile reflect the rain density,into conventional Convolutional Sparse Coding(CSC);(ii)solving the weighted CSC using Deep Unfolding,i.e.,unrolling classic iterations into limited neural layers and training the whole network in an end-to-end manner.Besides,we further sparsely encode the rain streaks on multiscale dictionaries and present a multiscale CODENet(mCODE-Net),with the consideration of multiscale appearance due to different captured distance.Compared with the existing deep learning methods,our proposed networks,derived from traditional sparsity model and integrating the continuous intensity prior,are with better interpretability and robustness.(2)For ’Evaluation’ issue,based on the weighted CSC model and its learnable weights,by analyzing the weights related with rain density,we propose a simple and effective Rain Density Estimation(RDE)method.RDE could quantify the rainfall degree of the input image in a continuous state,in other words,it is able to quantitatively reflect the cleanliness of the deraining results.Different from the commonly quantitative evaluation methods such as PSNR and SSIM,RDE can quantitatively evaluate rain removal performance directly without labeled images.(3)For ’Generalization’ issue,based on the supervised model,we propose a cycleconsistent semi-supervised framework attempting to feed unsupervised real rainy images into the network training,which enables the network in learning to derain using synthetic dataset while generalizing better using unlabeled real-world images.Specifically,cycle-consistency conducts the alignment for cross-domain distribution in sparsity space and forces the network to learn common features,which avoids the bias of network towards learning the specific patterns of the synthetic rain and improves the generalization of network in real scenes.(4)We in experiments first verifies the effectiveness of continuous rain intensity estimation,including implicitly considering density prior,evaluating unlabeled deraining results and improving the performance of other methods.Later,compared with recent state of the arts,experiments on synthetic and real-world data demonstrate the effectiveness and superiority of our methods,in terms of both quantitative and qualitative results.In addition,we validate that our methods can benefit other visual tasks of image restoration and object detection.