Deep Learning-based Methods For Single Image Deraining

Outdoor visual devices often suffer from the interference of rain streaks when capturing pictures on rainy days.These rain streaks not only obscure background details,but also have similar shape and color to some objects,thus affecting the execution efficiency of subsequent tasks such as object recognition and detection.Therefore,it is important to execute the task of image deraining during the pre-processing of visual tasks and is imperative to study the deraining algorithm.In recent years,deep learning has made significant breakthroughs in various image processing tasks.However,the existing deraining methods still have two shortcomings,one is that they tend to over-blur the background details on extremely synthetic rainy images,and another is that their applicability are limited in real scenarios.In order to alleviate these shortcomings,this thesis combines deep learning techniques to conduct in-depth study on single image deraining.The main contents and contributions are summarized as follows:Firstly,a recurrent deraining approach based on the multi-scale structure is proposed.In order to remove rain streaks while restoring clean background details as much as possible,we design a multi-scale recurrent network with two cascaded branches,among which,the first branch is used for removing rain streaks progressively,and the second branch is used for restoring clean background details.By jointing these two branches for training,the proposed network can optimally balance the ability of rain streaks removal and background restoration.In addition,we propose a multi-scale residual squeeze-and-excitation block and use it in each branch to extract the corresponding multi-scale information.Experiments comparison and ablation studies demonstrate the effectiveness of the proposed method on the task of single image deraining.Secondly,a deraining method guided by physical similarity-diversity is proposed.In order to improve the applicability of deraining approach in real scenarios while maintain the interpretability on the physics level,we first explore the inherent relationship between the rain model and the haze one established up to date.We discover that these two models experience similar degradations in the low-frequency components(i.e.,similarity)but diverse degradations in the high-frequency areas(i.e.,diversity).Based on these observations,we develop a Similarity-Diversity model to describe these characteristics.Afterwards,we introduce a novel deep neural network to restore the rain-free background,which guided by the similarity-diversity model,namely deep similarity-diversity network(DSDNet).Extensive experiments demonstrate the effectiveness of our DSDNet both on synthetic and real rainy images.Moreover,our method can also promote the recognition rates of vehicle and pedestrian with Google Vision API on rainy images.Thirdly,a conv-patches enhanced attention deraining network is proposed.Long-range dependency is vital for single image deraining.In order to capture such dependencies from rainy images precisely,we first develop a novel conv-patches enhanced criss-cross self-attention module to efficiently collect the neighborhood information around each pixel combined with convolution operation,which are then used to assist the computation of subsequent attention map and thereby improving the ability for modeling long-range dependencies.Furthermore,we put forward to a dual attention enhanced U block as the basic unit of the proposed attention network,embedding with two complementary attentions to calibrate the feature responses during aggregation process.Finally,dense connection are utilized to joint several dual attention enhanced U block to improve the utilization efficiency between the features with different levels.Experimental results demonstrate the effectiveness of our network for single image deraining.In addition,we apply our architecture to the image dehazing task and still achieve satisfactory performance.