Research On Rain Removal From Images Based On Deep Learning

Rain is a common weather condition that can directly affect the quality and visual effect of images,hindering many computer vision applications such as autonomous driving and outdoor monitoring.Therefore,research on image deraining is crucial in image restoration.Compared to traditional 2D images,continuous video frames can record rich spatial-temporal information of the target scene.By utilizing the temporal correlation information between adjacent frames and the current frame,significant improvements in the quality of video image deraining can be achieved.This provides a huge advantage in image deraining tasks.Therefore,this paper proposes a fast spatial-temporal aggregation method for video image deraining and a multiscale feature fusion method for semi-supervised video image deraining.The main content includes:1)A spatial-temporal aggregation network for fast video deraining is proposed.To fully utilize the spatial-temporal information of continuous video frames,a dual-branch spatialtemporal feature extraction module is used to extract temporal correlation information and spatial representation information of consecutive video frames,which are then adaptively aggregated by the spatial-temporal feature aggregation module.Finally,the temporal refinement module is used to further refine the corresponding region of the current frame with the rain-free information of adjacent frames and restore the rain-free video frame.By using a lightweight encoder-decoder structure to extract spatial features of the current frame and optimizing the temporal correlation extraction structure between adjacent frames,the speed of video image deraining is accelerated and the efficiency of deraining is improved.Experimental results show that the proposed network can quickly and effectively remove rain streaks from video frames and outperforms other methods in terms of performance.2)A semi-supervised video deraining method with multi-scale feature fusion is proposed.Rain streaks in the air show obvious self-similarity at the same or different scales.First,continuous video frames are input into a multi-scale feature extraction module to extract contextual information within frames and long-range dependency between frames.Then,a multi-scale feature fusion module is used to accurately extract rain streak information by fusing features from different scales,followed by a recursive neural network to restore the background.To improve the robustness in synthetic scenes and the generalization ability in real scenes,supervised and unsupervised losses are respectively used to constrain the model training.Experimental results show that quantitative and qualitative evaluations on synthetic and real scenes demonstrate the effectiveness of the proposed method.