Thick Cloud Detection And Thin Cloud Removal In Remote Sensing Image Based On Multi Scale Features

In recent years,the number of land satellites with earth observation as the main task has seen an explosive growth,among which optical remote sensing images have been widely used in land use classification,urban planning,crop yield estimation,disaster monitoring and other fields.However,optical remote sensing images are easily affected by clouds.In order to extract and recover the damaged information in remote sensing image,thick cloud detection and thin cloud removal are the key steps of remote sensing image pretreatment.Different processing methods are generally adopted for thick clouds and thin clouds.Thick clouds completely block the surface information and generally focus on detection.Thin clouds can transmit part of the surface information,mainly for removal.In recent years,deep learning,which is popular,has also been used in thick cloud detection and thin cloud removal,and has achieved good results.But currently used in more depth study of the supervision of all methods,need a lot of reference image as the study object,the model training requires a lot of artificial marking samples,time-consuming,and huge models,and the number of millions,there are a large number of redundant,and most of the method only using the high spatial resolution of multispectral image bands.In order to solve these problems,this paper takes medium and high resolution remote sensing images as the research object,uses published Landsat 8 and Sentinel-2 images to carry out research on thick cloud detection and thin cloud removal methods,and proposes a block scale weakly supervised thick cloud detection model and a thin cloud removal model taking into consideration the bidirectional transmission of clouds.And the thick cloud detection method and thin cloud removal method based on multi-scale feature fusion.The main work and contributions of this paper are as follows:1.A weakly-supervised cloud detection model based on block level labels(CDBLL)is proposed.The model combines the generative adversation network with the attention mechanism,which can realize the accurate prediction from block level labels to pixel level labels.Considering the supervised thick clouds detection methods based on deep learning reqiures a lot of manually labelled cloud masks as the training data,and labelling the cloud mask is very time-consuming,this paper proposes to use block level labels(whether there is a cloud in the image)of images to train a model for pixel level classification(whether the pixel is cloud in the image).By analyzing the visual characteristics of thick clouds in remote sensing images,this paper assumes that the most significant difference between cloud image and clear image is cloud.From the perspective of solving the difference among images and where the difference is,an attention mechanism that can accurately locate the significant difference of images is constructed to realize the detection of thick clouds.In order to solve the problem that attention would spread to the whole image,the attention shrinkage mechanism was designed to constrain model only focused attention on the thick cloud regions.Aiming at solving the problem that the highlighted ground objects are easily confused with clouds,an attention optimization mechanism is proposed.For the completely clear images,a reference mask is introduced to make the model pay no attention to clear images.In order to solve the detection of significant difference among the images requires images that have space consistency,this paper proposes to use clear images from other area as reference to cloud images from the current area and introduce generative adversarial networks to measure the significant difference between them,then guide attention mechanism to optimize in the right direction and put more attention on the thick cloud resions.The experimental results show that the CD-BLL achieves the overall accuracy of 82.93% and 96.56% on Landsat 8 and Sentinel-2 thick cloud detection datasets.2.A cloud removal model considering two-way transmission in physical model of cloud distortion(CRTTPM)is proposed.In this paper,the interaction between solar radiation and thin clouds is firstly analyzed,and the downward transmission and absorption of thin clouds are considered to build a new cloud distortion physical model,which can more accurately describe the interaction process between solar radiation transmission and thin clouds.Aiming at the problem that the two-way transmission process is complicated and difficult to separate,a correction term is introduced to make the model satisfy an identity,which is convenient to solve the parameters.By assuming that the reflectivity of the ground object is strongly related to its own attributes and weakly related to thin clouds,the extraction principle of cloud distortion layer in thin cloud image is established,and the cloud distortion layer is extracted by using generated adverse network.Considering that the image in which thin cloud is removed has strong correlation with the original thin cloud image,the image reconstruction process is designed to ensure that the background information is retained as much as possible while the thin cloud is removed.According to the principle that the clear area should not be changed during thin cloud removal,an optimization process is designed to make the model only change the information of the thin cloud area and keep the cloudless area unchanged.In the Sentinel-2 thin-cloud removal dataset distributed globally,CRTTPM has a peak signal-to-noise ratio(PSNR)of over 21.49 and a structural similarity of over 0.79 in the visible and near-infrared bands.3.A cloud detection algorithm fusing multi-scale spectral and spatial features(CDFM3SF)and a thin cloud removal algorithm fusing multi-scale spectral and spatial seatures(CR-FM3SF)are proposed.The two algorithms use multi-input/output full convolutional network to automatically fusion multi-spectral and multi-resolution bands to improve the accuracy of thick cloud detection and thin cloud removal.Aiming at a low resolution of multispectral satellite generally contain the short-wave infrared wavelengths,such as Sentinel-2 A/B,CBERS 02-04,ZY-1 D and HJ-1 B,etc.,most of the thick cloud detection and cloud removal method only deals with high resolution band or resample all bands to the same spatial resolution and process them together.The former can not make full use of the spectral information of other bands,the resampling process will introduce noise.Multiple full-convolution input branches are proposed to automatically process multi-resolution images layer by layer while fusing multi-spectral features.Multiple output branches are designed to provide supervision information for thick cloud detection at different scales and thin cloud removal at different bands according to learning objectives.At the same time,the convolutional neraul networks have a lot of parameters which makes it difficult to train.We propose mixed depth-wise separable convolution module(MDSC),double-path depth-wise separable convolution module(DDSC)and shared dilated convolution residual block(SDRB),parallel down-sample residual block(PDRB),and Concatenation and Sum and CS+.MDSC and DDSC can extract and fuse multi-scale features while greatly reducing network parameters.The SDRC can expand the receptive field exponentially without increasing the parameters,and can eliminate the grid effects caused by dilated convolution.PDRB,CS and CS+ structures are designed for different subsampling strategies in thick cloud detection and thin cloud removal methods respectively.A fast channel of multi-scale feature fusion is established without adding parameters by susing PDRB,CS and CS+,so that the output of each layer can reach the bottom layer.The CD-FM3 SF achieves 98.57% accuracy on the Sentinel-2 thick cloud detection data set distributed in China.CR-FM3 SF has a peak signal-to-noise ratio(PSNR)of over 31.03 and a structural similarity of over 0.90 in the globally distributed Sentinel-2 thin cloud removal dataset.4.A wuhan university cloud detection and removal dataset(WHU-CDR)which includes block level labels is produced.It contains 5 subsets:· WHUL8-CDb is for Landsat 8 images,its samples are distributed in 285 areas around the world,covering 9.69 million square kilometers;WHUS2-CDb is a Sentinel-2 dataset,with samples distributed in 94 areas in China mainland, covering 1.12 million square kilometers.Both of them are the first thick clouds datasets with block level labels.· WHUS2-CDV improves cloud mask resolution to 10 meters for the first time. It is a Sentinel-2 validation dataset with samples distributed in 36 areas in China mainland,covering 430,000 square kilometers;· WHUS2-CRv is a Sentinel-2 thin-cloud removal dataset covering 1.47 million square kilometers of short-interval thin-cloud/cloudless image pairs distributed in 123 regions around the world;· WHUS2-CRb is the first Sentinel-2 cloud removal dataset with block level labels,in which the thin cloud and clear samples are distributed in the 246 areas around the world,covering 2.94 million square kilometers.