Semi-Supervised Implicit Code Decoding Based Sever Weather Restoration

With the successful application of deep learning in computer vision,many methods are now applied to severe weather recovery tasks,such as rain,fog,snow and dust removal tasks.At present,however,almost all algorithms can only recover from a particular type of weather,rather than using a unified algorithm for multiple types of severe weather.This leads to the fact that in the deployment process of the actual bad weather recovery algorithm,only one front-end algorithm can be used to determine the type of weather,and then a specific back-end algorithm can be selected to restore the weather.The main problems with this type of deployment are twofold.Firstly,because of each weather must deploy a specific algorithm,so the required parameter quantity is large.Secondly,the effects of bad weather recovery will largely depend on the front end for weather classification.But in actual implementation,this paper find that the quality of the classifier is not ideal,that is to say the quality problem of the front-end algorithm will greatly influence the final recovery of bad weather.For the current academic and industrial,severe weather recovery algorithms still have problems.This paper aims to design an algorithm based on deep learning that is applicable to a variety of severe weather.This will not only reduce the number of deployment parameters,but also reduce the poor quality of front-end bad weather discrimination resulting in poor final bad weather recovery.At present,the biggest problem in constructing a unified recovery algorithm for severe weather is that the number of marked images for severe weather is insufficient.For haze weather,there are a large number of public data sets;for rainy days,there is a moderate amount of data;but for snow and dust weather,there is hardly any marked data.Therefore,this paper firstly conducts clustering for severe weather,and finds that dust weather and fog weather,rainy days and snow days have very similar features,which can be treated in the same category of weather.In this way,the quantity demand for the marked data is reduced.We only need to fit the data of fog and rainy days,and we can expect that the network can also have good generalization for other weather.Firstly,in view of the lack of data on dust weather and snow day at present,this paper uses clustering method to prove the similarity between dust weather and fog day,snow day and rainy day,and makes experimental proof for the rationalityof constructing universal sever weather image restoration algorithm through the generalization of neural network.To be specific,the biggest problem in constructing a universal sever weather image restoration algorithm is that there are not enough labeled images of sever weather.Although there are a large number of public data sets for haze weather and a certain amount of data for rainy days,there are almost no labeled data for snow days and dust days.Therefore,by clustering severe weather,this paper finds that dust weather has very similar features to fog weather,rainy day and snow day,and can be classified into one type of weather for treatment,thus reducing the amount of data to be labeled.We only need to fit the data of fog days and rainy days,and we can expect the network to have good generalization for other weather.Secondly,this paper constructs a neural network based on implicit code decoding,which improves the generalization ability of the neural network compared with the classical neural networks.When using the neural network fitting task,this paper first tries to use a variety of end-to-end networks to fit the data of fog and rainy days,including FCN,RES-NET and GAN.Although these networks have certain generalization ability,they are not as effective as traditional methods for the generalization ability of dust weather and snow days.Inspired by the traditional image processing de-noising algorithm,this paper combines the good generalization ability of the traditional image processing algorithm with the strong fitting ability of the neural network,and proposes a neural network structure based on implicit code decoding.In the implicit code decoding structure,the filtering kernel is learned by using neural network in the way of traditional image processing filtering,and the neural network encoder is used to encode the image into high-dimensional features,and the feature space is filtered.The filtered feature is then decoded into a restored image using a neural network decoder.In addition,the neural network will also use the pre-trained VGG network classifier as an additional supervision of the image,so as to constrain the network in the absence of real labels.Finally,experiments show that the generalization performance of this method based on implicit code decoding is significantly improved compared with the direct use of end-to-end neural network.Then,in order to further improve the generalization ability of neural network,this paper proposes a semi-supervised training method based on teacher and student network.This training method only needs to use a small number of target weather image pairs to train the image of specific weather.In this paper,a teacher network is constructed to compress the target image and obtain thehigh-dimensional feature,which can improve the recovery performance of the student network in the training process.The teacher network uses the performance of students after learning from the teacher as feedback,and updates the teacher network with this as the optimization goal.Afterwards,the student network can not only update the specific weather image pair with labels,but also update the specific weather image without labels,realizing the semi-supervised learning.The experiment proves that the learning method proposed in this paper can improve the network performance by using only a small amount of data sets on the specified weather.Finally,this paper combines the proposed network based on implicit code decoding with the semi-supervision method based on teacher and student network,and compares it with other best single weather recovery networks.The experimental results show that the proposed algorithm can restore the degraded images caused by different severe weather while maintaining a relatively competitive restoration effect.