Research On Incremental Learning Method Based On Self-supervision And Channel Decouplin

Today,deep learning methods have achieved great success in various fields and application scenarios,including image recognition,image synthesis,and object detection.However,all these models face a major performance challenge when applied to previous tasks after continuously learning multiple tasks,that is,when the model adjusts its parameters to learn new tasks,the tasks learned in the previous data will become a cliff-like forgetting,this phenomenon is called catastrophic forgetting.Various incremental learning methods have been proposed to address this problem,aiming at alleviating the forgetting of previous tasks and income promising learning of new tasks such that all learned tasks can be completed at any given point in time.This paper focuses on the convolutional neural network pre-trained by selfsupervised learning,discovers the channel information in the neural network,and conducts research on the regularization method of incremental learning,aiming to propose a new method to improve the effect of incremental learning.The main contributions of this paper are divided into the following three parts:1.A self-supervised learning method is proposed to decouple the channels of convolutional neural networks.This paper adopts the gating mechanism,that is,adding a trainable weight parameter after the output channel corresponding to each convolution of selfsupervised learning.Through the method of self-supervised knowledge distillation,the parameters after different channels are sparsely separated,so that in subsequent tasks In,the importance of all channels in the neural network for different tasks is judged by the parameter weights after different channels.2.Achieve channel interpretability and subtask classification through self-supervised knowledge distillation.Through the gating mechanism,a set of control gate vectors can be trained for each picture.In this paper,firstly,the average value operation is performed on the control gate vectors of the same type in the subtask to determine the channel importance ranking in the category of the subtask,and the important channels are selected to form a subnetwork to realize the classification of the subtasks.On a small data set,by masking out about 1-10% of the parameters,the original accuracy can be improved on the binary classification sub-problems;Accuracy was improved by about 10%.In order to prove the effectiveness of this method,this paper uses semantic analysis methods such as feature dimensionality reduction,important channel analysis,and channel category similarity on the channel vector,which proves that the channel decoupling method can obtain certain semantic information at the channel level.This has a certain guiding effect on interpretability and subsequent downstream tasks.3.Add a gating mechanism to the existing incremental learning method based on regularization to realize the task increment of the self-supervised network.Through the limitation of parameters in different channels by the gating mechanism,the training process of the control gate is optimized to make the training process more efficient.Then on the basis of this method,the SE module is added and the gating mechanism is incorporated,and the control mechanism and the parameter importance of the SE module are used in the self-supervised learning,so as to be used in the incremental learning L2 regularization method,and the accuracy of the result is further improved.In summary,this paper devises a method for self-supervised learning of channel decoupling in networks.Through this algorithm,it can be used in subsequent work on subtask classification,channel interpretability,and regularization methods to optimize incremental learning.