Research On Lightweight Convolutional Neural Network Based On Filter Pruning

In recent years,convolutional neural networks have shown significant advantages in the field of computer vision,including image recognition,object detection,and semantic segmentation.However,while the performance of convolutional neural networks is improving,the accompanying drawbacks of high storage and high power consumption make it difficult to deploy in resource-limited applications,especially mobile robots,drones,and smart phones.For this reason,constructing a lightweight convolutional neural network model to improve the inference speed of the model has great theoretical significance and application value.Aiming at the problem of a large number of redundant parameters in convolutional neural networks,this paper starts from the task of making neural networks lightweight,and conducts in-depth research on the parameter pruning method of convolutional neural networks.The specific research contents and innovations include:1)This paper proposes a filter pruning method based on sparse constraints.Aiming at the disadvantage of poor adaptive ability caused by the current filter pruning and layer-by-layer fixed ratio reduction,it starts from the inherent properties of the convolutional neural network,making the scale factor of batch normalization layer be the theoretical basis whether the feature map and its filter is important.Thus,the optimal sub-network is designed through the correlation between layers.In addition,in order to prevent a certain layer from being completely pruned when the pruning rate is too large,a global-local threshold mechanism has been added.Finally,without increasing the amount of parameters in the model inference stage,an over-parameterized convolution method is used to improve the accuracy of the pruned model.Experimental results show that the method achieves the best balance between performance and compression ratio on multiple public image classification data sets.For example,when 80% of the channels of the original VGG-16 are trimmed,a 0.13% increase in accuracy is still obtained.2)This paper proposes a filter pruning method based on the saliency of feature maps.In view of the low training efficiency of the current filtering pruning method,from the perspective of efficient evaluation of the importance of the filter,the network model can be pruned with only a small amount of image data.In order to improve the accuracy of the model after being pruned,the over-parameterized convolution method is employed to improve the accuracy of the pruning model.The experimental results show that the method proposed in this paper is superior to the current state-of-the-art pruning method in terms of FLOPs,parameter saving and accuracy compromise.For example,on CIFAR-10,when the amount of parameters decreased by 83.3% and FLOPs decreased by 66.7%,VGG-16 only lost 0.39% in Top-1 accuracy.3)This paper proposes a filter pruning method based on dynamic channel selection.In view of the cumbersome process of the current filter pruning algorithm,a novel network training mechanism,called dynamic channel selection,is adopted to realize model pruning during the training process.First,select a group of channels to participate in the forward propagation process of the training phase according to the channel significance index(CSI).Secondly,the CSI of the selected channel will be updated synchronously during the back propagation process to achieve dynamic channel selection.Finally,when the training is over,the optimal sub-network structure is designed by reserving channels with higher CSI,and at the same time,the over-parameterized convolution is used to improve the accuracy of the pruned model.This paper uses the classic convolutional neural network to evaluate the algorithm on multiple representative image data sets.The experimental results show the superiority and robustness of the method.