Convolutional Neural Network Pruning Based On Clustering By Density Peaks

The success of convolutional neural networks in the field of image is due to the increase in the number of layers and neurons.However,this makes the deep convolutional neural networks limited by the cost of model calculation when deployed on hardware devices with limited resources.In order to solve this problem,this paper carried out research on the lightweight problem of deep convolutional neural network,and the main research work is as follows:The traditional magnitude-based weight pruning method only focuses on the weight value of the convolutional kernel,without considering the correlation between the convolutional kernels.In the case of large scale pruning,it is easy to destroy the spatial structure of the convolution kernel,resulting in a sharp decline in the model accuracy after pruning.Considering that each convolution kernel in the convolutional neural network model presents a certain category when extracting image features,similar convolution kernels will extract features from similar angles,which constitutes information redundancy.In this paper,a pruning algorithm based on density peak clustering was proposed.Based on density peak clustering,the algorithm uses the weight parameters of convolution kernels to extract the similarity information among the convolution kernels as much as possible,and classifies the convolution kernels with similar weight structure in each convolution layer into the same cluster.Then,in each cluster,Average Percentage of Zeros(APo Z)was used to reverse order the importance of the convolutional kernels from the perspective of activation value,and the redundant convolutional kernels with low importance were removed.Compared with the traditional magnitude-based weight pruning method,the proposed method can select the redundant convolution kernel more pertinently,improve the compression ratio as much as possible and avoid the excessive pruning of some clusters,so as to ensure the diversity of feature extraction after pruning.Experiments show that the proposed method achieves a high compression ratio of model parameters while maintaining a small precision loss.