Research And Application On Compressing Convolutional Neural Networks Via Filter Pruning

Recently,deep learning technology has developed rapidly.Deep learning networks represented by Convolutional Neural Network(CNN)have been widely used in various fields.However,these networks usually company with heavy computation burdens due to huge numbers of parameters and their high memory usage.Therefore they are difficult to apply directly to hardware platforms with limited computing resources.In order to further improve the practicality of deep learning networks,researchers have tried to compact deep learning networks for embedded platforms.Among them,filter pruning methods aim to evaluate and remove the entire redundant filters,which make the pruned compact network with the hardware platforms,are widely used to accelerate the inference process of CNN.It is worth mentioning that the soft filter pruning(SFP)method has recently gained increasing interest due to the weight value of the pruned filter is directly set to 0,which can avoid damaging the network structure.To improve the performance of CNN,this paper studies the network compression method based on SFP method.The main works of this paper are as follows:(1)A soft and smooth filter pruning method is proposed.With using the gradual decay of the weight values strategy in the training epoch,the method can force the weight values of filters which have small L2-norm value to gradually decay to zero,which can give the network more opportunities to adapt to changes,thereby retaining the pre-training information of pruned filters.The method is different from the traditional soft filter pruning method that directly resets the weight values of pruned filters to zero and leads to large change in network parameters,thereby affecting the inheritance of prior knowledge of the pre-training network and causing the loss of information.Experimental results suggest that with different depths of network and datasets,our method achieved better performance.In particular,when the pruning rate is 60%,accuracy of Res Net-20 on KSDD by the soft and smooth filter pruning method improves by 1.92% compared with SFP,achieving better model compression effect.(2)A hybriding pruning method that combined with cluster and the soft and smooth filter pruning method is proposed.The soft and smooth filter pruning method only prunes the filters which have small L2-norm value in the network.But the similar filters are not pruned,which could affect the performance of the CNN.Based on the soft and smooth filter pruning method,we further remove redundant filters which are similar as other filters by using the cluster method.In particular,the hybriding pruning method reduces the FLOPs of Res Net-56 by58.2% on the CIFAR-10 dataset.And compared with soft and smooth filter pruning method,the accuracy improves by 0.16%.Experimental results suggest that the hybriding pruning method is more advantageous than the soft and smooth filter pruning method in improving model performance.(3)An application system for detecting defects based on the pruning acceleration algorithm is designed and implemented.Combined with the hybriding pruning method proposed in this paper,the system for detecting metal surface defects is implemented by using the Py Qt5 toolkit.The system that mainly includes functions such as data preprocessing,model training and pruning,and defect detection,realizes rapid detection and recognition through the compression model obtained after training.Experimental results suggest that the compressed model trained by our method consumes less detection time than the original uncompressed model on the premise of realizing the detection function,which reflects the feasibility and application value of our method in industry.