Compression Of Convolutional Neural Networks Based On Groups Of Feature Templates

Convolutional neural networks have received increasing attention in academia and industry,achieving performance far beyond traditional methods on many machine learning tasks,such as image classification,object detection,and generative models.However,as the model performance improves,the increasing computing demand limits the deployment of the model to a large number of edge scenarios.Meanwhile,applying complex-dataset-oriented models to smaller datasets would increase network model redundancy and enhance the risk of overfitting,which will degenerate model generalization performance.As a conventional method of model compression,network pruning can remove the redundant structures in the model.However,there are still problems such as complicated iterative tuning of parameters and insignificant saliency gap between different substructures.To solve these problems,the pruning algorithm in network compression is improved and studied in this paper,and the main work and contributions conducted for this purpose are as follows:(1)A convolutional neural network compression method based on feature template sets is proposed,which performs L0 regularization to solve the problem that conventional L1 and L2 regularization cannot provide a clear pruning saliency gap.Where,the Half-Quadratic Splitting(HQS)method is applied to get the approximate solution of the L0 regularization,which enables the use of many gradient-based approaches to solving this NP-hard problem approximately.Experiments show that the proposed method can significantly expand the saliency gap between network structures,improve the confidence of pruning threshold selection,and help reduce the workload of threshold selection at each layer,compared with the conventional L1 and L2 regularization methods.(2)A parameter selection algorithm is proposed to redefine the hyperparameters in the original HQS based method.Compared with the original hyperparameter set,the redefined parameter set has obvious physical significance,and also reduces the difficulty of tuning the algorithm on different models and datasets.Experiments show that most of the parameters can be used without major changes to achieve good results on different data and models.(3)The effectiveness of proposed method in specific engineering applications is verified by applying it to the aerial powerline semantic segmentation model,and the common workflow of model compression is also be shown,which including baseline model selection,data enhancement,loss function optimization and model compression.In practical applications,the proposed method shows a better compression effect.