| Convolutional neural networks have become one of the indispensable algorithms in artificial intelligence applications.In recent years,supported by the massive data brought by the Internet and advance of the computing power,deep learning technology has developed rapidly,and the scale of convolutional neural network models has also become larger and larger.The expanded network scale on the one hand allows the convolutional neural network to handle more complex tasks,on the other hand,it also brings huge computation and storage consumption.As convolutional neural network algorithms enter devices such as smartphones,self-driving cars,and the Internet of Things,the problem of mismatch between the computing and storage requirements of network models and the capabilities of hardware devices has become increasingly prominent.Under this background,this paper focuses on the compression and acceleration problem of convolutional neural networks and proposes a model compression method based on global filter pruning.The main work and contributions of this paper are as follows:On the one hand,to get the global ranking of the importance of the filter,we propose an algorithm called Gate Decorator(GD).By introducing trainable parameters and combining with Taylor expansion formula,the GD algorithm can provide a more accurate ranking estimate than other methods.On the other hand,to solve the contradiction between accuracy and speed in the iterative pruning process,we propose a pruning framework called Tick-Tock.The Gate Decorator algorithm and the Tick-Tock pruning framework constitute a complete filter pruning method.We verified the effectiveness of our method on various data sets.Experiments show that our proposed method exceeds the current state-of-the-art filter pruning algorithms.For example,we reduced the computational cost of Res Net-56 network by 70% without causing a significant loss of accuracy.On a larger dataset such as Image Net,we pruned the Res Net-50 network,which improved the Top-1 accuracy by 0.31% compared to the baseline model,while the FLOPS was reduced by 40%.Based on our proposed pruning method,we constructed a real-time facial landmark detection system which is deployed on mobile phone.We reduced the FLOPs of the landmark detection model to one-fifth of its baseline model,enabling the system to run at about 15 frames per second on the ARM instruction set CPU.Aiming at the problem that overparameterized models are easy to overfit,we also propose a regularization method called adversarial noise.Experiments show that adversarial noise can bring a 10% relative improvement to the over-parameterized model. |
