| Deep neural networks have been widely applied in several fields,but the large amount of parameters and FLOPs of DNNs prohibit the further development.Recently lots of research works focus on the compression and acceleration of DNNs,and have proposed several effective methods.But currently DNN compression and acceleration methods often depend on manually designing pruned model structures and single pruning algorithm,which is inflexible with different hardware or compression targets,also hard to maintain the original performance.Therefore,we propose that the process of DNN compression and acceleration should be more automatic,and search the pruned model architecture and pruning algorithm for each layer independently,thus save human resources and reduce the performance degradation.We propose to combine DNN compression and acceleration with AutoML,the details of our research works are described below.We propose an automatic pruning method based on fast hybrid search.For the two steps in structured pruning of the deep convolution neural networks:design pruned model architecture and pruning algorithm,we firstly propose to search the pruned model architecture and pruning algorithm simultaneously.Specifically,we search for the pruning ratio and pruning algorithm of each layer.Besides,to balance search efficiency and search accuracy,we propose to divide the search process into two phases.In the first phase we use evolution algorithm to explore the original huge search space,also with the efficient adaptive-BN to validate the candidates.Then in the second phase,we search in a small set based on the results of the first search phase,with the more accurate finetuning to validate the candidates.Our proposed method has less performance degradation compared with the previous methods,and we are the first to search for the pruning algorithm for each specific layer. |
