| Deep neural networks have become a hot research area in industry and academia because of many advantages such as automatic learning of target depth features and strong model generalization ability.Neural networks have the problem of computational resource constraint in time and space complexity,which limits the application extension of advanced neural network models under limited computational resource conditions.In order to streamline network models for applications on resource-constrained devices,a sparse compression training method and a model compression algorithm incorporating a dual-attention mechanism are used to carry out compression experiments and analysis.The specific research in this paper is as follows:(1)In response to the problem that some compression methods of neural networks fail to effectively reduce redundant parameters,resulting in expensive pre-training time costs,an improved network compression training method with sparse binary masks is proposed.First,a random mask is used to initialize the neural network,and the compression training process of the network is adaptively adjusted according to the sparsity and weight scaling coefficients set in advance by the network layers during the network compression training.The method introduces the same number of random weights by eliminating the redundant parameters of the network and traversing the weight parameters in the network layers to be pruned with the help of the network scaling weight ratio,which in turn improves the model training speed and efficiency and achieves effective compression and inference acceleration of the network model.Experimental results in CIFAR10 and CIFAR100 show that the algorithm accuracy of the method can reach 92.78% and 69.91% for Res Net-32 and VGG-19 architectures at 90% pruning rate,respectively;and 89.27% and 60.18% at a higher pruning rate of 98%.(2)For the problem that some neural networks cannot efficiently learn the sample information in the dataset during compression,which leads to poor prediction accuracy and generalization ability of the network,an improved channel space attention fusion network spatial loss compression method is proposed.Combining the channel attention and spatial attention mechanisms in the module for network compression training,on the one hand,the network does not need to be pre-trained when compressing,and at the same time,the network compression process is optimized as a spatial loss minimization constraint problem,using the distribution of weights in the network to dynamically measure the spatial loss,and eliminating redundant parameters by continuously iteratively updating the network weights,aiming to obtain a more streamlined network model.Experiments of the model compression algorithm are carried out on the public datasets CIFAR10 and CIFAR100,and the results show that the method can effectively refine the key information of data samples after a higher degree of compression on the CIFAR10 and CIFAR100 datasets,and the accuracy of the algorithm in this paper is better than that of similar classical methods,which provides an idea for fusing attention mechanisms to carry out network compression.In this paper,we start from the problems of existing methods and conduct research on deep neural network model compression,which can better understand the internal structure and characteristics of deep neural networks and deploy excellent streamlined network models in portable devices to realize applications such as image recognition and smart home management to enhance user interaction experience. |
