Relationship Between Correlation Of The Convolution Kernels And The CNN Architecture Optimization

Deep learning is one of the most popular research topics in machine learning,it was proposed by Hinton et al.in 2006.As an excellent machine learning model,it has been favored by researchers since it was introduced and has aroused the interest of many researchers.Convolutional neural network,as an important network model in the field of deep learning classification,has also attracted the interest of many researchers.It is a feedforward neural network and a special form of deep neural network.Experimental data shows that convolutional neural networks are a very effective network model for image processing,video recognition and classification.However,the performance of CNN is greatly affected by the number of convolution kernels.Too many convolution kernels in the network structure will result in too many training parameters,which will increase the time cost of network training,greatly consume computer resources and easily lead to over-fitting.Too few convolution kernels will result in insufficient data training and affect the final result.Therefore,it is very important to choose the optimal convolutional network structure.How to quickly determine the optimal number of convolution kernels is still an urgent problem to be solved.Based on the proposed problem,to optimize the number of convolution kernels in the convolutional neural network,the analysis method based on the weight correction range and the correlation analysis method based on the weights in the convolution kernels is used in this paper.Experiments show that the analysis method based on the weight correction range has great instability.On the one hand,it is affected by different convolutional layers in the convolutional neural network,because each convolution layer has different effects on network performance.On the other hand,it is affected by the error backpropagation,because of many times of error backpropagation during the training process.The weight of convolution kernel will be corrected in each backpropagation and it can not be determined which backpropagation is the most informative one in the training process.The correlation analysis method between weight of the convolution kernels is more valuable.Through experiments,it is found that the weight correlation threshold is only affected by the kernel size but not affected by different kinds of datasets.There is indeed a correlation between the correlation threshold and the number of convolution kernels.Experimental results imply that the weight correlation of the convolution kernels is an important indicator for optimizing the architecture of CNN.