Study On Design And Application Of Lightweight Spiking Neural Networks Based On Tensor Decomposition

Spiking Neural Networks(SNNs)are the third generation of neural networks inspired by the structure and function of the human brain,and commonly used in the field of artificial intelligence for processing spatial-temporal information and promoting energy-efficient computing.However,the increasing complexity of deep learning problems has resulted in a further increase in the number of parameters in SNNs,which leads to enormous memory and time costs when running on hardware.Lightweight techniques are required to compress SNNs and improve their efficiency.Therefore,we used tensor decomposition theory to construct tensorized SNNs.To verify the compression performance of tensorized SNNs,experiments were conducted on three types of datasets.The following results are obtained:Firstly,we summarized the decomposition patterns of tensorization method in convolutional and fully connected layers of SNNs by applying four basic tensor decomposition methods(Tensor Train,CANDECOMP/PARAFAC,Tucker,and Tensor Ring(TR)).Specifically,each convolutional layer and each fully connected layer are decomposed into multiple convolutional layers and tensor contractions.Based on theoretical analysis,the TR decomposition is relatively superior.In addition,we proved that tensorized SNNs can achieve higher memory utilization and computational efficiency than the original SNNs.Secondly,we found that four types of tensorized SNNs can achieve high parameter and computation compression while maintaining accuracy by conducting experiments on a typical dataset.Moreover,the TR decomposition can achieve higher accuracy with the same parameter compression rate and computation for the convolutional layers.For the fully connected layers,the TR decomposition also requires fewer parameters and lower network parameter compression rate at the same accuracy loss,which is consistent with theoretical analysis.Thirdly,tensorized SNNs were experimented on three types of datasets,including static visual,dynamic visual,and speech datasets.We found that tensorized SNNs can maintain an accuracy loss of less than 5% while achieving the network compression of50-100 times and the computation compression of 5-10 times.These results demonstrate that tensorized SNNs have significant lightweighting effects and high generalization capabilities.Furthermore,compared to existing algorithms,tensorized SNNs can further reduce network compression rate while ensuring an accuracy loss within 5%-8%,and achieve lower accuracy loss at the same compression rate.Our results suggest that tensor decomposition is highly effective for lightweighting SNNs with strong generalization ability in classification tasks.Tensor decomposition also exhibits stronger performance advantages compared to existing methods,providing a new solution for lightweighting SNNs.Our research combines tensor decomposition and SNNs and summarizes the relevant patterns of tensorized SNNs,offering support and reference for the application and performance testing of new tensor decomposition methods.