Research On Deep Spiking Neural Network Model And Algorithm

Deep Spiking Neural Networks(DSNN)simulate the working mechanism of biological neural networks,possessing advantages such as efficiency,low power consumption,plasticity,and adaptability,and therefore have broad application prospects in fields such as pattern recognition,image processing,natural language processing,and intelligent control.DSNNs benefit from the latest research results in these two fields as a cross-cutting field of neural science and computer science.In the field of neural science,DSNN draws on the behavioral characteristics of biological neurons and neural networks,such as spike firing,time-domain coding,and sparsity,which help DSNN handle complex tasks more efficiently.In computer science,DSNN utilizes advanced computer hardware technology and optimization algorithms to process large-scale networks and complex tasks.However,due to the discrete nature of spiking neurons,DSNN cannot directly use the commonly used error Back Propagation(BP)algorithm in deep learning,which makes DSNN lack efficient training algorithms.At the same time,with the increase of network scale,DSNNs face problems common to large deep models today,lacking efficient training algorithms and difficulty deploying on resource-constrained platforms.To address the issues above,this dissertation focuses on three aspects of the DSNN model,training algorithm and lightweight strategy,from micro to macro,studies the neuron model,local network module,and overall network architecture in the DSNN model,and optimizes the activation function derivation and training process in DSNN algorithm.DSNN lightweight focuses on local module lightweight and network architecture lightweight.The main contents are as follows:1.A Resonate Spiking Neuron Model(RSNM)and a learning algorithm based on neural oscillation and phase locking are proposed.The discrete activation function of spiking neurons limits the direct application of the BP algorithm.The proposed model effectively solves the problems of non-differentiability,gradient explosion,and dead neurons caused by the discrete activation function.Subsequently,combined with the RSNM,two architectures of fully connected networks and Convolutional Neural Networks(CNNs)are proposed.Experimental results show that the proposed RSNM can directly apply the BP algorithm,and only one layer can handle nonlinear problems in practical applications,achieving excellent results.2.An Internal Spiking Neuron Model(ISNM)and a Gradual Surrogate Gradient(GSG)learning method are proposed.Due to the discrete nature of spike trains,DNNs cannot directly use mature operations in deep neural networks such as max pooling and batch normalization.At the same time,the surrogate gradient algorithm with a fixed surrogate gradient function is not conducive to the efficient convergence of DSNN.ISNM re-mathematically models traditional SNN to enable new models to integrate various operations in DNN seamlessly.In GSG,surrogate gradient functions gradually change during model training,ensuring that more spiking neurons participate in learning in the early training stage and achieve higher accuracy later.Experimental results show that the proposed ISNM and GSG methods can effectively improve the performance and training convergence speed of DSNN,enabling them to handle complex image classification tasks.3.A Spiking Neural Network with Working Memory(SNNWM)is proposed.DNNs only integrate information in one direction and have limited receptive fields in time dimensions.This dissertation introduces working memory mechanisms in DSNN and proposes a new network structure with multi-delay synapses.This dissertation also provides an effective method to reduce parameter quantities by increasing model generalization ability to reduce overfitting risks and improve model performance.Experimental results show that SNNs with working memory can effectively aggregate and correct spatial and temporal features,thereby improving their ability to process spatial and temporal data.4.A Light Spiking Vision Transformer(LSVi T)is proposed.Although SVi T models perform better than most CNN structures when trained with sufficient data,they still face issues with significant parameters,high computational complexity,and high training costs.This dissertation analyzes various parts of the SVi T to remove Multi-head Self-Attention in shallow transformer blocks and split the one in deep blocks to focus on high-frequency and low-frequency information in images respectively.In addition,this dissertation also introduces a new scheme for position encoding that can effectively expand information loss caused by light architectures.Experimental results show that the proposed method can effectively reduce model training and inference time while maintaining accuracy.