Research On Method Of Directly Training High Performance Spiking Neural Network

The spiking neural network is considered to be one of the basic architectures of the next-generation neural network.It uses the spike form of 0 and 1 to transmit signals between layers.It has theoretically higher computational efficiency and is more suitable for neuromorphic chips.But precisely because of the form of the spike signal,there is an inherent non-differentiable problem in the spike neural network,which makes the traditional backpropagation(BP)algorithm unfeasible.At present,the method of directly training high-performance spiking neural network from scratch mainly uses surrogate gradient(SG).The surrogate gradient is an approximation to the Dirac function,thus providing a relatively smooth gradient in the backpropagation process,which can be sure degree to help alleviate the non-differentiable problem.However,most works keep a fixed surrogate gradient for all layers,ignorant of the fact that there exists a trade-off between the effective domain of gradients and the approximation to the delta function under the given dataset.Using fixed surrogate gradients may hurt the overall performance of the model.To guide the adaptive optimization of surrogate gradients in spiking neural networks,thesis proposes the indicator ,which represents the proportion of parameters with non-zero gradients in backpropagation.In addition,thesis proposes an algorithm CPNG based on the index for the trade-off between the shape of the surrogate gradient and its effective region,and conducts a series of ablation experiments to verify it.It is worth mentioning that the algorithm proposed in thesis requires only a very low overhead and can be easily integrated into the existing training process.The main contributions of thesis are as follows:(1)Thesis investigates the effect of the shape of the surrogate gradient on the training results of spiking neural networks.(2)Thesis proposes a statistical index to represent the effective area of the surrogate gradient,and proposes an algorithm CPNG based on ,which adaptively adjusts the shape of the surrogate gradient during the training process and guides the surrogate gradient to function approximation,and keep the spiking neural network can be trained normally.(3)Thesis proposes a CPNG algorithm to improve the classification accuracy of spiking neural networks on static image datasets(CIFAR10,CIFAR100,Image Net)and eventbased image datasets(CIFAR10-DVS).The algorithm proposed in thesis uses the SEWRes Net34 model to achieve an accuracy rate of 68.93% on the Image Net dataset.(4)The method proposed in thesis is still effective when the time step is 1,which may inspire the training of binary networks.