Bio-inspired Hierarchical Spiking Neural Network And Its Applications

With the development of cell biology and neurological technology,people have learn more about the structure and function of biological visual system.Biological neurons use spike sig-nal to respond to changes of environment.These visual signal processing pathway begins at the retina,and signal passes through the ganglion/synovial cell layer to the primary visual cor-tex V1 layer.The simple feature information is passed to the V2/V4 middle layer and finally reaches the inferior temporal(IT)layer.In addition to the structure,biological neuroscience research has also found that Spike-timing-dependent plasticity(STDP)learning rules can be used to change synaptic plasticity between neurons.Moreover,lateral suppression mechanisms and homeostasis have an important significance for the healthy and stable development of the spiking neural network.On the basis of the neuron model revealed by biological neurology,this paper designs a spiking neuron model which is suitable for computer simulation.Based on the hierarchical network structure in biological visual system,a bio-inspired hierarchical spiking neural network is designed.And combined with biological neural mechanisms,such as receptive fields,STDP learning,lateral suppression,etc.,visual ventral pathways is simulated to extract shape information from images and images are classified based on these shape features.With the simulation of biological visual system,we also propose a network training method to convert weights and biases based on convolution neural network,and design a deep spiking neural network for natural image classification.In addition,based on the reinforcement learning mechanism found in biology with dopaminergic neurons,a spiking neural network is designed that combines reinforcement learning with plasticity learning and the feasibility of this learning method is validated by XOR benchmark problem.Based on the neural mechanism of biological visual system,firstly,we summarizes the ba-sis of neuro-mechanism related to biological vision.On the basis of this research,a bio-inspired hierarchical spiking neural network is proposed.The structure of this hierarchical spiking neural network contains seven layers:signal encoding layer,edge extraction layer,V1 simple cell layer,V1 complex cell layer,V2 simple cell layer,V2 complex cell layer and SVM classifier layer.Combined with STDP learning rules,homeostasis,lateral inhibition and other neural mecha-nisms,shape features of input images are extracted and then used for image classification.In this paper,the spike responses of the edge extraction layer of the network are visualized and show that the neurons in this layer can extract the edge information from natural images.Orien-tation selectivity experiments on V1 simple cell layer verify that the neurons in this layer have been trained and obtained orientation selectivity.Finally we test this hierarchical spiking neural network with MNIST handwritten character dataset.With the unsupervised learning method,this network successfully complete classification task and it’s classification accuracy can reach to 96.3%.The proposed hierarchical spiking neural network needs to extract more and more com-plex image features when dealing with more complex input images.However this hierarchical spiking neural network can not effectively solve this problem with unsupervised STDP learning method,because it is hard to train deep network with this learning rule.Therefore,the net-work parameters(weights and bias)come from a convolution neural network(CNN).We train a CNN,convert and utilize the parameters in a deep spiking neural network(SNN)as the param-eters in CNN with the similar structure,which makes the deep SNN be capable of classifying images.Because the CNN is composed of analog neurons,there will be some transfer losses in the process of conversion.Some reasonable optimization strategies are proposed to reduce the losses while retain a higher accuracy.The deep spiking neural network proposed in this paper is closer to the biological mechanism in the design of neurons and our work is helpful for understanding the spike activity of the brain.The proposed deep SNN is evaluated on CIFAR and MNIST benchmarks and the experimental results have shown that the proposed deep SNN outperforms the state-of-the-art spiking network models.The accuracy rate of classification on CIFAR dataset was 86.43%,and the classification accuracy on MNIST dataset was 99.09%.Although the above-mentioned deep spiking neural network can achieve good results in the classification of natural images,since the synaptic weights are transformed from a convolutional neural network with similar network structure,transformation errors will inevitably occur.Even if the conversion error is very small,the final classification accuracy can only be infinitely ap-proximated.So for bio-inspired spiking neural networks,a network training method that is more consistent with biological mechanisms should be developed.In the study of biological nerves,dopamine neurons are found to be associated with reinforcement learning mechanisms.In com-bination with this neural mechanism,a suitable but simple learning algorithm is proposed in this paper.Reinforcement learning rules are implemented by adjusting synaptic model parameters,and synaptic weights are adjusted by plasticity learning method.This spiking neural network is used to solve XOR benchmark problem.In the training process,the synaptic model param-eters are tracked and analyzed.This framework of spiking neural networks,which combines reinforcement learning and plasticity learning,is a new applications for the processing of neural signals and computations using biological synaptic models.