Research On Object Tracking Technology Based On Deep Spiking Neural Network

Spiking neural networks(SNNs)are fundamentally different from the neural networks that the machine learning community knows.In SNNs the information is represented and propagated using spikes,which are discrete events that take place at points in time,rather than continuous values.Spiking neural networks have attracted widespread interest as the thirdgeneration of neural networks due to their event-driven and low-powered nature.SNNs,however,are difficult to train,mainly owing to their complex dynamics of neurons and non-differentiable spike operations.Furthermore,their applications have been limited to relatively simple tasks such as image classification.In this study,we investigate the performance of SNNs in a more challenging computer version task(i.e.,object tracking).The main contents are summarized as follows:1.The self-attention mechanism is introduced into Generative Adversarial Networks(GANs),and the self-attention adversarial network is applied to data enhancement,and the real-time generation of fake data is used to adjust the convergence of the model during the training process of the spiking neural network.2.Starting from the end-to-end training method of SNN,this thesis considers detailed simulation verification and comparative analysis from the aspects of spiking neuron modeling,image spiking coding,encoder self-circulation,and the relationship between spiking neurons and activation functions.Using surrogate gradient methods,referring to the VGG network,a deep convolutional SNN called Spiking VGG was built to identify the structural damage of the concrete bridge surface,which proved the feasibility of using the gradient surrogate methods to directly train the SNN.The experimental results show that: in the case of using convolutional encoding and convolutional self-feedback encoder,the Spiking VGG has achieved an evaluation accuracy of 98.44% on the bridge damage dataset,which is very close to the accuracy 98.67% achieved by the convolutional neural network.3.In addition,based on the Fully-Convolutional Siamese Network(Siam FC),a Spiking Siam FC is designed for realizing object tracking,which achieves a precision score of 0.7111 and a success score of 0.5182 on the OTB100 dataset,which is only 6.3% lower than that of the conventional Siam FC.Spiking Siam FC can not only achieve high-precision target tracking,but also have obvious advantages such as fast inference speed,high computing efficiency,and low power consumption,which provides the feasibility for model porting to edge devices such as artificial intelligence chips or neuromorphic hardware.The research in this thesis also provides a reference for the future application of SNNs in more computer vision fields,which makes SNN more practical valuable.