Research On Communication Mechanism Of Large Scale Distributed Brain-like Computing System

Driven by massive annotation data and fast-growing computing power,artificial intelligence technology represented by deep learning has developed rapidly and has been applied in many fields.However,with high accuracy,deep learning has increasingly exposed many limitations,such as high computing cost and weak intelligence level.Brain-like computing based on Spiking Neural Network(SNN)has higher brain-like characteristics compared with traditional neural network.Therefore,Spiking Neural Network is the basic platform for the development of perceptual computing,and a new generation of artificial intelligence and the exploration of autonomous learning.Due to the huge scale of the brain,which has hundreds of millions of neurons and billions of synapses,the computing resources required for such a large-scale SNN simulation,which far exceeds the computing power of a single computing node or chip.Therefore,in order to simulate brain characteristics,study brain mechanism and explore Spiking Neural Network,researchers need to build large-scale simulation clusters to make up for the lack of single node computing performance.NEST(NEural Simulation Tool)is a widely used Spiking Neural Network simulator.It is very suitable for the research of brain-like computing system with large-scale distributed computing characteristics.Although the usual communication mechanism has been improved to a certain extent in NEST,which improves the computing efficiency,the communication efficiency is greatly reduced by the limitation of the improved collective communication.In response to these problems,the main work and innovations of this thesis have the following aspects:Firstly,the parameters of NEST simulator are quantified,and the communication optimization points of NEST simulator are obtained through analysis and experiment,which provides theoretical support for the communication research in the following paper.Secondly,according to the characteristics of NEST simulator,a neuron relocation algorithm(RELOC)based on SNN subgraph cross node optimization is proposed.We analyze the closeness of the connections between neurons in SNN model and get SNN subgraph cutting with lowest cost.Then we perform the neuron relocation operation before simulation.And the number of neurons on each node are controlled to make the load more balanced.Thirdly,the communication mechanism of NEST is improved,and sparse exchange is proposed.Node to node communication is used to replace the original collective communication.Only processes that need spike exchange can communicate,so as to increase the communication efficiency.By running different applications on the distributed NEST platform,the experimental results show that the average sparsity of Re LOC algorithm is improved by 20%,and the communication data is reduced by 73 times with sparse exchange compared with circular allocation mechanism used by NEST.