Research On Echo State Network Of Multi-reservoirs

Echo state network,a novel recurrent neural network,which only needs to train the output weights,to overcome the existing problem of the traditional recurrent neural network,which include complex training algorithm,slow convergence speed,easily fall into the local minimum,and so on.In order to further improve the modeling ability of the echo state network,especially in the face of the approximation ability to realize tasks such as superposition(MSO)of multiple sinusoidal functions,this paper proposes two new multi-reservoirs structures.The first is to construct a new type of multi-reservoirs echo state network from leaky integral neurons,which is called Multi-reservoirs integral echo state network(MLESN).MLESN adopts the idea of combining top-down and bottom-up to build an echo state network.First,a top-down approach is used to construct the echo state network.It is assumed that its reservoir is composed of P-type distinct leaky integral neurons,and each type of neuron group will form a sub-reservoir.Then use a bottom-up method to build an echo state network to generate P different central neurons,which respectively represent P sub-reservoir.The neuron state of each sub-reservoir must be the same or similar to the state of the central neuron.The P central neurons A new virtual child reservoir is formed through random sparse connections.During the state update of the reservoir,it is still necessary to keep the difference between the neurons in the sub-reservoir small,and the big difference between the states of the different sub-reservoir neurons.The second is based on MLESN,which proposes a new multi-reservoirs echo state network,namely Multi-reservoirs sniffing echo state network(OFESN).OFESN generation idea is derived from the olfactory bulb of drosophila melanogaster,each sub-reservoir is equivalent to one olfactory bulb,and a new type of connection between sub-reservoir is given,That is,the connection between sub-reservoir is transformed into the connection between main neurons,and then the connection weight matrix of a new type of reservoir is constructed,which greatly simplifies the coupling connection betweenneurons.OFESN network can be equivalent to a small number of neurons in each actual sub-reservoir and a large number of sub-reservoirs.To obtain more number of neurons in reservoir and ensure the heterogeneity of neuron state,the prediction accuracy of echo state network is improved,and the calculation amount is greatly reduced.Finally,the Matlab simulation software is used to implement and compare with the leaky integrated echo state network(Leaky-ESN)prediction performance.The simulation results show that the two methods proposed in this paper have the characteristics of higher prediction accuracy and less volatility of prediction error.