Study On Long Short-Term Memory Networks Based On Forgetting Memristor Synapses

Long short-term memory network(LSTM)is a kind of temporal recurrent neural network with long-term memory ability,which can learn long-term dependence,and has a good effect on the processing of sequence information.The research of LSTM network mainly includes the application research on software and the implementation on hardware circuit.The implementation on hardware circuit is mostly based on the implementation of traditional circuit,which has the limitations of high power consumption,complex circuit and slow processing speed.The memristor with high integration,low power consumption and high speed operation has become the preferred choice of simulated synapses,and has a broad application prospect in the hardware implementation of LSTM networks.At present,most memristive long short-term memory(MLSTM)networks are based on nonvolatile memristors and MLSTM networks have only one state.The forgetting memristor with long-term memory and short-term memory can represent two kinds of weights in the storage of neural network weights,which provides a new choice for the realization of memristor neural networks.In this thesis,the MLSTM network is designed by combining the forgetting memristor with LSTM network.In particular,a method of setting forgetting memristor resistance with controllable decay rate is proposed,and the conversion between short-term memory networks and long-term memory networks is realized by using forgetting memristor bridge synapses.The research content of this paper mainly includes the following two points:First,the forgetting memristor model with controllable decay rate is proposed,and a method of setting the resistance and decay rate of forgetting memristor based on current source is designed.Based on the memristor bridge synaptic circuit,the expression of the weight of the forgetting memristor bridge synaptic circuit is derived by circuit analysis and formula reasoning,which can completely store the positive,negative and zero weight,and the weight has the state of long-term memory and short-term memory.Furthermore,the method of setting the weight of forgetting memristor bridge written independently and the method of setting the weight of forgetting memristor bridge written in batches are proposed.The method of setting decay rate under two kinds of weight setting methods are designed.Based on the two kinds of forgetting memristor bridge,the similarities and differences of setting the weight and decay rate of forgetting memristor bridge are analyzed.Furthermore,the images with long-term memory and short-term memory are represented by the forgetting memristor bridge synaptic array,and the short-term memory images evolve to long-term memory images.The picture evolution effect under differernt decay methods are analyzed.Second,the LSTM network based on forgetting memristor bridge synaptic array is proposed,which has two states of long-term memory and short-term memory network,and the short-term memory network state will deacy to the long-term memory network state.Based on the forgetting memristor bridge synapses with controllable decay rate,three modes of short-term memory decaying to long-term memory network are proposed,and the simulation tests of data sets are carried out on the algorithm,and the accuracy of MNIST,KMNIST,and FASHION-MNIST are 98%,89.5% and 87.5%,respectively.The results show that the LSTM network based on forgetting memristor bridge synapses can realize the recognition between the two data sets,and the recognition accuracy is good.The reset signal is proposed during the conversion process of the short-term memory network state to the long-term memory network,which can maintain the recognition effect of the short-term memory network on the data set.