The Structural Design And Application Of LSTM Neural Network Based On Data Driving

Long short-term memory（LSTM）neural networks can simulate the memory characteristics of the human brain,and solve complex problems by building models through data-driven method.However,it is difficult to determine the hidden layer size and internal structure of network that matches with the specific tasks.It makes cost of computing and hardware deployment are improved,and the generalization ability of model is weakened.In addition,LSTM neural networks are established by the complete data.But the data missing introduces the problems with uneven network inputs and reduced sample sizes,which influences network performance and limits the application range.Therefore,the design of network structures and algorithms which match the actual task data is important,which can develop the theory of LSTM neural networks and promoting their practical applications.The main research contents and highlight of innovation of this thesis are as follows:（1）The design of hidden layer size of LSTM based on HCPSO algorithmTo solve the problem that the scale of hidden layers of LSTM neural network is difficult to be determined based on actual task data,the hybrid coding particle swarm optimization（HCPSO）algorithm is proposed.Firstly,the discrete-continuous hybrid coding mechanism is proposed,which realizes the mapping of particle coding to the scale and weights of the hidden layer of network;then,the hybrid update algorithm based on discrete update strategy（DUS）and adaptive non-linear moderate random search strategy（ANMRSS）is designed,which is used to adjust the size and weights of the network hidden layer,dynamically.The convergence as well as local and global search capabilities of HCPSO are guaranteed and balanced by the DUS and ANMRSS.The validity of proposed method is corroborated by the experiments of time series prediction and nonlinear dynamic systems identification.（2）The design of sparse LSTM based on HCPSO andl₁ regularizationFor the redundant structure and poor generalization ability of the dense LSTM neural network,the sparse LSTM with HCPSO and ₁l regularization（SLSTM-HCPSO-₁l）neural network is designed.Firstly,the weight structure and values are encoded as particles,and the relationship between the particles and the weight structure and values is established;secondly,the fitness function that contains the network error and regularization term is designed to guide the search direction of particles;finally,the DUS and ANMRSS of HCPSO are used to adjust the discrete and continuous coding of particles,respectively.The network structure is simplified and the network generalization ability is enhanced.Experimental results show that SLSTM-HCPSO-₁l has a sparse structure and good generalization ability.（3）TG-LSTM neural network design for incomplete time series forecastingFor the problem that the missing data in the time series cause uneven input and fewer samples in the LSTM neural network,the LSTM with time gate（TG-LSTM）neural network is proposed.Firstly,the TG-LSTM unit structure is proposed,which realizes network input filling and output prediction;then,according to the TG-LSTM unit structure,the alternate forward propagation mechanism is designed to solve the problem of uneven network input that is caused by data missing;finally,based on the network structure and forward propagation mechanism,the union learning algorithm is established,which is used to train the filling and prediction tasks,simultaneously.The network weights can be shared by the filling and prediction tasks,which matches the two tasks.Experiments on incomplete datasets show that TG-LSTM neural networks can not only fill in missing input data,but also improve the accuracy of prediction.（4）The online evaluation of surface water environmental water quality based on TG-LSTMFor the problem that the lack of water quality index and it is difficult to evaluate the water environment conditions online,relying on the sub-project of"The Plan Water Pollution Control and Governance Major Science and Technology Project"in China,"Development and Research of Big Data Platform for Water Environment Management in Beijing-Tianjin-Hebei Region",the online evaluation method based on TG-LSTM neural network is designed,which realizes the evaluation of water quality categories and water body nutrient status.Firstly,the TG-LSTM neural network is driven by the collected water quality index data to establish the model and fill the missing data;secondly,the TG-LSTM neural network is driven by the collected and filled water quality indicator data to predict the water quality indices;then,according to the water quality indicator data,the water quality categories and nutritional status are evaluated with reference to the related national standards,and the availability of the proposed method is illustrated by experiments in the real water environment.finally,the"Online Evaluation System for Surface Water Environmental Water Quality in the Beijing-Tianjin-Hebei Region"is developed,and the evaluation algorithm and interactive interface are developed using Python and JAVA respectively,which realizes algorithm integration and data visualization.