Regularization Of Neural Network And Its Application In Geological Prediction

We mainly study the regularization of the spiking neural networks(SNNs)in this thesis.SNNS is called the third generation neural network.It simulates the biological neuron model to transmit information in the form of sending out spikes.Spike sequence enhances the ability of processing large data in time and space.In theory,SNNs are more powerful than the sec-and generation neural networks,but it is not widely used in engineering problems.There are two main reasons:Firstly,SNNs can not be effectively trained.Secondly,the simulation of the SNNs are computationally intensive.In order to construct some efficient and low-energy struc-tures and algorithms of SNNs,this thesis first proposes SpikeProp algorithm with smooth L1/2 regularization term,which can eliminate redundant or unnecessary connection weights in SNNs and regularize the network,so as to improve the generalization and convergence speed of SNNs while considering the accuracy.Secondly,in order to further study the regularization of SNNs,five Drop-SpikeProp algorithms are proposed.Finally,for the large amount,multi-source,het-erogeneous and complex data generated in the process of tunnel excavation,the research on the analysis and processing of the big data of tunnel construction is carried out,and the geological prediction model of artificial neural network(ANN)is constructed according to the physical and mechanical indexes of geological type,which lays a foundation for the realization of intelligent tunnel construction and engineering application of deep SNNs.The main work of this thesis includes:1.For the overfitting and slow convergence speed of SNNs,the SpikeProp algorithm with smoothing L1/2 regular term is proposed.In the process of network training,the smoothing L1/2 regular term is introduced into the error function to make some weights very small.Finally,these weights can be removed from the network,so as to reduce the network complexity and improve the convergence speed and generalization performance of the network.In addition,the convergence of the algorithm is proved under some conditions.The experimental results show that the SpikeProp algorithm with smoothing L1/2 regularization can not only prune the network,but also improve the convergence speed and generalization of SNNs under the condition of ensuring the appropriate classification accuracy.In addition,compared with L1 and L1/2 regularization,smoothing L1/2 regularization can achieve higher sparsity effect.2.For the overfitting of SNNs,because the synaptic connection of biological neural network is a random mechanism in the formation of human brain neural network.Two Drop-SpikeProp algorithms(SPDO and SPDC)with fixed random drop probability are proposed.On this basis,two kinds of Drop-SpikeProp algorithm(SPADO and SPADC)with adaptive drop probability and one kind of Drop-SpikeProp algorithm(SPGAD)with population adaptive drop probability are proposed.All these five algorithms effectively improve the generalization of SNNs and lay a foundation for the research of deep SNNs.In addition,the convergence of SPDC is proved under some conditions.The experimental results show that the adaptive algorithms(SPADO,SPADC and SPGAD)are better than the non-adaptive algorithms(SpikeProp,SPDO and SPDC)on generalization performance,and the SPGAD algorithm is best.In addition,the convergence speed of three adaptive Drop-SpikeProp algorithms is faster than that of non-adaptive algorithms.3.For the difficulties of engineering data analysis and prediction caused by the high com-plexity and uncertainty of the construction environment in the process of tunnel excavation,a set of solutions suitable for the data analy sis and processing of the tunnel boring machine(TBM)in large-scale complex environment is proposed,and a framework for real-time interpretation of the operation data of the TBM is constructed.Then,according to the physical and mechanical indexes of geological types,the ANN prediction model is constructed to realize the intelligent geological prediction,which provides conditions for the model optimization of the SNNs and opens up a new field of the application of the SNNs in engineering.The experimental results show that:the feature augmentation in the process of data processing does improve the predic-tion accuracy;the performance of the ANN model constructed according to the geophysical and mechanical indexes is superior in the test set,and the prediction accuracy of the ANN model constructed is higher than many widely used learning models,such as support vector regression,random forest and XGBoost.In this thesis,for the problems of overfitting and slow convergence speed in the training process of SNNs,we first introduce the smoothing L1/2 regular term into the error function un-der SpikeProp algorithm,and propose SpikeProp algorithm with smoothing L1/2 regular term to realize the sparse structure of SNNs,so as to improve the generalization ability and conver-gence speed of SNNs.Secondly,based on two kinds of drop randomization technology,five Drop-SpikeProp algorithms are proposed to further regularize SNNs.In addition,through the integration method,k-nearest neighbor algorithm and difference method to analyze and process the complex big data of the project,a real-time interpretation framework of the TBM operation data is constructed to realize the intelligent geological prediction,which lays a foundation for the research and engineering application of the deep SNNs.