Dynamical Predictions With Machine Learning And Dynamics Of Machine Learning

In recent years,machine learning(ML)has become a research hotspot in many disciplines of natural science.It is widely used as an important method to extract and utilize information based on data and is changing all aspects from industrial production to scientific research.The main interest of physical science for ML lies in two folds,on the one hand,to use ML methods as effective tools for data processing and analysis,and on the other hand,to take ML model,a kind of complex dynamic system,as the object of research,and use theories and methods of statistical physics and nonlinear dynamics to study its mechanism,and provide a foundation for developing the theory of ML.Based on the above background,this dissertation explores the following several related issues and has made some progress.In the first part of this dissertation,we propose a new ML model and a new strategy for model-free prediction of the evolution of spatiotemporal dynamic systems.Prediction of the system’s evolution is an important field of the application of ML and it is generally realized by a recurrent neural network.As is shown by our analysis,it is equivalent to the time-delayed dynamic mapping of high-dimensional input and output.Numerical results show that this kind of learning machine fails to generalize to the spatial points that are not involved in training,and we discover that it is the "relative time scale",which is determined by different training sequences,that attribute to this failure.Further,based on the theory of time-delayed phase space reconstruction,we propose to use multiple time series sampled from various spatial points to train a one-dimensional time-delayed learning machine to perform prediction.It is found that this kind of learning machine has a good generalization ability to the spatial points that are not involved in training.The significance of this discovery is that we can further avoid the "curse of dimensionality" faced by traditional strategies,which need to evolve simultaneously all the possible spatial dimensions of the system even those dimensions that we focus on are just a small part.This provides a new strategy for the prediction of spatiotemporal systems,that is,to achieve a fine-grained prediction from a learning machine trained on coarse-grained data.This strategy is verified in two typical models,the Kuramoto oscillators model and the Barrio-Varea-Aragon-Maini model.In the second part,we propose a new framework and method for analyzing the neural network learning machine.It is called the weight-pathway-analysis(WPA).Though ML has made great achievements in various application fields,an important problem encountered by the research of ML is that the understanding of the mechanism of machine’s "learning" is still lacking.Many technical and structural designs on experiences have made lots of improvements to the performance of ML,while the principle behind it has not been clearly understood.This is why we call the learning machine the "black box".The WPA realizes the vertical perspective of the internal structure and function of the learning machine,making the "black box" transparent.Based on this,the linear and nonlinear learning modes are found,and it is proved that the former and the latter are used to extract the linearly separable and inseparable features,respectively.Further,we describe the dynamics of the training process,explain how the under-and over-learning phenomena occur,and understand from one aspect part of the mechanism of the optimized performance brought by increasing the network’s width and depth,so as to deepen our understanding of what to learn,how to learn,and how to learn better.In the third part,we attempt to design,based on unsupervised or partially supervised,a new strategy for singular data mining,which is one of the important applications of ML.The mislabeled data mixed in the dataset,strange data generated from a new mechanism,or data that is seriously distorted can all be regarded as singular.Traditional methods of singularity mining have,intentionally or unintentionally,defined"singularity" in advance.A better strategy,however;should remove this constraint since the singularity is inherently undefined.We attempt to design a "wolves strategy"to realize the singular data mining without defining the singularity(unsupervised).To improve the mining efficiency,combined with the results of the study of the learning mechanism,we further introduce a "guidance"(semi-supervised)to the "wolves strategy".The effectiveness of this strategy is shown by giving out singular pictures in dataset MNIST,fashion-MNIST,and Cifar-10,and cases that are likely to be misdiagnosed in the medical dataset Diabetic Retinopathy Debrecen and Mammographic Mass.