Two Neural Networks Methods For Numerical Solutions Of Partial Differential Equations

In recent years,the using of deep learning technology based on neural networks to solve partial differential equations(PDEs)has been a hot topic.This thesis first introduces neural networks and draws a point that the core of neural networks is back propagation algorithm whose core is gradient descent method,while the core of gradient descent method is automatic differentiation,so the key to realize neural networks is to realize automatic differentiation.Secondly,this thesis introduces the commonly used automatic differentiation techniques such as symbolic differentiation,numerical differentiation,forward mode automatic differentiation and reverse mode automatic differentiation,and compares their advantages and disadvantages.Then based on numerical differentiation,the model of numerical differential neural networks for solving PDEs is implemented,and the model of reverse mode automatic differential neural networks for solving PDEs is implemented by Tensorflow.In the first model,the neural networks and the numerical differentiation of the finite difference method are combined.We use the central difference quotient to approximate the differential operator of equations and use the first order forward difference quotient to calculate the gradient of the loss function with respect to weights and biases.In the second model,we used techniques such as data augmentation,xavier initialization,mini-batch data sets with special proportions and dropout,which played a role in reducing model complexity,avoiding overfitting,and accelerating model convergence.The last part is numerical experiment.The advantages of the model of numerical differential neural networks for solving PDEs are low space complexity,and there will be no vanishing gradient problem even if the networks is very deep when using the sigmoid function as activation function.The disadvantages of it are high time complexity,low accuracy,and it can only solve simple partial differential equations.The advantages of the model of reverse mode automatic differential neural networks for solving PDEs are low time complexity,high accuracy,and it could solve complex partial differential equations.The disadvantages of it are high space complexity,and it needs to consume a lot of storage space and computing power.There will be vanishing gradient problem when using the sigmoid function as activation function.