Research On Automatic Focal EEG Identification Based On Deep Reinforcement Learning

Epilepsy is a common human neurobrain syndrome,which is extremely difficult to treat.Abnormal synchronous electrical activity of brain neurons is the cause of epilepsy.When it continues to attack,it will make patients difficult to move,physical activity disorders,and may even cause irreversible brain damage.A significant number of patients with epilepsy are resistant to medication.For them,ordinary medication is hardly effective,and surgical removal of the epileptogenic zone to control epilepsy is the best treatment route.Therefore,precise localization of the epileptogenic zone is a prerequisite for successful surgery.The Electroencephalogram(EEG)records critical physiological information of the brain.However,the traditional EEG identification of epileptogenic zone is the visual interpretation of EEG by doctors,which is too subjective,and the EEG has characteristics such as large data volume,so the misclassification rate is high.Therefore,it is important to develop an effective method for EEG signal recognition in the epileptogenic zone for clinical application.In this thesis,the Deep Q-Network(DQN)is improved by adding Additional Functional Modules(AFM),and thus a new method for automatic recognition of epileptogenic zone EEG,namely,AFM-DQN.In this deep model,reinforcement learning sequential decision making weakens the current validity of the behavior and it looks at the long-term cumulative reward in the future.Under this principle,the model does not stick to local recognition information,but focuses on the information that is valid for the overall task of EEG recognition in the epileptogenic region in an integrated manner.Compared with the traditional reinforcement learning,this model combines the deep convolutional neural network,and combines the decision-making ability of Q-learning with the perception ability of convolutional neural network,thus greatly improving the learning ability of the network.AFM include pre-training,high-performance classifier,reward control mechanism,double Q-learning,dueling architecture and retrace.The presence of pre-training allows the main training to start with a positive classification prediction,greatly improving the training speed of the model.When high-performance classifiers are added to the model,it can work with the neural network to process the EEG data to achieve an overall improvement in model performance.The reward control mechanism can realize the automatic decay of learning rate and early stop of the model to better ensure the convergence efficiency of the objective function and avoid the occurrence of model overfitting.Double Q-learning,dueling architecture and retrace are three advanced reinforcement learning-related techniques that serve to stabilize and accelerate the training process,endowing the model with stronger generalization ability.In order to further reduce the computational burden of the network while maintaining the correlation and interdependence of EEG signals,this thesis introduces different types of EEG features and a semi-supervised feature selection algorithm(Semi-JMI algorithm)based on Joint Mutual Information(JMI).The evaluation standard of JMI is a measure of the redundancy,correlation and interdependence of the EEG features,and focuses on adding complementary information between the features.The semi-supervised correlation algorithm can process both labeled and unlabeled EEG data.The combination of all data,which can provide structural information,and labeled data,which can provide category information,is more efficient.The feature selection algorithm can combine some "soft" knowledge to search for optimization.The proposed automatic recognition method of epileptogenic zone EEG based on deep reinforcement learning is tested on two different EEG databases,and the classification accuracy reaches 95.87% and 97.5%,respectively,which proves the effectiveness and generalization performance of the method.On the one hand,compared with common high performance networks such as Discriminative Prototype Dynamic Time Warping(DP-DTW),Transformer,Residual Network(Res Net)and Fully Convolutional Network(FCN),the proposed method is significantly superior to these common network models.On the other hand,the method in this thesis has better classification results compared with the recently published EEG identification methods for epileptogenic regions.In future work,a larger EEG dataset will be collected in the clinic through collaboration with hospitals to further validate and refine the method proposed in this thesis.