Time-frequency And Connectivity Feature Extraction During Speech Tasks And Its Application In Brain Computer Interface

Crucial frontier science nowadays include exploring human brain mechanism for intelligence and consciousness,understanding human behavior and emotion,innovative approaches for diseases diagnosis and treatment,neuromorphic computing and intelligent robot.Both brain-computer interface(BCI)and brain network are the important researching contents.This dissertation mainly explores the time-frequency and temporal-spatial characteristic of human brain mapping during speech tasks and its application in BCI.The auditory event related potentials(ERPs)and time-frequency features during speech processing were analyzed and extracted from electroencephalography(EEG)and multi-scale electrocorticography(ECoG)separately.A directed connectivity network was built and evaluated for exploring the dynamic speech processing.In addition,time-frequency and connectivity features were used to classify and explore the application in speech BCI.This study provides the foundation for speech BCI and brain network,which has significant values for brain science and cognitive neuroscience.The main research contents are as follows:1.Research on ERPs extraction from single trial using empirical mode decomposition(EMD).An auditory oddball paradigm was designed and the data were acquired from standard EEG channels.EMD and correlation coefficients were applied to extract features from single trial.P300 and N200 ERPs as well as N2 ac features were trained and tested by support vector machine classifier for binary classification.The results showed that EMD was able to detect the auditory ERPs in single trial and better performance can be achieved by involving N2 ac.2.Multi-scale ECoG time-frequency analysis and speech functional mapping.Both standard and micro electrodes were used for acquire the ECoG signals during syllables reading task.Event related activity was explored by time-frequency power spectrum.Compared with electrocortical stimulation(ECS),the results demonstrated that high gamma response can provide reliable speech localization and assistant the clinical surgeries.Also,brain activity during articulation was significantly different for three different places of articulation.3.Explore classification performance using different ECoG electrode scales.The magnitude of the high gamma(70-110 Hz)band was extracted from micro-and standard-ECoG recordings during syllable reading task and used to train a linear discriminant analysis(LDA)classifier for four classification problems.The results showed that syllable classification accuracies depend on array placement.The best decoding accuracies were often achieved with features extracted from multi-scale recordings(standard-and micro-ECoG),suggesting that the combination of high resolution recordings over vSMC with broad spatial coverage afforded by clinical macroelectrodes could improve the performance of speech BCI.4.Explore the dynamic processing of language in human language cortex and evaluate the network measures using complex network theory.A directed connectivity network was built in single trial involving both standard ECo G electrodes and micro-ECoG arrays using time-varying dynamic bayesian networks(TV-DBN).Degree centrality and eigenvalue centrality were used to evaluate the importance of nodes in brain networks,which washelpful for preoperative evaluation during epilepsy surgery.5.Connectivity dynamics for speech BCI.Cross-correlation and TV-DBN were selected to measure the functional and effective brain connectivity during syllable reading task,which can capture neural signals from distributed cortical cortex during speech processing.Both connectivity features were used to classify different types of syllables.The results showed that TV-DBN connectivity performe better than cross-correlation connectivity,and as well as the high gamma feature in most cases.