EEG-based Classification Of Imaginary Mandarin Tones

In recent years,the development of brain-computer interface technology based on external stimulation had become increasingly mature.However,for patients with external communication organ injury,the role of external audio-visual stimulation and feedback was very limited,so it was difficult to use the existing brain-computer interface technology based on external stimulation.For most paralyzed patients,the nervous system was still intact,leading researchers to propose brain-computer interfaces based on phonological imagery to help them communicate with the outside world.However,as the current brain-computer interface technology without external stimulation was still in its infancy,many problems need to be solved.How to decode the electroencephalogram(EEG)signal to identify the speech recognition system is a significant problem.In order to recognize speech signals,we need a speech perception system that can recognize different vowels,consonants or tones.For non-tonal languages such as English,as long as different consonants can be recognized,the speech perception system can have a better speech recognition effect.However,for tonal languages such as Chinese,the additional tone information of its finals has the function of distinguishing sounds,so it plays an important role in speech recognition.However,the existing researches on speech perception mainly focused on the influence of vowels,consonants,intonation and other factors on speech perception,and the important role of tone in speech perception was often neglected.Therefore,this paper probed into the problem of multiple classification of mandarin tone pronunciation imagination based on EEG,which was of great significance to the speech synthesis system based on pronunciation imagination brain-computer interface technology.Based on previous studies,this paper proposed a brain-computer interface paradigm based on speech imagery,and compared the effects of visual stimuli and audio-visual stimuli on the results of phonetic imagination classification of Mandarin tones based on EEG.Based on the different cue conditions,the experimental paradigm could be divided into paradigm 1(visual stimulus condition)and paradigm 2(audio-visual combination stimulus condition).Both of the two experimental paradigms were composed of the cue stage and the imagination stage.The visual and audio-visual combination of the stimulus conditions would only appear in the cue stage,while the two experimental paradigms were identical in the imagination stage.Under the two stimulus conditions,the subjects were required to imagine the corresponding tone pronunciation in the imagination stage according to the stimulus in the prompt stage,and at the same time,EEG data were collected to classify the tone.In this paper,the common space model algorithm was used to extract the feature vectors of different tones,and then the support vector machine and neural network were used to realize the feature classification of different tones of Mandarin.The experimental results showed that compared with the visual stimuli(average classification accuracy:67.7%),the EEG based Mandarin tonal pronunciation imagination task has a better classification accuracy(average classification accuracy: 80.1%)in the audio-visual combination of stimuli.The results of this paper showed that it was easier to decode the tone information of Mandarin from the recorded EEG data in the task of imagining Mandarin tones based on EEG,especially under the stimulus condition of audio-visual combination.