| Emotion is a physiological and psychological activity that occurs when a person faces the objective world and plays an important role in human communication.Compared with normal people who have complete auditory function,the lack of hearing function in deaf subjects easily leads to the problems of emotion expression impairment and emotion cognition bias.With the development of cognitive science and brain science,brain-computer interface technology can use EEG signals to better characterize the intrinsic changes of emotions.Traditional EEG emotion recognition methods only target single temporal information or spatial domain information,ignoring the influence of inter-channel on the genesis of emotion.Therefore,this paper proposes a domainadapted(HVF-DANN)emotion classification method based on the horizontal vertical representation of EEG signals in deaf people,with the following main studies.(1)An experimental paradigm of video-evoked emotional stimulation of EEG signals of deaf subjects was designed,and EEG signals under three emotions(positive emotion,neutral emotion,and negative emotion)were collected from 15 deaf subjects,and each subject collected EEG data three times according to a certain time interval,and an emotion dataset containing three times of EEG data from 15 deaf subjects was established.(2)An emotion recognition model considering EEG signal flow and subject variability was proposed.After EEG signal preprocessing,frequency domain differential entropy features(DE)were extracted and considering the flow and diffusion effects of EEG signals,EEG emotion representations based on the forms of horizontal and vertical representations were designed,and long and short-term memory networks(LSTM)were used to obtain the spatial domain features of horizontal and vertical representations of EEG signals.Then,the DE features,horizontal representation features and vertical representation are fused using the attention mechanism method,and the inter-class distance is increased and the intra-class distance is decreased by adding the central alignment loss function.Finally,the domain adaptation method is used to classify the sentiment representation features.(3)The proposed method was validated on the deaf dataset and the public emotion dataset(SEED dataset),and the experimental results were 75.90% and 93.99% under the subject-dependent emotion three classification task and 65.99% and 84.22% under the cross-subject emotion three classification task,respectively.In addition,the effectiveness of the model structure was further demonstrated by the ablation and visualization experiments.The results of the ablation experiments demonstrated that the domain discriminator and the central loss function could effectively reduce the intersubject variability and thus enhance the cross-subject sentiment classification ability. |
