Facial Emotion Recognition In Deaf: Eye Tracking And ERPs Study

The people of hearing loss bring inconvenience to their daily communication.At the same time,hearing loss may also affect their facial emotion recognition ability,which further affects their interpersonal communication.Previous studies have shown that there are some defects in the emotional recognition ability of deaf people's faces,especially negative emotion recognition.However,previous studies mostly used behavioral performance as an observational index,but little was known about the potential cognitive characteristics of deaf face emotion recognition.In this paper,eye movement and event-related potential analysis methods are used to study the potential cognitive abilities of deaf people in face emotion recognition tasks.According to eye movements and ERPs,we can analyze the cognitive characteristics of deaf people's face emotion recognition defects,so as to further understand the cognitive neural mechanism of deaf face emotion recognition,which can provide ideas and empirical basis for future deaf education.The study consisted of 4 experiments(two eye tracking experiments and two EEG experiments).Experiment 1: Facial emotional recognition in deaf.The study used eye tracking technology to examine the influence of face parts(eyes,mouth,nose)on the facial emotion recognition in deaf people,in order to understand whether the deaf has visual processing differences to compare with hearing people in the process of recognizing facial emotions;Experiment 2: Deaf people Emotional matching recognition research.Using the matching task of emotional faces and emotional words to examine the visual differences in the process of face-face emotion matching by deaf people;Experiment 3: Research on emotional perception of deaf faces.Through the EEG experiment,the deaf ability to perceive emotional faces and non-face images is examined,and then the perceptual characteristics of deaf people on emotional faces are analyzed.Experiment 4: Research on matching tasks of emotional faces and emotional vocabulary.Through EEG technology,we further examine the cognitive characteristics of deaf people in the process of emotional matching.It was found that in the first experiment,the accuracy rate of the deaf people's recognition of fear was significantly lower than that of the hearing person.The first gaze time of the deaf face was lower than that of the hearing person,and the preparation time of the deaf person entering the main part of the face when identifying the fear mood was more long.Experiment 2 showed that the deaf people were significantly lower than the hearing control group in the face-emotional(FW)and face-face(FF)matching tasks,but there was no significant difference in the matching task of the emotional word-face(WF).In addition,the fixation duration of the reference face and the target face in the FF task is longer than that of the hearing person.Experiment 3 shows that both the deaf and the hearing person have good ability to judge both face and non-face tasks,and there is no significant difference in behavioral results.However,EEG data showed that the N170 amplitude caused by deaf people was smaller than that of the hearing control group.Experiment 4 shows that the results of deaf behaviors are not significantly different from those of hearing people in the process of matching emotional words.EEG results show that the N400 amplitude of deaf people is significantly larger than that of hearing people.In addition,the accuracy of deafness matching in the emotional face matching task is lower than that of the hearing control group,and the N170 amplitude caused by the deaf is greater than that of the hearing person,but the amplitude of the LPP component is smaller than that of the hearing person.That is,in the early stage of the face matching task,the deaf people invested more cognitive resources,while the late stage of processing invested less resources than the hearing people.The results show that there are difficulties in the emotional recognition ability in deaf,mainly in the recognition of negative emotions,such as the recognition of fear and sadness.Especially in the recognition of fear,the deaf need to take longer to perceive the main part of the face.This difficulty is also accompanied by the fine processing stage of the face.The difference in cognitive processing may be one of the main reasons for the difficulty in emotional recognition of deaf faces.Although this study does not take the language level as an examination index,from the perspective of the matching task of emotional words,the deaf person shows different cognitive characteristics from the hearing person.