Stranger Faces—Cross-Channel Matching Of Voice Identity Information

In order to investigate people’s ability of matching unfamiliar person’s face and their voice,and the internal connections between face and voice,we conducted two experiments.Experiment 1 used Eye-movement technique and adopted modified XAB paradigm,presenting participants a piece of voice and two faces simultaneously and asking them to choose the right face which matched the voice.In subsequent experiments participants were required to do recognition tests and CRT(Combined Raven’s Test)to explore the cause of the matching ability.Experiment 2 was to investigate the internal mechanism of this phenomenon,so we adopted the ERP technique,presenting participants a series of spoken sentence,each one was followed by a face photo.The sentence was either spoken by the person shown as the photo followed,or by the other person,so as to constitute congruent and incongruent conditions.Participants were required to judge whether the face and the previous voice belong to one person.The results of Experiment 1 showed the accuracy reached to 57.55% which is significantly above random probability(p < 0.01);Eye movements data showed that only for the AOI of eyebrow & eye regions,the IA First Run Dwell Time and the IA Dwell Time of match are longer than the time of mismatch,and the IA Fixation Count of the match is more than the mismatch.After experiment,participants ran face recognition,voice recognition and combined Raven’s Test ’.These results showed that participants’ accuracy was in proportion with their memory and inferential capability.The results of Experiment 2 showed the accuracy was significantly above random probability(p < 0.01);ERP data showed that in male voice-face block,incongruent conditions compared to congruent conditions induced a more negative component in the prefrontal lobe and temporal lobe of left hemisphere,the time window are 680ms～780ms and 250～550ms respectively.In female voice-face block,incongruent conditions elicit a more positive component than congruent conditions within the time window of 800ms～900ms in the left prefrontal lobe.The conclusions are follows:(1),People have the ability to correctly match strangers’ voices with their faces.And this ability is influenced by humans’ learning experience,memory ability,inferential capability,etc.(2),there is an inherent connection between specific voices and faces,"specific faces match specific sounds",which has the characteristics of natural selection and evolutionary significance.