Effects Of Endogenous Attention Directed To Channels On The Audiovisual Integration Of Emotional Information

People usually integrate facial expression(visual channel)and emotional vocabulary(auditory channel)in the brain,recognition of simultaneous presentation and semantically consistent dual-channel information is significantly faster and more accurate than any single-channel information.This phenomenon is called the audio-visual integration of emotional information.In recent years,more and more researchers have focused on the influence of endogenous attention conditions on the integration of audio-visual channels,but related studies have focused on simple audio-visual stimuli,ignoring the implications of the integration of complex stimuli such as emotional information.In this study,the experimental materials were expanded to explore the influence of endogenous attention and attention load on audio-visual integration of emotional information.Using face expression pictures and single-syllable emotion words as visual and auditory channel information,designed three experiments: experiment one was 2(attention conditions: selective attention,distributive attention)x 3(target stimulus: visual channel,auditory channel,audio-visual channel)within the subjects of design,the experimental task was emotional valence judgments,the results showed that under the condition of distributive attention,the participants’ reaction of audio-visual dual channel was significantly shorter than any single channel(p<0.001),showing the redundant signal effect,and under the condition of selective attention did not appear the effect;experiment two based on experiment one,increased visual attention load to the experimental material,2(load conditions: visual load,no visual load)x 3(target stimulus: visual channel,auditory channel,audio-visual channel)within the subjects of design,it was found that,in the absence of visual attention load conditions,the participants’ reaction of dual channel was significantly shorter than any single channel(p<0.001),showing the redundant signal effect,in visual load condition,did not appear the effect;experiment three based on experiment one,increased auditory attention load to the experimental material,the dual task paradigm,2(load conditions: auditory load,no auditory load)x 3(target stimulus: visual channel,auditory channel,audio-visual channel)within the subjects of design,the results showed that regardless of auditory attention load,the participants’ reaction of dual channel were significantly shorter than any single channel(p<0.05).Conclusion:(1)Endogenous channel attention affect the integrated of audio-visual channels of emotional information,and only attention to visual and auditory emotional information(distributive attention)conditions produce the phenomenon of audio-visual integration processing of emotional information,indicating that the multi-sensory integration effect exists in the integrated processing of more complex emotional stimuli in addition to simple stimulation and speech integration processing;(2)the effect of visual and auditory attention load changes on the phenomenon of audio-visual integration processing of emotional information is asymmetric.