The Effects Of Audiovisuo Modal-based Attention On Sound-induced Illusory Flash

Although the human brain faces information from multi-sensory channels at all times,it does not give equal weight to the information in each sensory channel,which in some cases causes the auditory information to be prioritized to dominate the information from other sensory channels.Auditory dominance refers to that auditory information competes for preferential access to consciousness in multisensory integration.Sound-induced flash illusion(SIFI)is an auditory dominance phenomenon in multisensory integration,which is the perception of brief visual stimuli could be qualitatively altered by concurrent brief sounds.It has been well documented that SIFI reflects cross-modal interactions at a perceptual level,and the illusion is too robust to change in many stimulus parameters.It remains unknown how modal-based attention influences the SIFI effect.In study 1,we adopted the classical paradigm of the SIFI,directed the attention to the visual modal or the auditory modal endogenously,to investigate the modal-based endogenous attention on SIFI.Experiment 1 asked the participants to conduct the classical paradigm of the SIFI as the baseline.In experiment 2,the ratio of the visual modal,auditory modal and audio-visual modal was 3:1:1,that is the attention was directed to the visual modal endogenously;in experiment 3,the ratio of the visual modal,auditory modal and audio-visual modal was 1:3:1,that is the attention was directed to the auditory modal endogenously.The results showed that fission illusion(F1B2,F refer to the flash and B refer to the beep,1 and 2 were the number of flash or beep)was decreased when endogenous attention was directed to the visual modal,and fission illusion was increased when endogenous attention was directed to the auditory modal.However,regardless of the attention was directed to the auditory or visual modals endogenously,fusion illusion(F2B1)was not affected by the attention.The results indicated that the modal-based endogenous attention could affect the fission illusion in SIFI,but not affect the fusion illusion.In study 2,we investigated the adaptation of attentional channels to investigate the effect of altered pre-stimulus neural activity in the sensory system on the direction of sensory dominance,that is,whether the modal based adaption can affect the SIFI.Experiment 4 asked the participants to adjust the visual modal adaption,and experiment 5asked the participants to adjust the auditory modal adaption.The results showed that the visual modal based adaptation could affect the SIFI,and the ACC of the visual adaptation once was significantly higher than that of visual adaptation twice,this indicated that the ACC was higher when the number of visual adaptation flash was same as the number of visual flash.However,there wasno significant difference between visual adaptation flash once and twice.That is,visual adaptation does not affect the amount of illusions.In addition,the auditory modal based adaptation can affect the SIFI,the ACC of the auditory adaptation twice was significantly higher than that of the once,this also indicated the ACC was higher when the number of auditory adaptation beep was different fromthe number of auditory beep.Similarly,the auditory modal based on adaptation can influence the SIFI.In study 3,we not only manipulated the modal-based attention(selective attention vs.divided attention)to ask the participants to focus their attention either on the visual modal or the auditory modal,but also manipulated task difficulty(high vs.low)to investigate how the cognitive control influences the SIFI.In experiment 6,asking the participants to judge the number of flashes first,and judging the number of sounds.In experiment 7,we increased the task difficulty,asking the participants to judge the number of flashes first,then judge whether the number of flashes and sounds were consistent.Experiment 8 investigated the neural mechanisms of the modal-based attention on the SIFI.The results showed that fission illusion was larger than fusion illusion,and when the participants were initiative to pay attention to the auditory modal,resulting to enlarge the fission illusion,while it did not influence the fusion illusion.The results indicated that regardless of the single modal-based attention or the cross modal-based attention,the fission illusion was larger,and was more affected by the distribution of attentional resources.Through the three studies(8 experiments),we revealed the mechanisms of modal-based on attention to SIFI,especially fission illusion:(1)the endogenous modal based attention by regulating the ratio of the visual,auditory and audio-visual modals could affect the SIFI;(2)the visual and auditory adaptation could influence the SIFI;(3)paying attention to the visual and auditory modals could affect the SIFI,and this effect hadnothing to do with the difficulty of the task.