The Characteristics And Neural Mechanisms Of Novel Stimuli In Multi-Sensory Paired Associative Learning

With the development of multimedia technology in teaching,the modals for people to receive information have become rich and diversified,from unisensory such as vision,hearing and touch,to multi-sensory such as audio-visual,visual touch information reception,integration and learning,and the number of different sensory directly affect the learning effect.Some researchers have found that compared with unisensory,multi-sensory information receiving speed is faster,information integration learning effect is better.There are many studies on unisensory and multi-sensory influencing factors,such as spatial attention,expectation evaluation and so on.However,less studies has been done on the characteristics of the stimulus itself,such as the novelty of the stimulus.Therefore,based on the deficiencies of existing studies,our study aims to investigate the differences in learning effects of stimulation of novelty on unisensory and multi-sensory,and further explore its neural mechanism in combination with ERP research.In study 1,a two-factor group design was adopted with the independent variable of 2(sensory:unisensory and multi-sensory)×2(stimulus:ordinary and novel),and the experimental paradigm of paired associative learning was adopted to investigate whether the new and unusual features of stimulus would promote associative learning under the condition of audio-visual integration.The study found that:(1)compared with unimodal,crossmodal can effectively promote the reception and integration of stimuli,which only exists in the early and middle stage of learning.(2)compared with ordinary stimuli,novel stimuli are more difficult to receive and integrate quickly.It shows that in the process of learning,compared with the unimodal information input,multi-sensory information input can make people better receive and integrate,and the learning effect is better.Novel stimuli increase cognitive load for people,and learning new knowledge slows down cognitive processing.In study 2,a two-factor group design was adopted with the independent variable of 2(sensory:unisensory and multi-sensory)×2(stimulus:ordinary and novel)to investigate the neural dynamic process of new and novel features of stimulation in joint learning under the condition of audio-visual integration.The study found that:(1)in the first learning stage,the amplitude of the cross-modal is corrected,while the amplitude of the unimodal is more negative.(2)in the second stage of learning,the amplitude difference between the integrate learning of novel stimuli and the integrate learning of ordinary stimuli is significant.(3)in the last learning stage,both unisensory and multi-sensory induce positive wave and crossmodal correction.(4)CNV is one of the indicators of multi-sensory learning.It shows that there is a significant difference between unisensory and multi-sensory learning,and the integrate learning of novel stimuli is more difficult than the integrate learning of ordinary stimuli.