| In real life and work,we often encounter situations where perceptual information is incomplete and uncertain.The fusion of multi-sensory information can improve the accuracy and speed of brain perception and decision-making.Visual and auditory information are the main channels for people to perceive the outside world.The study of the brain working mechanism of perception and decision-making under different matching degrees of audiovisual information is of great significance to the development of cognitive neuroscience and the construction of brain-like cognitive model architecture.However,at present,there is no unified conclusion on the internal electrophysiological activities of the brain in the perception and decision-making process under different audiovisual information matching degrees,and the brain working mechanism is still unclear.To solve the above problem,this thesis designs experimental paradigms for perceptual decision-making under different matching degrees of audiovisual information,and collects behavioral and EEG(Electroencephalograph)data of 16 healthy subjects.From four aspects of behavior,ERP(Event-related potential),ERP signal source location,and dynamic causal network based on ADTF(Adaptive directed transfer function)algorithm,research the brain working mechanism of perception and decision-making under different matching degrees of audiovisual information,complete The main work and research conclusions are as follows:(1)The design of experimental paradigm.The ambiguity of visual information and the matching degree of audio-visual information are defined,and the matching degree of audio-visual information is set to 60%,80%,and 100%.(2)Statistical analysis of behavioral results.A one-way analysis of variance and verification of the behavioral data of the subjects showed that the improvement of the matching degree of audiovisual information significantly improved the accuracy and speed of perceptual decision-making.(3)Statistical analysis and discussion of ERP components.A one-way analysis of variance and a post-mortem test were performed on the ERP data.The different stages of perceptual decision-making were divided,and the four most important inducing components at different stages were determined: the early visual perception P1 component,the early auditory perception N1 component,and the mid-term audiovisual component.Information integration component P200 and decision-making component CPP.The statistical test results show that: different audiovisual information matching degree mainly affects the early audiovisual information perception stage and the midterm audiovisual information integration stage,and does not significantly affect the decision-making stage.(4)EEG source imaging analysis.Source imaging of ERP components and Sn PM(Statistical non-parametric mapping)statistical tests were performed to determine the activation of cerebral cortex sources at different stages of perception and decisionmaking under different matching degrees of audiovisual information.The brain areas activated in the early stage of audiovisual information perception mainly include:occipital visual joint cortex,temporal lobe auditory cortex,parietal lobe-frontal confluence area;mid-term audiovisual information integration stage activated brain areas mainly include: occipital visual joint cortex,prefrontal joint Cortex: The brain areas activated in the later decision-making stage mainly include: the prefrontal higher sensory cortex,the primary motor cortex,and the premotor cortex.The statistical test results show that different audiovisual information matching degree mainly affects the early stage of audiovisual information perception and the intermediate stage of audiovisual information integration,and the most obvious impact on the intermediate stage of audiovisual information integration.(5)The analysis of the cortical source signal ADTF dynamic causal network.Based on the source location results,the ADTF algorithm is used to construct a dynamic causal network of cortical source signals.Preliminarily determined the information exchange status of brain regions at different stages of perception and decision-making under different matching degrees of audiovisual information.There are significant two-way causal connections between the occipital visual cortex,the temporal lobe auditory cortex and the parietal-frontal junction in the early stage of audiovisual information perception;the two-way causal connection between the occipital visual cortex in the middle stage of audiovisual information integration The relationship is strongest.There is also a two-way causal connection between the temporal lobe auditory cortex and the occipital visual co-cortex;there is a significant two-way causal connection between the prefrontal co-cortex and the parietal-frontal junction in the decision-making stage;information flow in the decision-making stage The intensity is significantly higher than the other two stages.In general,the information exchange patterns of the activated brain regions under different matching degrees of audiovisual information are basically the same;with the improvement of the matching degrees of audiovisual information,the strength of causal connections between activated brain regions has increased,and the intensity of information exchange has increased.At the same time,through the cluster analysis of the dynamic causal network matrix,the six main dynamic activity patterns of the brain in the process of audiovisual information perception and decision-making are determined,and the information flow process of the brain in the process of audiovisual information perception and decisionmaking is further determined.The process is as follows: The visual and auditory channels respectively perceive information,and then the visual and auditory brain regions carry out information interaction to realize the integration of audiovisual information,and then the integrated information is collected through the parietalfrontal junction area to the prefrontal higher sensory cortex for decision-making. |
PDF Full Text Request