Neural Correlates To Text-Picture Integration

Text and picture are two basic visual forms to convey information. In this ERP study we examined the neural correlates to text-picture integration by two semantic consistence evaluation experiments, which asked participates judged if the two stimulates(text-text/text-picture) can integrate into a reasonable semantics. Text-text condition as the baseline for both experiments.-Experiment 1 put the pictures behind the text for picture’s instantaneous processing. Experiment 2 put the Pictures in front of the text for separating picture’s processing and semantic integrating. Only the trials can be analyzed which could be integrated and be responded right.-The results in experiment 1 showed that in the stage of semantic integration, text-picture induced a more positive LPC than text-text. And text-picture activated more right hemisphere of the brain than text-text for the semantic fuzzy character of picture. Also text-picture unique activated the right side of the frontal lobe in the late period of integration. But in experiment 2, the LPC evoked by semantic integration had no significant difference between text-text and text-picture, the brain regions actived by semantic integration also had no significant difference.-This is because the picture has been fully processed before semantic integrating. In both experiments picture’s processing evoked a N200-300 over bilateral frontal lobe than text for representation’s switch from perceptual representation into propositional representation. The total results suggest that text-picture’s integration different from text-text integration, The former needs a representation’s switch involved in prefrontal area and a regulatory mechanism involved in the right side of the frontal lobe, but the final form of the semantic processing is the same.