Research On Brain Encoding And Decoding Technologies Via Functional Magnetic Resonance Imaging

Understanding the relationship between external information and corresponding brain activity patterns is an essential step in elucidating the fundamental mechanisms of brain functions.In order to address this problem,researchers have developed a variety of brain encoding and decoding models based on functional magnetic resonance imaging technology(fMRI).Encoding models aim at describing how the information about the external stimulus or task is presented in patterns of brain activity and how well the brain activity can be predicted from various sensory stimuli information.In contrast,decoding models attempt to predict information about external stimuli from observed brain activity patterns.These studies help us better understand the mechanisms of brain' response to external stimuli and provide practical solutions to many applications,such as brain-computer interfacing(BCI),clinical diagnosis and artificial intelligence.However,current brain encoding and decoding studies mainly focus on experiments design and there are still many limitations in terms of the key technologies including the model design and brain network detection methods.In order to alleviate these problems,in this dissertation,we focus on the key technologies involved in brain encoding and decoding studies.Briefly,we proposed a novel encoding model,a novel decoding model,an individual brain network detection method and a group-wise brain network detection framework.The details of our work are as follows:Firstly,we propose a novel DICCCOL-based(Dense Individualized and Common Connectivity-based Cortical Landmarks)encoding model to predict human brain's activity patterns under free viewing of video clips.Specifically,the video stimuli are quantitatively represented by a variety of representative visual features in the computer vision community which describe the motion information,local shape information,global color distribution and spatial information embed in the videos.After that,the DICCCOL system is adopted to locate the large-scale brain networks/ regions across different participants and then functional connectivities between different brain regions are calculated as the brain's activity patterns responding to video clips.Finally,the encoding model which bridges stimulus features and brain activity patterns is trained with least-squares support vector regression(LSSVR).Compared with traditional method,our method quantitatively describes the external stimuli and generates a unified encoding framework across different subjects based on DICCCOL system.Further experiments demonstrate the brain activity patterns during free viewing of videos can be accurately and robustly predicted by those visual features across different subjects via our encoding model.Secondly,we propose a novel sparse representation based decoding model to reconstruct audio saliency features from recorded fMRI data.First,we extract biological-plausible auditory saliency features for each audio excerpt.Then,for each subject and audio exerpt,the wholebrain fMRI signals during free listening to audio excerpts are utilized to identify brain activity pattern dictionaries involved in audio comprehension via online dictionary learning approach.Finally,the auditory saliency features in each audio clip are effectively reconstructed via sparse representation method using the identified brain activity patterns from the second step.Compared with traditional methods,this is the first time,sparse representation is utilized as a unified framework to model the interactions between audio saliency features and fMRI signals.Experiment results demonstrate these audio saliency features can be effectively reconstructed from brain activity patterns across different subjects and audio exerpts.Thirdly,we propose a supervised dictionary learning framework to identify brain networks under task condition.First,each participant's whole-brain fMRI signals are extracted and aggregated into an fMRI signal matrix.After that,the signal matrix is sparsely represented with a hybrid dictionary basis and coefficient matrix via supervised online dictionary learning method.Please note that each column in dictionaries represents a typical brain activity pattern and each corresponding coefficient vector represents the spatial distribution of the brain network.Our hybrid method combined the advantages of both model-driven method and datadriven method and could identify task-related brain networks and intrinsic brain networks simutaneously.Experiments on the publicly available human connectome project(HCP)taskbased fMRI datasets have demonstrated the great advantage of the proposed framework.Fouthly,we propose a multi-stage sparse coding framework to identify accurate groupwise consistent brain functional networks across subjects in a principled way.To start with,we concatenated all the fMRI signal matrice temporally and adopt online dictionary learning method to characterize the initial group-wise functional maps across different participants.Then,we constrain these initial group-wise maps in dictionary learning procedure to identify individualized temporal patterns from individual fMRI data.After that,we fix these individualized temporal patterns in another dictionary learning procedure to generate individualized functional networks from individual fMRI data.Finally,we employ statistical mapping method to characterize group-wise consistent functional networks using individualized functional networks.Compared with current group-wise brain network detection methods,our model is more powerful and sensitive and the result is easier to interpretation.Application of the proposed framework on two groups task-based fMRI data has demonstrated the superiority of the proposed framework in identifying group-wise consistent functional brain networks.