Research On Analytical Methods To FMRI Functional Brain Networks Based On Deep Neural Networks

In recent years,the advances in functional magnetic resonance imaging(fMRI)has enabled people to record and study the functional activities of the human brain with remarkable spatiotemporal resolution non-invasively.Notably,the research of functional brain networks is one of the frontiers of brain science,which provides important value for understanding the brain and clinical applications in brain diseases.However,existing methods for fMRI functional brain network analysis face critical challenges.Most of them are based on conventional shallow linear models and hence perform poorly in coping with the hierarchical features and complex nonlinear spatial-temporal correlations embedded in fMRI data,rendering limitations such as poor separation of functional brain networks,and being vulnerable to noise and weak consistency in group-wise spatial or temporal patterns.Compared with conventional shallow linear models,DNN(deep neural networks)-based methods have been making rapid progress in recent years,showing better performance in terms of nonlinear feature learning and complex data expression.The advances in DNN bring new opportunities for developing fMRI data analysis methods.This thesis focuses on developing DNN-based analytical methodologies to address the limitations of existing methods in four aspects,including blind source separation of functional brain networks,hierarchical feature analysis of fMRI data,functional brain network state recognition in multi-task,and multimodal information fusion in brain network analysis.The major innovations and contributions of this thesis are summarized as follows:1.A blind source separation(BSS)method based on nonlinear restricted Boltzmann machine(RBM)was proposed to address the limitations of existing linear BSS methods in separating source components related to functional interaction among spatially distributed brain regions.The main innovations of this method are two-fold: 1)fMRI time series instead of volumes were treated as modeling samples,alleviating the contradiction between high model complexity and limited training samples;2)The KL-divergence sparsity regularizer was adopted to explicitly constrain the activation rate of hidden layer units,improving model interpretability and performance in BSS.The experimental results on Human Connectome Project(HCP)task-fMRI data showed that,compared to existing ones,the proposed BSS method not only suppresses the intermix effect and false-positive activation in task-related source components,but also effectively separates the source components that can reflect the functional interaction among multiple brain regions.2.A hierarchical fMRI data analysis method based on deep convolutional autoencoder(DCAE)was proposed to address the limitations of existing approaches based on shallow models,including limited feature learning abilities,ignoring the hierarchical temporal structure of fMRI time series.The proposed DCAE-based method used one-dimensional convolutional layers as building blocks to construct a deep auto-encoder,which aims at reconstructing the input fMRI data as accurately as possible.This strategy enables the DCAE to model fMRI data in an unsupervised manner and thus avoids labor-intensive sample labeling in supervised learning.By regulating and controlling the receptive field of convolutional kernels,the DCAE can learn the hierarchical features of fMRI signals more effectively.In addition,the ability of the DCAE to deal with the diverse shifts of fMRI signals could be significantly improved by combining the convolutional layers and max-pooling layers.Two novel validation experiments were designed to validate the proposed method.1)The relationship between the hierarchical features learned in DCAE and a set of theoretical hemodynamic response models were established to verify the effectiveness of obtained hierarchical features.2)A brain network identification method based on a sparse representation algorithm was applied to the high-level features learned in DCAE,and the identified brain networks were compared with that applied to the original input fMRI data.The experimental results showed that,compared with shallow models,this method could learn the hierarchical temporal features of functional brain activities more effectively.And the features learned on the deepest layer of DCAE can best reflect the diversities of theoretical hemodynamic response models.3.On the basis of the aforementioned DCAE,a functional brain state recognition method based on mixture of deep expert network(Mo DEN)was proposed to deal with the challenge caused by the high dimensional characteristic of fMRI.This method consists of two phases:the ground-truth maps extraction phase and the brain states recognition phase.In the first phase,the DCAEs’ convergence speed was improved by using deconvolutional layers which tied the weights of their decoders and encoders.Then,the improved DCAEs were adopted to extract ground-truth maps of different tasks from low-dimensional fMRI signals,alleviating high model complexity faced by modeling directly on high-dimensional fMRI volume.In the second phase,The Mo DEN was constructed by taking the previously proposed DCAE model as building blocks.Brain network state recognition was achieved by calculating the similarity between the unknown brain state and the ground-truth maps,which not only avoided the training of traditional classifiers but also improved the recognition accuracy.In addition,the discriminative brain networks of different tasks could be obtained by statistically comparing the ground-truth maps in different tasks.The experimental results on HCP task-fMRI data showed that high accuracy of 97.36% could be achieved by the proposed method in recognizing brain network states during social,language,and working memory tasks of HCP data.The learned discriminative brain networks can effectively characterize the difference in functional brain activities among different cognitive tasks.4.Two DNN-based multimodal fusion methods were proposed to explore the hierarchical spatial structure of the brain network and the functional brain networks underlying selective visual attention,respectively:· A method based on deep belief network(DBN)that fuses structural and functional brain networks was proposed to explore the hierarchical structure of brain networks.This method fuses functional and structural brain networks to improve brain network consistency across multiple subjects,which is essential for exploring the hierarchical structure of brain networks.Specifically,the "structure trace-map" and "function trace-map" were proposed for feature representation of structural and functional networks,respectively.A DBN model was trained to fuse structural and functional brain network trace-map features.The hierarchical structure of brain networks was inferred from the hierarchical features learned in multi-layers of the DBN model.The experimental results showed that the spatial consistency of corresponding sub-networks in different subjects could be significantly improved after fusion and then hierarchical brain networks can be effectively obtained.Compared with the low-level brain networks learned in the shallower layer of DBN,the high-level brain networks learned in the deepest layer of DBN have more complete global information in structure and function.· A multimodal fusion method based on biased competition theory was proposed to study the functional brain networks underlying selective visual attention in a natural stimulation environment(movie watching).This method integrates visual contents of movies,eye-tracking recordings,and naturalistic paradigm fMRI data to establish the relationship between visual contents and functional brain activities,and then to explore functional brain networks underlying selective visual attention.The proposed study pays attention to both "attentionrelated contents" and "neglected contents" in visual scenes,while existing studies mainly focus on the former one.A deep neural network that mimics biased competition theory was proposed to establish the relationship between visual contents and functional brain activities,which significantly mitigated the difficulties in quantifying attention-related measures in the movie stimulus.The experimental results showed that the proposed method could effectively identify functional brain networks related to selective visual attention.