Research On Task-state Functional Magnetic Resonance Image Decoding Based On Deep Learning

Exploring the brain’s activities and understanding brain cognitive mechanisms has been one of the major topics in neuroscience.Decoding brain cognitive states from neuroimaging signals is commonly used in brain functional characteristic research.Functional magnetic resonance imaging(fMRI)serves as a noninvasive,whole-brain covering method of neuroimaging that could provide an effective measure for decoding brain activity.In recent years,deep learning has been widely recruited for multiple brain state decoding due to its superior performance in several domains.However,several open challenges still need to be addressed in this field.The key idea is to propose a high-accuracy,interpretable,transferable,and repeatable deep neural network framework for fMRI brain decoding.Inspired by these challenges,this study focuses on the foundation,interpretability,and application of deep learning-based fMRI brain decoding.The research contents of this paper are as follows:First,4D convolution and attention modules are proposed in this study to improve the performance of the fMRI classification model.The high-accuracy classification model is the foundation of brain decoding.In order to fully utilize the inherent spatiotemporal interaction features in fMRI,this study proposed a 4D convolution to jointly extract the spatial activation distribution and time-dependent features in fMRI data.What’s more,the attention module was integrated into brain decoders to enhance the discriminative representation of extracted features.The proposed model was evaluated on seven cognitive classification tasks in the Human Connectome Project dataset,which showed high accuracy(97.4%)and faster convergence than previous studies.The model pretrained in multiple cognitive domains was transferred to individual feature regression and event-related design fMRI data for fine-grained visual stimulus classification.The results show that the transfer learning achieved good performance,which shows exploration significance for the application of fMRI brain decoding.Besides,this study explored the visualization methods of fMRI brain decoding.On the one hand,the visualization analysis of attention distributions based on the proposed attention modules showed that the high decoding performance was driven by the response of biologically meaningful brain regions.The attention modules have excellent coverage,which prefers to highlight the useful regions and diminish the noise areas.The focused layouts get finer and are remarkably task-specific as the layers become higher.What’s more,low-level attention masks remained similar to the source domain,whereas high-level attention masks changed adaptively after transfer learning.On the other hand,by decoding the different types of stimulus images in the BOLD5000 dataset through guided back-propagation,the result demonstrated that the proposed model could locate the corresponding brain area correctly even under the event related design fMRI data,which provides new evidence for the functional organization of highlevel visual cortex.In addition,knowledge distillation based on feature maps can improve the performance of simple parameter models,which provides a new idea for the application of fMRI brain decoding.