Identifying Spatio-Temporal Patterns Of Holistic Functional Brain Networks Via Deep Learning Methodologies

It has been widely demonstrated that complex brain function processes are realized by the interaction of holistic functional brain networks which are spatially distributed across specific brain regions in a temporally dynamic fashion.Therefore,identifying spatio-temporal patterns of holistic functional brain networks based on fMRI data provides a foundation for understanding complex brain function.Initially,traditional methods such as correlation analysis were proposed to characterize functional brain networks.Subsequently,matrix decomposition methods such as sparse representation were utilized to decode spatio-temporal patterns.Compared to these conventional methods,recent deep learning methodologies achieved superior performance.Although promising results have been achieved,existing deep learning methods still have two limitations.First,they underutilized spatio-temporal features of fMRI and merely focused on one single（spatial or temporal）pattern identification during functional brain network characterization.Second,existing supervised deep learning methods merely modeled a single functional brain network instead of holistic ones.To address the first limitation,we proposed a novel Spatial-Temporal Attention 4D Convolutional Neural Network（STA-4DCNN）model.Specifically,STA-4DCNN model adopted a Spatial Attention 4D CNN（SA-4DCNN）to model the spatial patterns,and a Temporal Guided Attention Network（T-GANet）to characterize the temporal patterns guided by the corresponding spatial patterns of functional brain network.Compared to previous studies,this approach employed guided attention mechanism to help focus on both spatial and temporal patterns identification,and adopted a novel 4D Convolution to fully characterize both spatial and temporal features during spatio-temporal pattern identification.To address the second limitation,we proposed a Multi-Head Guided Attention Graph Neural Network（Multi-Head GAGNN）model.Specifically,a spatial Multi-Head Attention Graph U-Net was adopted to model the spatial patterns of holistic functional brain networks,and a temporal Multi-Head Guided Attention Network was utilized to model the temporal patterns though the guidance of the identified spatial patterns.Compared to previous studies,this approach not only employed guided attention mechanism and novel Attention Graph Convolution to fully characterize both spatial and temporal features of fMRI data,but also utilized the branching structure to model holistic functional brain networks.Both of the proposed methods were evaluated on seven task fMRI datasets from the public Human Connectome Project（HCP）and one resting state fMRI dataset from the public Autism Brain Imaging Data Exchange I（ABIDE I）.The experimental results show that both STA-4DCNN and Multi-Head GAGNN achieve superior identification ability and generalizability of functional brain networks when compared to other state-of-the-art（SOTA）methods.We further applied these functional brain network spatio-temporal pattern modeling methods onto brain disease detection and cognitive behavior prediction.First,the proposed STA-4DCNN was applied on the ABIDE I dataset and successfully identified abnormal spatio-temporal patterns of functional brain networks in autism spectrum disorder（ASD）when compared to typical developing（TD）subjects.Second,we utilized the identified spatio-temporal patterns via the proposed Multi-Head GAGNN to predict the individual cognitive behavioral measures and achieved better prediction performance compared to other SOTA methods.In general,our proposed methods for spatio-temporal pattern modeling of functional brain networks provide useful tools for functional brain network characterization at individual level,and provide novel insights in brain disease identification as well as understanding brain-cognitive behavior associations.