Multi-modal Deep Learning-based Brain MRI Segmentation

The brain is one of the most important organs in the human body,which is closely related to the health of the nervous system and affects human cognition.In clinical medicine,magnetic resonance imaging(MRI)segmentation is a critical step in diagnosing and analyzing brain diseases as well as aiding surgical treatment.Multiple sclerosis(MS)brain lesion segmentation is a representative task for diagnosing and analyzing brain diseases,and aiding surgical treatment focuses on the cerebral vein segmentation.Usually,experts combine multiple modalities of MRI to enhance segmentation performance by using the complementarity of information between them.Theoratically,multi-modal deep learning algorithms can also be used for the segmentation of MS brain lesions and cerebral vein.However,for MS segmentation,existing deep learning-based multi-modal fusion strategies couldn’t fully exploit the correlation and complementary knowledge between brain MRI modalities,ignoring the classification characteristics of MS lesions in different modalities.In addition,MS brain lesions are small and numerous.These issues make it difficult to achieve expected MS segmentation performance.Currently,traditional multi-modal fusion strategies are used to address cerebral vein segmentation problems.However,they are time-consuming and highcomputational.And it’s limited by less data and enormous manual annotation costs to introduce deep multi-modal fusion strategies into cerebral vein segmentation.To address these issues,the thesis proposes the following solutions:1.Regarding the multi-modal fusion strategy of MS brain lesion segmentation,the thesis proposes local-global feature-based algorithm,LGMS-Net.The design fully considers the correlation and complementarity of information between MRI modalities for MS patients.In the encoder,LGMS-Net proposes an additional local edge feature extraction module(LEFE)to acquire edge information,according to the iron ring in the Phase modality.In the decoder,LGMS-Net proposes a local-global feature fusion module(LGFF),which fuses the information of FLAIR and Phase modality to segment MS brain lesions.Experiments evaluate and validate the algorithm’s segmentation performance and the fusion strategy.Additionally,LEFE visualization provides a possibility for MS pathological grading.2.Dealing with the obstacle of introducing multi-modal deep learning methods into cerebral vein segmentation,the thesis proposes a fast multi-modal cerebral vein segmentation algorithm based on deep learning plug-and-play(PnP)modules.The algorithm uses deep unrolling and other technologies to design some PnP modules instead of the time-consuming processes of traditional multi-modal methods.Experiments show PnP deep learning-based multi-modal algorithm far exceeds traditional methods in cerebral vein segmentation efficiency and can also achieve the expected segmentation performance at the same time.3.In order to explore engineering value of the proposed algorithms,the thesis designs a brain MRI segmentation system based on the flask framework.The system consists of user management module,data management module and segmentation algorithm module,which integrate the two proposed algorithms into a front and back-end separated B/S platform.The brain MRI segmentation system fully considers the security of medical data and the daily diagnostic and treatment needs of doctors,providing a user-friendly brain MRI segmentation system interface.