Research On Brain Tumor Image Segmentation Based On Transformer And Contrastive Learning

Precise brain tumor image segmentation is crucial for patients’ clinical treatment and serves as a prerequisite for implementing surgical procedures such as tumor resection.At present,deep learning-based brain tumor image segmentation techniques exhibit unique advantages in terms of segmentation accuracy and efficiency.From the perspective of fundamental computational frameworks for deep learning models,existing methods can be categorized into those based on convolutional computation and those employing self-attention mechanisms.Currently,self-attention-based segmentation models have surpassed their convolutional counterparts in performance;however,the former necessitates dividing three-dimensional volumes into two-dimensional slices as model inputs,inevitably leading to the loss of volumetric information,which is critical for the model to learn inter-slice dependencies.Most deep learning-based brain tumor image segmentation methods predominantly fall under the category of supervised learning techniques,relying on large quantities of annotated data for model training.As the collection process for brain tumor images is relatively complex and manually annotated images are scarce,effectively harnessing limited annotated images and utilizing unlabeled data for model training have emerged as pressing challenges.To address the issue of volumetric information loss resulting from two-dimensional slicing,we propose TBT-Unet,a self-attention Transformer-based brain tumor image segmentation method.This approach ensures the integrity of volumetric data in its inputs,preserving information in a hierarchical manner along the volume axes.During the decoding process,cross-attention mechanisms are introduced to establish connections between the decoder’s query and the encoder’s key and value pairs,preserving comprehensive global information.To tackle the scarcity of annotated images,we propose CFE-BTS,a brain tumor image segmentation method based on contrast feature enhancement,aiming to maximize the information extracted from limited annotated images.In clinical practice,physicians often compare normal and pathological images to identify tumor regions.In CFE-BTS,contrast learning is employed to align features in non-tumor areas of normal brain images and tumor-bearing brain images,using normal brain images as reference points to accentuate and enhance the image features of tumor areas,consequently improving brain tumor image segmentation accuracy.Regarding the dependency on large amounts of annotated data for model training in brain tumor image segmentation scenarios,we propose TCLP-BTS,a Transformer-based and contrastive pre-training brain tumor image segmentation method to enable model training using unlabeled data.By combining transfer learning and contrastive learning,we pre-train Transformers and subsequently apply the pre-trained models to the downstream brain tumor image segmentation tasks,ultimately achieving the goals of model training and feature learning with unlabeled data.TBT-Unet and TCLP-BTS each propose Transformer-based brain tumor image segmentation methods from the perspectives of improving model architecture and refining pre-training strategies,respectively.CFE-BTS and TCLP-BTS both employ contrastive learning in model internals and model pre-training components to enhance target region features and improve model feature learning effectiveness,respectively.Collectively,these research directions present supervised and semi-supervised learning methods for brain tumor image segmentation based on Transformers and contrastive learning.