Self-Supervised Learning And Segmentation Methods For Microscopic Hyperspectral Pathology Image

Pathological analysis is the gold standard for diagnosing liver and gallbladder diseases,but diagnostic errors can occur due to factors such as fatigue and differences in experience among pathologists.With the rapid development of artificial intelligence,intelligent diagnostic technology combined with medical pathological images can partially alleviate the pressure on doctors.Compared with color images,hyperspectral images contain richer spectral information.The combination of hyperspectral imaging technology and deep learning has shown great potential in the medical field.However,there are some challenges in microscopic hyperspectral pathological image analysis.Firstly,due to the small number of cases,the hyperspectral pathological image dataset has a small amount of data.Moreover,labeling pathological images is very time-consuming,leading to difficulties in obtaining labels.Secondly,the dense and continuous spectral bands in hyperspectral images can lead to high information redundancy,affecting task performance.Finally,some annotations are of low quality and insufficient accuracy.To address the difficulty in obtaining annotations for microscopic hyperspectral pathological images,self-supervised learning can achieve excellent performance by using a large amount of unlabeled data for unsupervised pre-training and then fine-tuning with a small amount of labeled data.However,due to the small amount of data,hyperspectral pathological images are not suitable for direct application of self-supervised learning.Therefore,this paper proposes a weight expansion algorithm for RGB2 HSI based on the Vision Transformer(Vi T)that transfers the pre-trained model of large-scale color image datasets directly to the Vi T model designed for hyperspectral images.By using self-supervised training on microscopic hyperspectral pathological image data,the algorithm reduces the private domain information of natural color images in the pre-trained model and learns the private domain information of microscopic hyperspectral pathological images.To address the high information redundancy in hyperspectral images,this paper proposes a grouped residual encoding technology.Based on video residual encoding,grouped residual encoding groups and re-encodes hyperspectral images to remove redundant spectral information.Based on this,this paper proposes the Grouped Residual Encoded Patch-embedding Layer(GREPe L),a hyperspectral pathological image segmentation network that redesigns the image block mapping layer of Vi T to remove redundant information from hyperspectral images and efficiently extract spatial-spectral information.Furthermore,this paper combines the network with the RGB2 HSI weight expansion algorithm based on Vi T to further improve performance in microscopic hyperspectral image segmentation tasks.This paper used a microscopic hyperspectral imaging system to collect pathological data on liver cancer,processed the collected images,annotated with the guide of pathologist and created a microscopic hyperspectral liver cancer pathological segmentation dataset.The proposed method was validated on the collected microscopic hyperspectral liver cancer pathological segmentation dataset and a publicly available dataset of microscopic hyperspectral cholangiocarcinoma cancer pathological dataset.Some dataset had large labeling noise due to rough labeling.To address this issue,this paper re-annotated some data with high quality labels with the guide of pathologists and combined the proposed method with semi-supervised learning to use a small amount of high-quality labels instead of a large amount of low-quality labels,reducing the workload of annotating and improving the performance of the network in the segmentation task.The Vi T-based RGB2 HSI weight expansion algorithm proposed in this paper and the hyperspectral pathological image segmentation network based on GREPe L achieved 78% m Io U,86.94% m Dice,91.25% accuracy,and 73.91% Kappa in the microscopic hyperspectral liver cancer pathological image segmentation task.After combining with semi-supervised learning and reannotating,the proposed method achieved 79.68% m Io U,88.19% m Dice,92.85% accuracy,and76.4% Kappa in the microscopic hyperspectral cholangiocarcinoma cancer pathological image segmentation task.The experimental results show that the proposed method can effectively distinguish between cancerous and normal areas,improve the efficiency and accuracy of hyperspectral liver and gallbladder pathological image diagnosis,and have practical application prospects in the future.