Research On Microscopic Hyperspectral Pulmonary Cytopathological Image Segmentation Based On Deep Learning

Pulmonary cancer is currently one of the cancers with the highest morbidity and mortality in the world.Accurate pathological diagnosis of pulmonary cancer is of great significance for the stage determination and subsequent treatment.With the rapid development of artificial intelligence in the medical field,a large number of computer-aided diagnosis algorithms and digital pathological analysis algorithms based on microscopic imaging technology have been proposed.These algorithms can provide objective and reliable diagnostic data for clinical diagnosis.However,the limited spatial information in color images captured by traditional microscopic imaging systems has resulted in a bottleneck for pathological analysis of pulmonary cancer cells with similar features.Thanks to advances in microscopic hyperspectral imaging,a new field of research in pathological image analysis is being fostered to explore additional spectral information compensate for the aforementioned shortcomings.However,it is difficult to obtain pathological samples,and it is even more difficult to obtain their high-precision pixellevel manual annotations.Therefore,how to extract representative pathological cell features from limited or noisy microscopic hyperspectral images is an urgent problem to be solved.In this paper,the research of microscopic hyperspectral pathological cell segmentation combines the advantages of microscopic hyperspectral images and deep covolutional neural networks to study the texture and spectral characteristics of pulmonary cells.This paper starts from the optimization and improvement of the 3D image feature extraction algorithm based on clean data.Then,this paper tries to extract effective pathological cell features based on noisy labels to obtain regional cell segmentation results.Finally,this paper extracts key pathological cell features based on limited highquality data to obtain pathological cell segmentation results.The main contributions of this paper include:· To extract representative image features from clean data and eliminate the information redundancy in microscopic hyperspectral images,this paper proposes a three-dimensional convolutional neural network(3D-Pul CNN)based on princi- pal component extraction algorithm and a convolution combination unit to accurately classify three types of pulmonary cancer cells.Different types of cells are segmented based on the classification results to provide quantitative virtual analysis data for pathologists.In order to efficiently utilize the pathological features of cells in microscopic hyperspectral images and remove the redundant information,the model uses the principal component extraction algorithm to obtain the most important 8 bands,which greatly reduces the model’s computational cost.In order to simultaneously extract multi-scale spatial features and spectral features,the proposed model designs a new convolutional combination unit module to fuse features of different scales,thereby constituting an effective and efficient classification model.To prove that 3D-Pul CNN can effectively improve the classification performance of pulmonary cancer cells,a pulmonary cancer microscopic hyperspectral image dataset is established which includes pulmonary squamous cell carcinoma,pulmonary adenocarcinoma and small cell carcinoma.Extensive experimental results on this dataset show that 3D-Pul CNN can obtain an overall accuracy of 0.962 and precision of 0.963,which proves the superiority of the proposed model.Finally,this study segments all pulmonary cancer cells based on the classification results and calculates their morphological characteristics.These characteristic data not only re-verifies the performance of the classification model,but also provides quantitative virtual analysis data for the follow-up research of pathologists.· In regards of model instability caused by noisy label,this paper proposes a twostage adaptive joint training framework(TAJ-Net)based on few-shot learning and denoising algorithm.In order to obtain the regional segmentation results of various pathological cells,different from the traditional convolutional neural network based on semantic segmentation or instance segmentation,this study proposes a two-stage framework to solve the problem of small inter-class differences and high overlap of cells in pulmonary adenocarcinoma images.In the first stage,a few-shot segmentation algorithm is proposed to learn a high-precision cell mask to remove the interference of the cluttered background for the subsequent regional cell segmentation.In the second stage,the model proposes an adaptive joint training model based on a classify-and-segment strategy,which can adaptively denoise and analyze small patches based on both texture and spectral features of pathological cells.Finally,this model proposes a cell classification result mapping algorithm,which maps the classification results to the original microscopic hyperspectral image in the form of a circle to obtain regional cell segmentation results.This study establishes and publishes a hyperspectral image dataset of pulmonary adenocarcinoma with pixel-level annotation(https://bio-hsi.ecnu.edu.cn/).And TAJ-Net can obtain a Dice similarity coefficient of 0.7671 on this dataset which is far superior to the popular segmentation algorithms.And a large number of ablation studies also proves that spectral information can be a good supplementary information for the spatial information.· In regards of poor feature learning ability caused by the lack of high-quality labeled data,this paper proposes a few-shot segmentation model(Spread-Net)based on spatial-spectral relationship and adaptive prototype feature extraction.In order to make full use of the three-dimensional features of microscopic hyperspectral images,this model randomly crops them into small patches of three kinds of pathological cells according to manual annotation for subsequent pathological feature analysis.However,the boundaries between pathological cells and the background are blurred and too many cells are overlapped with each other.Therefore,this model proposes an image feature encoder based on spatial-spectral relationships,which enhances the cell boundary features by calculating the relationship between the foreground and surrounding background.Next,in order to efficiently extract the prototype features of each category as well as preserve local information,this model designs an adaptive prototype feature extraction module,which adaptively selects class-level prototypes or local prototypes based on the proportion of foreground and background in image features.The prototype feature extraction algorithm makes full use of the advantages of both class-level and local prototypes to improve the performance of few-shot pathological cell segmentation.This study also conducts comprehensive experiments on the micro- scopic hyperspectral image dataset of pulmonary adenocarcinoma.Spread-Net can obtain a Dice similarity coefficient of 0.7315,which proves that the model can overcome the shortcomings of limited number and cell types as well as blurred boundaries based on microscopic hyperspectral imaging technology.Finally,it can achieve accurate pathological cell segmentation results.