Exploring Self-supervised And Weakly-supervised Learning For Histopathological Image Analysis With Limited Annotations

Pathological examination of tissue samples has long been considered as the gold standard for cancer diagnosis.Digital pathology is an emerging field that utilizes digital tools and systems to scan,analyze,and manage tissue samples.Whole slide images(WSIs)are digital representations of tissue sections with gigapixel sizes,which could contain hundreds of thousands of cells.Cancer diagnosis is a time-consuming and subjective task that heavily relies on the expertise of pathologists and is prone to inconsistency.However,computational pathology has emerged as a promising solution to these challenges,which involves the utilization of computer technology and image processing techniques for automating the analysis of pathological images.Computational pathology assists pathologists in optimizing the diagnosis workflow,reducing their workload,and improving the objectivity and consistency of pathology reports.Furthermore,it may help pathologists uncover valuable medical information,such as biomarkers from routine hematoxylin and eosin(H&E)-stained histopathological images,thereby accelerating the development of digital pathology in clinical practice.Existing computational pathology algorithms have demonstrated the ability to surpass human recognition accuracy on certain tasks such as prostate cancer classification,given sufficient training data and fine-grained annotations.However,the unique characteristics of WSI(e.g.,gigapixel image size,multiple cancer types,and wide staining variations)make acquiring large amounts of annotations costly and challenging.As a result,histopathological images often lack fine-grained annotations.This paper addresses the shortage of annotated histopathological images by utilizing self-supervised and weakly-supervised learning techniques.The proposed approach involves three research components: a universal self-supervised feature extractor for histopathological images,a general weakly-supervised WSI classification algorithm,and a weak-annotation-guided WSI search algorithm.The contributions and main contents of this paper can be summarized as follows.(1)This work proposes a universal self-supervised feature extractor for H&E-stained histopathology images.Traditional self-supervised contrastive learning methods use two augmented views of a sample(or instance)as a positive pair,which is not suitable for histopathological images due to their gigapixel size and heterogeneous tissue distributions.Since numerous similar tissue phenotypes exist both within and across WSIs,and these publicly available WSIs lack detailed labels,this paper proposes a semantically relevant contrastive learning(SRCL)method.SRCL compares the relevance between instances to mine visually matched positive pairs,thereby enhancing the diversity of positive samples and resulting in more informative representations.A hybrid model(CTrans Path)is employed as the backbone,which integrates a convolutional neural network and a multi-scale Swin Transformer architecture.CTrans Path is pretrained on massively unlabeled histopathological images,and it serves as a collaborative local-global feature extractor to learn universal feature representations more suitable for histopathology image-related tasks.The proposed feature extractor is applied to various tasks,including colorectal tissue classification,weakly-supervised WSI classification,mitotic detection,colorectal gland segmentation,and histopathological image search,achieving state-of-the-art performance.The results show that our SRCL-based visual representations are also more robust and transferable than other self supervised learning methods and Image Net pretraining(both supervised and self-supervised methods).(2)This work proposes a generalized weakly-supervised WSI classification algorithm for H&E-stained histopathology images.The challenge of the weakly-supervised WSI classification task is reflected in the limited availability of information(with only weak annotation available at the patient or WSI level),while the unknown information dimension encompasses billions of pixels within the WSI.The separability within WSIs and the information interaction between global WSIs are not considered by existing methods.To address this problem,this paper designs a weakly-supervised classification method based on cross-slide contrastive learning(called SCL-WC),which consists of two parts: self-supervised feature pre-extraction and weakly-supervised feature refinement and aggregation.The feature extractor uses the CTrans Path.The feature aggregator consists of three modules: a class-specific deep attention module,a positive-negative-aware module,and a weakly-supervised cross-slide contrastive learning module,which implement attention-based feature aggregation,the separation of abnormal and normal patches within each WSI,and the pulling of WSIs with the same disease types closer and pushing of different WSIs away,respectively.SCL-WC achieves state-of-the-art performance in four different classification tasks: lymph node metastasis prediction,breast cancer classification,prostate cancer classification,and microsatellite instability prediction.Our method also exhibits superior flexibility and scalability in weakly-supervised localization and semi-supervised classification experiments.(3)This work proposes a content-based WSI retrieval algorithm that uses weak annotation as a guide.The gigapixel size of WSI poses significant challenges in feature encoding,WSI similarity measurement,and interpretability.To address these challenges,we propose a WSI retrieval framework that integrates a clustering-guided self-supervised feature learning method and an uncertainty measurement-based WSI retrieval technique.By converting the WSI retrieval process into patch retrieval,this method returns not only a set of WSIs that are similar to the query WSI but also identifies subregions that exhibit high similarity between the returned WSIs and the query WSI.This provides pathologists with easily interpretable search results.The proposed WSI retrieval framework has been evaluated on the tasks of anatomical site retrieval and cancer subtype retrieval,using over 22,000 slides,and our results show a significant improvement over other state-of-the-art methods.Moreover,the feature representation learned from our framework can also be used for patch retrieval,outperforming the use of Image Net pretrained features.