Research Of Contrastive Learning-based Deep Clustering

With the development of the big data,more complex data has become easier to obtain.By analyzing these data,we can extract the relationship between them and apply the relationship of these data to multiple fields,such as facial recognition,sentiment analysis,intelligent manufacturing,etc.At the same time,models used for classifying data with different labels have become the basis for many applications.For labeled data,people often view labels as the most important information guiding modeling,but label acquisition often comes with high costs.Therefore,finding a quantifiable objective method to guide the clustering modeling process on unlabeled data is crucial.In the past few decades,many clustering methods have been proposed,including distance-based clustering,density-based clustering,distribution-based clustering,hierarchical clustering,etc.However,these shallow models only work when the data dimension is low or the features are very representative.Due to their poor feature learning ability,these models are usually limited in performance when dealing with complex data.Therefore,to map the original complex data to a feature space that is easy to cluster,many clustering methods focus on feature extraction or feature transformation.Among these methods,deep neural networks have become an effective method due to their outstanding non-linear mapping ability and flexibility in different scenarios.In deep neural network-based clustering,deep clustering based on contrastive learning has been a research hotspot in recent years.Contrastive learning aims to make the model learn the similarity and difference between samples,which is in line with the goal of clustering tasks.This paper is based on deep clustering using contrastive learning,first introducing existing methods,then proposing solutions to address their shortcomings and our structure.We first propose a contrastive clustering method that does not require negative samples.Current methods for contrastive learning-based clustering models mostly compare each sample with other similar samples to formulate positive pairs to learn the similarity between samples,and compare them with dissimilar samples to compose negative pairs to learn the dissimilarity between samples.As a result,more accurate contrastive learning will inevitably lead to larger batch size,and the problem of imbalanced positive and negative samples in the training set will increase the systematic bias of the dataset,making the training very unstable.Therefore,(1)we introduce a contrastive learning architecture based on the BYOL structure,which only learns the membership of samples through the alignment of positive samples.(2)This work also introduces the idea of generative adversarial networks to further optimize the membership through discriminative networks.(3)To further improve clustering accuracy,we also introduce the idea of contrastive between clusters,to further distinguish different cluster centers.This design significantly improves the model’s training efficiency and generalization ability,and because it does not require negative samples,our method has higher training efficiency.During training,each sample is sampled only once per iteration without considering negative samples.In addition,our model is insensitive to systematic biases in the training data,which means the model can also have good applicability to unseen samples.Next,we propose a contrastive clustering based on variational networks that can perceive contextual information.Current contrastive learning-based self-supervised learning uses instance-level contrast objectives to achieve contrast between samples and class-level contrast to distinguish between classes.However,existing contrastive learning methods do not consider the entire data class situation and assign a single data point to a particular class,which often leads to suboptimal clustering allocation results and insufficient feature representation.To address this issue,our work proposes a novel deep clustering solution based on contrastive learning that can focus on sample contextual information:(1)We incorporate the idea of variational autoencoder into our method,to learn the overall distribution information of the data rather than the features of individual samples.(2)We obtain more representative category representations by using features sampled from distributions,expanding the existing contrastive learning’s ability to explore contextual information without changing the basic framework of contrastive learning,thereby improving clustering accuracy.(3)We derive the variational lower bound and jointly optimize the variational lower bound and contrastive learning loss,promoting each other to optimize the global feature distribution and membership attribution.