Research On Microscopic Image Target Recognition Based On Zero Shot Learning

Microscopic examination result is an important basis for modern medical diagnosis,but for a long time,microscopic examination has mainly relied on manual inspection by doctors.Because of its complex operation,low efficiency,and the dependence of test results on doctors’ experience,object recognition technology based on deep learning has gradually become a new trend in the future development of our country’s medical care.However,compared with traditional image processing techniques,deep learning methods still have a strong dependence on a large amount of high-quality training data manually labeled,which often brings higher costs.This problem is more pronounced in medical microscopic image object recognition.Compared with natural images in traditional object recognition tasks,medical microscopic image samples are scarce,difficult to collect,difficult to identify objects,and more expensive to label.In recent years,deep learning technology based on zero-shot learning has made great progress,and even the performance on some specific tasks has been able to match the results obtained by some supervised methods.Therefore,this thesis starts from the knowledge graph-based zero-shot object recognition technology for microscopic images,and proposes a series of methods that are conducive to improving accuracy and reducing labeling requirements.The main research results of this thesis are as follows:(1)This thesis proposes a deep Graph U-Net with 2 Attention-based graph pooling layers(GUN-2A).Aiming at the Laplacian over-smoothing problem existing in deep graph convolutional networks,this thesis introduces a U-shaped graph convolutional neural network.The structured deep graph convolutional neural network aggregates messages from the global perspective of the graph,and proposes a self-attention-based graph pooling method to alleviate the over-smoothing caused by the deep graph network.Test results on ImageNet 21 K show that the proposed method outperforms the baseline methods on all 5 Hit@k metrics.(2)This thesis proposes a graph contrastive learning method based on topologically symmetric dual knowledge graphs(CGUN-2A).Aiming at the semantic interval problem of zero-shot learning in a single knowledge graph,an additional CLIP knowledge graph is introduced to establish a topologically symmetric dual knowledge graph to assist the alignment of visual-semantic space manifolds;graph comparison learning is introduced on the basis of the aforementioned GUN-2A The paradigm,CGUN-2A,is proposed to constrain class expression,maintain intra-class similarity and expand inter-class separability,and alleviate domain drift.On the ImageNet 21 K dataset,the method achieves state-of-the-art performance under all Hit@k metrics.In terms of alleviating the offset problem,compared with the previous method,under the settings of general zeroshot learning and generalized zero-shot learning,Hit@1 is relatively improved by 33.3%and 47.1%,respectively.(3)This thesis realizes the microscopic image target recognition algorithm based on CGUN-2A.Establish a medical microscopic image knowledge map,introduce super depth of field information and optical flow information to optimize zero-shot learning from the visual side.