| As traditional Chinese medicine(TCM)has demonstrated clear advantages in the global fight against COVID-19,and with continuous encouragement and promotion policies from the government,as well as a favorable environment for high-quality research,TCMs is experiencing a new trend of globalization and overall positive development with improved quality awareness.However,the demand for TCMs is increasing rapidly while contradictorily highlighting issues of quality.Quality concerns are particularly evident with regards to clinical drugs and raw materials used in preparations.The traditional "quality identification by appearance" approach relies heavily on human,making it subjective.Therefore,visual feature-based quality inspection has garnered attention.However,machine learning has been limited in their ability to capture deep associations,leading to low efficiency and accuracy.While deep learning has improved discrimination accuracy,suffering from limited data and overfitting,and end-to-end training difficulties.This dissertation focuses on the practical need for rapid inspection of TCMs,combines the current market circulation situation,and selects representative varieties such as Chuanbeimu,Banxia,Shanzha,and animal-derived Guijia,to pioneer a series of visual-analysis methods for quality inspection of TCMs based on Transformer.The main contents are as follows:(1)To solve the problems of subjectivity,single dataset,low accuracy,and difficulty inheriting traditional experience in current rapid quality inspection of TCMs,this dissertation proposes and implements an image data collection system that conforms to the special attributes of TCMs.We design and construct a suitable TCM image database for research needs,with Chuanbeimu,Banxia,Shanzha,and animal-derived Guijia being the main examples.(2)To address the limitations of existing deep learning-based evaluation methods such as overfitting,limited applicability,low recognition rates,and difficult end-to-end training,we propose an improved masked autoencoder-based method for the rapid identification of TCMs.Based on the data of in our datasets,our method involves a random data local augmentation preprocessing technique applied to the constructed dataset.This technique enhances the representation of features by randomly cropping images in a proportional manner.Moreover,we introduce a supervised pre-training network model with a hybrid multi-layer convolution architecture.This model incorporates parallel supervised training branches and adopts a CNN+Transformer network structure.These modifications enable the model to effectively integrate Masked Autoencoders(MAE)for capturing global image features,enhancing the network’s ability to learn local-global features,and extracting latent information from images more comprehensively.This method uses self-supervised transfer of massive "common knowledge" to expand visual feature data and enhance feature capture capabilities.Experimental results show that Ours outperforms baseline models and has good application prospects.(3)In response to the central challenge of fast and accurate differentiation of authentic medicinal herbs,we aim to overcome the limitations in fine-grained visual image classification accuracy discussed in the preceding chapter.The integration of shallow features and high-level information by injecting prior shallow knowledge to guide high-level semantic information.Based on this,we propose a new local spatial attention module that smooths the obtained attention weight by Softmax linearly transformed features,and enhances the perception of local finegrained features while resolving the current problem of spatial attention wherein max pooling more focuses on high-frequency textures and overlooks local details.The proposed method was validated using different datasets of Shanzha and Banxia in Chapter 2,and it outperformed others,thus improving the accuracy of fine-grained images from different regions.(4)Based on the practical goal of achieving efficient and accurate quality assessment of TCMs,we present a multi-view fusion algorithm for image classification utilizing the Transformer model.This algorithm specifically addresses the challenges of blind spots and occlusion of key points encountered in single-perspective research.By incorporating a cross-attention mechanism,it effectively models the local semantics of different views and investigates the interplay between local regions across multiple perspectives,thereby mitigating the information loss inherent in single-perspective image features.Additionally,we introduce a consistency loss that computes the cosine similarity between features from two perspectives,imposing a constraint on their distance.This facilitates enhanced feature interaction and mutual reinforcement among views,enabling the exploration of correlations and complementarities among features in multi-view herbal images.Experimental results show that ours obtains richer visual features,achieves better identification performance than others,and further improves the accuracy of TCMs.Moreover,it holds the potential to further enhance the accuracy of rapid quality assessment in the domain of TCMs.(5)basing on the key issues of subjective dependence on experiential skills of TCMs.Based on the current national codex standard property rules,we propose a transformer-based multimodal property feature generation model for TCMs by using dual visual features.We introduce the multi-scale mixed network from Chapter 4 and the joint Deformable DETR detector to capture the coarse-grained information and fine-grained details of images.Compared with existing models,our proposed method better enhances the spatial interaction of grid features and the upper and lower associations of regional features to achieve complementary representation of image information.Simultaneously,we design and implement an intelligent TCMs identification system based on deep learning,to achieve the inheritance and innovation of traditional experiential knowledge of the properties of TCMs. |
PDF Full Text Request