| Objective:This study attempted to break the bottleneck in Zheng Su differentiation by deep learning,with the powerful nonlinear fitting as its ability,and to construct a deep learning Zheng Su differentiation model by data-driven strategy,thus transforming Zheng Su differentiation from"knowledge-driven"to"data-driven"and providing new ideas for the development and improvement of Zheng Su differentiation.Besides,based on traditional Chinese medicine(TCM)objectification,this study applied multimodal fusion to model construction and optimization and achieved essential data portrayal by high-level feature extraction and fusion,thus solving the subjectivity in symptoms judgment and providing algorithmic support for full intelligence in TCM Zheng Su differentiation.Methods:This study was divided into three parts:the first part was the analysis of the theory and current situation of Zheng Su differentiation;the second part was the construction of a deep learning Chinese medical Zheng Su differentiation model based on data-driven strategy;and the third part was the optimization of the deep learning Zheng Su differentiation model.The experimental steps were as follows:1.Case data collection and labeling:suitable patients were selected based on the nadir criteria.Clinical data was collected by qualified physicians while tongue diagrams were captured by the TFDA-1 Tongue Diagnostic Instrument developed by Shanghai University of TCM.2.Case data labelling:53 generalized Zheng Su items were selected as diagnostic criteria in the Zheng Su Diagnosis and three experts were invited to perform the Zheng Su identification separately.Consistent results were considered as the final results,while inconsistent results were resolved by consultation among these three experts.Provincial-level or above famous TCM specialists were consulted in case of non-uniform consultation results.3.Zheng Su diagnosis based on traditional Zheng Su differentiation:The test set data(i.e.,the test set that were divided during the construction of the deep learning model)were identified by the Wenfeng-III TCM Intelligent Diagnostic Aid System.4.Construction of the Deep learning Zheng Su differentiation model:Taking the patient’s symptoms in Type 2 Diabetes Mellitus TCM Symptoms Questionnaire as input information,the model streamlined the variables and divided the dataset into training and test sets at 4:1 ratio,then performed embedding feature coding,followed by neural networks such as cascaded fully connected layer and batch normalization layer for task prediction.The binary cross-entropy function and the Adam optimizer were adopted for loss calculation and parameters updating.The optimal model was saved in case the error was no longer decreasing.F1 values,precision,and recall were performed to compare the diagnostic performance of traditional Zheng Su differentiation and deep learning Zheng Su differentiation model.5.Optimization of the deep learning Zheng Su differentiation model:two unimodal differentiation models and one multimodal fusion differentiation model were constructed.The unimodal prediction model refers to a tongue image-based Zheng Su differentiation model(T-Model)and a questionnaire data-based Zheng Su differentiation model(S-Model)on account of Type 2 Diabetes Mellitus TCM Symptoms Questionnaire,while the fusion model was a multimodal fused Zheng Su differentiation model with tongue image and questionnaire data as input information(TS-Model).In the T-Model,firstly,the tongue segmentation model was constructed based on U~2-Net,and its segmentation effect was evaluated by the average Dice coefficient;secondly,the segmented images were fed into the differentiation model with Res Net34 cascaded fully connected layer as the network structure for training,and the loss was calculated by binary cross-entropy function and the Adam optimizer was applied to update the parameters;at last,the optimal model was saved in case the error was no longer decreasing.S-Model adopted embedding layer cascading multiple fully connected layers and batch normalization layer as the network structure.TS-Model employed the feature fusion model,that is,extracting respectively 128 features for tongue map and questionnaire data by corresponding nets,followed by cascading the prediction network after feature fusion via concatenate mode.Finally,the prediction performance of the above models was compared by F1 value,accuracy,and recall.Results:1.Two thousand five hundred eighty-five cases were included in this study,of which 1425 were male,accounting for 55.09%.The mean age of all cases was 58.67±12.12 years old,while the mean BMI was 24.46±3.42 kg/m2.There were 825 smokers,accounting for 31.95%.A total of1330 cases had comorbidities(51.45%),and the top three comorbidities were hypertension,hyperlipidemia,and carotid plaque.2.In addition to the tongue and pulse,the common symptoms of type 2 diabetes mellitus were dry mouth,thirst and excessive drinking,numbness in the limbs,dimness of vision,pain in the limbs,dizziness,weakness,tingling,fatigue,bitterness in the mouth,insomnia,chest tightness,and polyuria.14 Zheng Su were detected in this study,including 6 location Zheng Su(kidney,stomach,lung,heart,spleen,and liver)and 8 nature Zheng Su(yin deficiency,dampness,blood stasis,heat,qi deficiency,Yang deficiency,phlegm,and qi stagnation).The most common location Zheng Su were kidney,stomach,and lung,accounting for 60.43%,48.55%,and 43.17%,respectively,while the most common nature Zheng Su were yin deficiency,dampness,blood stasis,and heat,accounting for 81.32%,45.45%,36.56%,and 32.22%,respectively.The combination of disease location and disease nature was more prevalent with double disease location and double disease nature,accounting for 39.8%and 40.77%.The deficiency-exuberance intermingled pathologies were mainly observed,accounting for 90.48%.3.The F1 values of the traditional Zheng Su differentiation in lung,stomach,kidney,spleen,heart,liver,qi stagnation,qi deficiency,yin deficiency,blood stasis,dampness,heat,phlegm,and yang deficiency were 13.600%,18.519%,62.777%,51.163%,91.765%,1.942%,27.273%,38.318%,76.512%,37.751%,46.369%,64.744%,35.088%,45.349%,while that in deep learning prediction model being 96.788%,92.683%,95.181%,91.525%,97.826%,0.000%,46.154%,73.543%,94.104%,76.440%,69.053%,70.358%,75.676%,and 99.048%.An overall higher level of F1values was found in the deep learning prediction model.4.Three models were constructed in the third part of this study.The average Dice coefficient of the tongue segmentation model in the T-Model was 0.93.The predicted F1 values of T-Model in lung,stomach,kidney,spleen,heart,liver,qi stagnation,qi deficiency,yin deficiency,blood stasis,dampness,heat,and phlegm were 47.401%,51.934%,65.702%,0.000%,14.085%,0.000%,0.000%,26.519%,86.726%,37.119%,73.814%,52.247%,and 0.000%,respectively,while that in the S-Model were 96.312%,92.216%,95.038%,91.525%,97.826%,0.000%,50.000%,68.085%,93.195,75.127%,51.572%,61.654%,80.000%,99.048%and in the TS-Model were 96.760%,92.754%,95.652%,91.525%,97.826%,66.667%,55.556%,74.146%,94.050%,77.083%,70.732%,73.139%,85.000%,99.065%.A stable and high F1 value was found in the TS-Model.Conclusion:1.Intelligent analysis of the general condition and the Zhengsu distribution of disease could be accomplished by data-driven strategies.2.The Zhengsu differentiation model based on the data-driven strategy,which could also be updated by accumulated data,was created.3.A better diagnostic performance was found in the tongue differentiation model predicting"dampness",demonstrating the critical value of the tongue image for TCM Zhengsu differentiation at the algorithmic level.4.Multimodal fusion technology could optimize and improve the diagnostic performance of the TCM Zhengsu differentiation model,thus could be applied for establishing a highly intelligent Zhengsu differentiation model with entirely objective four diagnostic information. |
PDF Full Text Request