Research On The Recommendation Algorithm Of Knowledge Graph And Multi-modal Knowledge Embedding In Telemedicine

With the rapid development of medical informatics in the era of big data and people’s continuous attention to the medical and health.systems,the analysis,processing and application of medical data play an important role in remote consultation.The text information of medical data is an important part of and the research basis for medical data.With the growing volume of medical data,scientific management and use of medical text data for remote consultation and auxiliary diagnosis and treatment remains being a pain-point and a challenge.The main reason is that medical text data contain not only the semantic information of the text,but also specific medical domain knowledge.However,the current models for text data are mainly based on general domain natural language processing algorithms,and the in-depth models and methods for medical domain are still lacking.Therefore,this dissertation constructs a recommendation algorithm based on the knowledge graph of auxiliary medical diagnosis and treatment in these three aspects:the construction of a Chinese medical knowledge graph model,the entity and relationship embedding of multimodal medical knowledge,and the parameter optimization of recommendation algorithm.The specific research contents are as follows:1.Having studied the existing knowledge graph and analyzed the existing medical knowledge graph and database,this dissertation proposes a new Chinese medical knowledge graph based on deep neural network BTA-Net.The knowledge graph construction model integrates the deep network models of BiLSTM,CRF and TextCNN.These deep network models enable the intellectualization of knowledge graph,ensuring the accuracy of prediction results and the operation efficiency.Furthermore,in order to improve the accuracy of the recommendation results and optimize the computational efficiency,this dissertation also introduces the selfattention mechanism into the deep network models.In the dissertation,600,000 sets of medical data that are collected are used to support the BTANet model and have achieved good performance.In the meantime,the prediction effect of the model is verified by using the mass electronic medical record data set.It is proved that the depth model introduced by the knowledge graph ensures the calculation efficiency,and the prediction accuracy reaches 90.84%,while the F1 score and the recall also perform at their best.The experimental results show that the knowledge graph model proposed in the dissertation can not only catergorize the diseases of patients,but also put forward treatment suggestions for patients and provide more accurate auxiliary diagnosis and treatment services for doctors and patients.2.Based on the further research of a new Chinese medical knowledge graph for telemedicine,the dissertation proposes a multi-BTA-Net knowledge embedding algorithm integrating multimodal knowledge.The algorithm not only embeds the Chinese medical knowledge graph,but also adds the medical text knowledge and medical image knowledge to the knowledge base of the algorithm in order to improve the accuracy of the algorithm,so that the medical structure knowledge,medical text knowledge and visual knowledge can serve the remote consultation recommendation algorithm.As it is difficult to integrate the existing knowledge graph with the neural network directly,the medical knowledge representation learning is used in the medical knowledge graph recommendation algorithm.After the learning of the vector representation of entity and entity relationship,the spatial relationship transformation from head entity to tail entity is learned,and then both learning are embedded into the proposed multi-BTA-Net neural algorithm.The results show that the proposed multi BTA-Net which integrates the characteristics of medical structured data,text and image is better than other models.Compared with other recommendation algorithms,the recall rate reaches the optimal 0.3126.3.Based on the further research of the multi-BTA-Net recommendation algorithm as proposed,the dissertation proposes to use the knowledge distillation structure to compress the parameters based on the visual knowledge processing module in the multi-BTA-Net recommendation algorithm,and the final recommendation algorithm,the simple-SE-ResNet is obtained.Since the medical knowledge graph recommendation algorithm proposed in the dissertation will eventually apply in telemedicine,it needs to consider the data interaction with the hospital information system.To ensure the performance efficiency in practical applications,it is necessary to effectively reduce the number of parameters introduced in the model.As a result the dissertation introduces the knowledge distillation algorithm structure into the multi-BTA-Net visual knowledge processing algorithm,while training a student model and a teacher model at the same time.The student model is a pre-generated model,which is the simple-SE-ResNet recommendation algorithm,and the teacher model is the SE-ResNet model for visual knowledge processing of the original multi-BTA-Net model.Through the combined training with the teacher model which distills the teacher model,the learned knowledge is transferred to the student model and enables the student model to obtain results similar to those of the teacher model with fewer parameters.In practical telemedicine applications,only the student model needs to be extracted for deployment.Experiments have proved that the simple-SEResNet model with a smaller complexity obtained after knowledge distillation has the final recall rate of 0.4410.On the basis of the knowledge graph network model,the dissertation completes a recommendation algorithm model integrating multimodal medical knowledge with step by step.Firstly,a Chinese medical knowledge graph model with deep neural network is built into the model.Secondly,the knowledge graph is embedded and introduced into the multimodal medical knowledge to realize the recommendation algorithm in the medical field.Then the distillation model is used to optimize the parameters of the practical recommendation algorithm.And finally,an accurate auxiliary diagnosis and treatment recommendation algorithm is created and make it avaible for use.