Research On Automatic Generation Of Medical Reports With History Information

Medical imaging plays a vital role in diagnostic management and medical treatment.Imaging inspection has become a very common inspection method.Imaging doctors need to browse a large number of images and write diagnostic reports with accurate content,standardized structure,and clear semantics,which brings great challenges and workload to doctors.In recent years,artificial intelligence,especially deep learning,has made great progress in various tasks,and deep learning has also provided more possibilities for the automatic generation of medical reports.The automatic generation model of medical report text can automatically interpret the content of medical images,analyze the lesion information contained in it,and then convert the information into medical reports.It has great clinical application value and has attracted widespread attention from scholars.The task of automatic generation of medical imaging reports can be roughly classified as generating text from pictures.From the early template generation method,to the encoding and decoding model,to the attention mechanism,Transformer,BERT,and GPT,it has greatly improved the smoothness of automatic text generation.Spend.However,since medical reports have completely different language characteristics and usage scenarios from other ordinary long texts,they need to be highly standardized and accurate,and there are still great challenges in this task.First of all,due to the unavoidable ambiguity and ambiguity of image interpretation,it greatly affects the accuracy of text report generation.The above problems can be alleviated by using historical report information.However,simply using historical report as input often leads to the situation that historical report information and current diagnostic information cannot be balanced,making historical report information become noise that affects the generation effect.Furthermore,historical reports contain rich structural information.How to use this structural information to make the model add logical and content constraints and prompts to the reports generated during the process of generating long text reports is an important issue that needs to be considered.In fact,in terms of model evaluation,most existing methods use models and indicators that are similar to long text generation tasks in other fields.The distinct structural characteristics of the specific style of medical reports and the need for higher wording accuracy have not been utilized and valued.This paper mainly focuses on the link of generating accurate and standardized text reports based on the keyword list interpreted from medical images.In response to the above challenges,research on the automatic generation of medical reports that integrates historical information is carried out.The main contributions of this paper are as follows:(1)Propose a multi-layer attention-assisted medical report generation model.In this model,keywords from historical reports and image interpretation are used as input,and two attention components are simultaneously built,one for learning important semantic and sequential information from keyword lists,and the other for learning current keywords Correlation between lists and historical reports.The model demonstrates generative performance that surpasses baseline models.(2)A tree-based medical report generation model is proposed.In order to further improve the accuracy of medical reports and make good use of its relatively fixed text structure,this paper introduces the tree structure into the text generation model based on historical reports,and proposes an automatic generation method for medical B-ultrasound reports based on tree structure.In this method,a text-tree structure conversion module,a tree structure-matrix conversion module,and a structure compression method are designed for keywords and historical reports respectively,and how to generate the tree structure,how to represent the tree structure,and how to scale the tree structure are solved.Control and other key issues.The model was tested on real medical data sets,and the evaluation results were close to or exceeded0.9 on BLEU1～BLEU4.(3)A tree-based medical report evaluation index is proposed.In order to measure the accuracy of medical report structure and wording,the text-tree structure conversion module is used to convert the generated report and the target report into a tree structure,so that a path from the root node to the leaf node can clearly and completely express the The characteristics shown by a certain trait or indicator in a certain place.On this basis,an evaluation index based on tree structure(Tree Structure Aware Matrix,TSAM),multi-focus index and corresponding evaluation method are proposed.The experimental results show that the evaluation index proposed in this paper can measure the quality of the generated report in a more fine-grained manner,such as whether the report describes the location of the lesion accurately,whether the lesion is missing,whether the corresponding description of the lesion attribute is accurate,etc.,which is more in line with the doctor’s concern about the quality of the report.