Automatic Generation Of Brain MRI Image Reports Based On Deep Learning

The automatic generation of medical image reports is a multidisciplinary research field involving computer vision,natural language processing,medicine and other disciplines.It aims to automatically analyze medical images through deep learning of existing data,so as to obtain human-like intelligence natural language image reports.It is a hotspot problem in smart medicine and computer-aided diagnosis field.This research can improve the work efficiency of radiologists,reduce the workload of radiologists,and shorten the waiting time of patients.It has important theoretical research value and good application prospects.In recent years,the development of deep learning technology has promoted the combination of computer vision and the medical field.However,due to the diverse characteristics of medical imaging and the high accuracy requirements of the medical field,the automatic generation of image reports needs further research.Unlike most previous studies focusing on the simple situation of lung CT,this paper deeply analyzes the characteristics of brain MRI imaging reports,cooperates with hospital neurology experts to establish a stroke MRI dataset,and proposes a multi-task method combining lesion detection,multi-label classification and generatiaon of reports.The main research contents of this paper are as follows:(1)We deeply analysis of the prevalence characteristics of ischemic stroke,and established a large-scale ischemic stroke imaging dataset Stroke QD,which provides a data basis for the subsequent research of this paper.The dataset includes brain MR images of patients with ischemic stroke and corresponding clinical diagnosis reports.The images are labeled by neurologists.The dataset can be used for lesion detection,classification and reports generation of ischemic stroke.(2)In order to fully explore the difference between the lesion area and the surrounding normal area and improve the effectiveness of feature extraction,this paper applies the deformable convolution mechanism to a variety of object detection networks to automatically obtain the accurate location of the stroke in the MRI image.The experimental results on the Stroke QD show that the deformable convolution mechanism improves the accuracy of lesion detection,and the VFNet with deformable convolution can effectively prevent missed detection and false detection.(3)Aiming at the location,shape,and amplitude characteristics of stroke lesions,a multi-label classification network(MD-MLC)based on multi-scale features and deformable convolution is proposed.This network can effectively solve the problem of key information errors in image reports.By combining global features and local features to form multi-scale features to meet the feature requirements of different types of labels.We continue to take advantage of deformable convolution to improve the accuracy of multi-label classification.and automatically generate the final image reports based on the logical relationship of multi-label data and conduct a detailed performance analysis.