Research And Application Of Key Technologies For Fetal Ultrasound Congenital Heart Disease Assisted Diagnosis

The morbidity and mortality of fetal congenital heart disease(CHD)in China have been the highest in Asia over the past 12 years and have become the leading cause of death in fetuses and children.Ultrasound techniques have become the most powerful tool for the diagnosis of CHD due to their non-destructive,safe,convenient,and fast imaging.In clinical practice,the diagnosis of fetal CHD involves a procedure of standard view recognition,standard cardiac cycle localization,cardiac view segmentation and measurement,and diagnosis.In addition,quality control of cardiac views significantly influences CHD detection.However,currently,these procedures depend on the subjective experience of the ultrasonographer heavily,which is time-consuming and has a low level of homogeneity.Furthermore,with the opening of the third-child policy,the number of elderly pregnant women and newborns is likely to increase,increasing the workload of obstetricians.To this end,this thesis proposes to explore key techniques for the diagnosis of fetal CHD.Specifically,it involves the automatic recognition of the standard view and the recognition of the cardiac cycle(CC).On this basis,we also propose the localization of standard cardiac cycles(SCC),the automatic segmentation of the four-chamber view,and quality control of the cardiac view.The main research tasks and contributions are as follows:(1)Research on the fetal standard section recognition task.We decompose the standard view recognition task into two subtasks: class the type of view and judge whether it is standard or not.And we propose a multi-task learning fetal ultrasound standard plane recognition model based on the spatial-temporal characteristics of the video stream,called FUSPR.The FUSPR model consists of a convolutional neural network(CNN)component and a recurrent neural network(RNN)module,which learns the spatial and temporal features of the ultrasound video stream in a multi-task way.The CNN component recognizes key anatomical structures of the fetus from each frame of the video and extracts spatial features of standard slices of the fetus.The RNN component obtains temporal information between adjacent frames,enabling accurate localization of fetal organs across frames.In addition,we introduce two feature fusion strategies into the FUSPR model,i.e.,CNN fusion and RNN fusion,to fit the spatial sequence and motion representation in the ultrasound video stream,thereby effectively improving the accuracy and robustness of the model.Extensive experiments are conducted on more than 1000 ultrasound videos,and experiment results show that the FUSPR model is superior to the competing baselines in terms of accuracy and performance.(2)Research on the fetal CC task.Unlike previous regression approaches to identify adult CC,we define this problem as a classification task and propose a hybrid classification algorithm to localize end-systolic(ES)and end-diastolic(ED)frames.The proposed architecture integrates the extracting region-of-interest(ROI)component based on object detection,retaining a temporal dependency module and classification module based on a domain transferred deep CNN.To the best of our knowledge,this is the first application of a classification algorithm for detecting fetal ES and ED frames.The algorithm integrates a region of interest localization component,a temporal information fusion module,and a deep transferred CNN classification module.The region of interest positioning module determines the location of the four-chamber view and filters out unrelated views.The temporal information module integrates the time difference information into the image channel to extract short-term information,and then the transferred CNN module extracts deep-level high-level features for final classification.Besides,we also explore the combination of different CNN backbones and temporal information fusion strategies.(3)Research on the fetal SCCs task.Previous studies mainly focused on the detection of CC,and the detection results contained non-standard frames,which may have a negative effect on the diagnosis of CHD.In the fetal ultrasound examination,automatic recognition of standard cardiac cycles(SCCs)not only accurately recognizes ES and ED frames but ensures that every frame in the cycle is a standard view.To this end,we propose an end-toend hybrid neural network based on an object detector to detect SCCs from fetal ultrasound videos efficiently.The proposed model consists of 3 modules,namely,Anatomical Structure Detection(ASD)module for filtering out background frames and irrelevant planes,Cardiac Cycle Localization(CCL)module for recognizing ES and ED frames,and Standard Plane Recognition(SPR)module for identifying the standardization.The ASD module identifies9 key anatomical structures,3 cardiac motion phases,and the corresponding confidence scores from fetal ultrasound videos.On this basis,we propose a joint probability method in the CCL module to find the cardiac motion cycle based on the 3 cardiac motion phases.SPR module can reduce the impact of individual structure detection errors on the overall accuracy of the standard plane recognition.The proposed method is evaluated on test fetal ultrasound video datasets and clinical examination cases,and the experiment results demonstrated that our method outperforms the state-of-the-art.(4)Research on the semantic segmentation task of multiple anatomical structures in the fetal four-chamber view.Due to the ultrasound imaging arising from artefacts and scattered noise,the variability of anatomical structures in different gestational weeks,and the discontinuity of anatomical structure boundaries,accurately segmenting the fetal heart organ in the apical four-chamber(A4C)view is a very challenging task.To this end,we propose a novel semantic segmentation model called Mobile UNet-FPN.The Mobile UNetFPN model extracts the high-level features of different stages of Mobile Net as encoders to process blurred ultrasound images and constructs a feature pyramid network in the semantic segmentation model to segment 13 key anatomical structures in the A4 C view.Extensive experiments were performed on fetal A4 C and femoral length views,and the experiment results show that our method achieved superior performance compared with the 9 advanced baseline methods.(5)Research on the quality control task of fetal cardiac views.There is a lot of anatomical knowledge in fetal ultrasound images,such as the fixed positional relationship of different anatomical structures.However,this prior anatomical knowledge is not well fused and extracted in current models.To this end,we propose a novel semantic graph knowledge reasoning model,i.e.,SGRDM,for quality control of four heart views.SGRDM is mainly composed of the graph knowledge constructing module(GKCM)and relational knowledge reasoning module(RKRM).GKCM constructs graph-structure relationships of anatomical structures in positive and negative sample generation stages based on the one-stage detector.RKRM is to fuse visual features and relational knowledge and perform reasoning and exploration of spatial graph knowledge.Extensive experiments are performed on four fetal cardiac views,and the results show that our method outperforms state-of-the-art detectors compared with 9 competitive detection methods.Finally,fetal cardiac cycle localization software,standard cardiac cycle detection software,four-chamber view semantic segmentation software,and quality control software are developed for application in hospitals.