| Pupillary size and pupillary light reflex(PLR),as part of neurological examinations,shed lights on early detection of elevated intracranial pressure(ICP)and brainstem dysfunction in acute critical patients.At present,medical staff has limited means of pupil measurements.The current most commonly used methodology to assess pupillary function is bedside manual light reflex performed by clinicians and nurses.However,the manual pupillary examination is limited,which has low accuracy and struggles to detect a subtle change.New pupillometric technologies,such as infrared pupillometer allow us to obtain objective measurements of pupils and other eye structures better.Bedside ultrasound is among one of those technologies.Nevertheless,there is still a lack of relevant research and exploration.Intracranial pressure(ICP)monitoring is another primary measurement of the neurological examination.Standard invasive and noninvasive ICP monitoring technologies include direct ventriculostomy,near-infrared spectroscopy(NIRS),and transcranial doppler ultrasound(TCD).The main downsides of direct ventriculostomy are invasive,posing risks to the patients,difficulty with placement,and high costs.On the other hand,as a non-invasive assessment,sonographic measurement of optic nerve sheath diameter(ONSD)has a linear relationship with ICP.Respiratory disorder is another common and significant theme in emergency care.Rapid diagnosis of pulmonary edema,pneumothorax,pulmonary embolism,pulmonary atelectasis,and pleural effusion,along with dynamic monitoring,is critical for targeted treatment.The current bedside assessments of pulmonary function,such as relying on a stethoscope,cannot acquire accurate results.Computed tomography(CT)scanning is thediagnostic gold standard for imaging most respiratory diseases.However,CT scanning is radioactive and expensive.Patients have to be transferred to the CT room,which is not feasible for repeated assessment due to transportation risk.Lung ultrasound is an effective measurement for lung injury in critical care.Lung ultrasound score(LUS)has been successfully used in different clinical contexts.It can be extremely helpful for semiquantitative assessment of pulmonary ventilation,dynamic monitoring of lung injury,and weaning from mechanical ventilators.Therefore,bedside ultrasound plays an important role in the evaluation of acute and severe respiratory function.Even though bedside ultrasound is easy to operate,convenient for transportation,real-time imaging,and free of radiation,its massive application still relies on doctors’ skills of scanning and reading.Restricted by both the training courses and time,clinicians have different proficiency levels in ultrasonography,which leads to a deteriorating accuracy and reliability or even misdiagnosis.In the last decade,with the continuous development of artificial intelligence(AI)in the medical field,AI becomes the trending topic in ultrasonographic applications.Machine learning has significantly increased the consistency in diagnosis while saving physicians’ time from reading the result.Most artificial intelligence technologies are currently adopted in the ultrasonography department,yet few mature ones in emergency and critical ultrasound.In this research,the author and partners preliminarily explored AI’s application in bedside ultrasonography of neurological and respiratory systems;established automated measurement technology of pupillary and optic nerve sheath based on bedside ultrasound;achieved intelligent recognition and automated quantification technology of LUS.This study will lay a scientific ground for the future development of AI applications in the bedside ultrasound.Part 1 Ultrasonic evaluation technology of pupil size and light reflection based on deep learning modelObjective:The aim is to establish an ultrasonic evaluation technology of pupil size and light reflection based on a deep learning model.Method:From June 2017 to December 2018,emergency physicians used bedside ultrasound to record the scanning video of the pupil size and light reflection of the patients in the emergency intensive care unit of the Second Affiliated Hospital of Zhejiang University School of Medicine.Aiming at the demand of ultrasonic pupil measurement,a new ocular ultrasound probe with light sources was designed by using an 8-12 MHz linear array probe.Combined with the RIBON AX8 Supper portable ultrasound device,the light stimulation during ultrasonic pupil detection and real-time recording of light reflection were realized.Two rows of light-emitting diodes(LED)were added to an 8-12 MHz linear-array transducer to provide light stimulation while recording signs of PLR simultaneously.A fixed physician with ultrasound proficiency used the ultrasound machine described above to obtain 500 ultrasound videos in patients’ PLR examination in the experiment.The physician captured 8366 images from the video,including 6317,1049,and 2000 images for training,validation,and test datasets,respectively.U-Net,a classical deep-learning segmentation model,was chosen to identify and segment the pupil area.A residual unit was added to the network.The trained network was downsized by quantification and pruning.In the training process,a focal loss function was added to the traditional Binary cross-entropy loss function to improve model accuracy on segmenting challenging pieces by increasing weights of the challenging samples.Data normalization was performed to enhance the generalization ability and robustness of the model.Pupil diameter was retrieved via ellipse fitting upon a binary graph form the binarized model’s predicted result.Features above integrated on a bedside ultrasound machine and accomplished realtime frame-by-frame automated pupil recognition,diameter measuring,and PLR evaluation.Result:1.Successfully designed a new ultrasound transducer with LED light sources.Automated illuminance and duration control of the LEDs enabled the PLR examination without using a flashlight.2.Established a preliminary automated ultrasound pupil measurement technologybased upon UNet,with 0.9991±0.0009 accuracy,0.9550±0.0585 precision,0.9398±0.0633 recall,and 0.9443±0.03755 F1-score.Specialized the initial model according to sample distribution and demands of PLR examination,which achieved 0.9991±0.0007 accuracy,0.9484±0.0650 precision,0.9530±0.0537 recall,and 0.9479±0.03721 F1-score.The optimized model is superior to the classical U-Net model.3.The optimized model was successively integrated into a bedside ultrasound machine.The system simply requires a physician to probe the patient’s orbit with the transducer,trigger light stimulus when the pupil is observed,and acquire an ultrasound video including the pupil.A computer then automatically calculates the pupil size reaction for light stimulation and plot the pupil diameter over time.In the meantime,it displays the min/max pupil diameters,average constriction velocity,maximum constriction velocity,average dilation velocity.Conclusion:This research established an ultrasonic evaluation technology of pupil size and light reflection based on deep learning model successfully.Part 2 The ultrasonic automatic recognition and measurement technology of optic nerve sheath diameter based on deep learning modelObjective:The aim is to establish an ultrasonic automatic recognition and measurement technology of optic nerve sheath diameter(ONSD)based on deep learning model and provide an objective,simple,non-invasive,and fast intracranial pressure monitoring program for the clinic.Methods:From June 2018 to December 2019,emergency physicians used bedside ultrasound to record the scanning video of the optic nerve sheath of patients in both eyes in the emergency intensive care unit of the Second Affiliated Hospital of Zhejiang University School of Medicine.Fixed ultrasound-trained physicians used the RIBON AX8 Supper portable ultrasound device with the new ocular ultrasound probe developed in Part 1 of the study for scanning.The scanning video of optic nerve sheaths was obtained by placing the probe laterally at the closed upper eyelid and rotating it successively along the axial and equal angles.A total of 230 cases of optic nerve sheath scanning video were collected.Engineers captured 7050 images with optic nerve sheath from the video pool,including 4775,1525,and 750 images for training,validation,and test datasets,respectively.The algorithm used the SD-UNet model – a variant of the U-Net model.A block attention module(BAM)was introduced to the model to make the model study featured areas and to improve the generalization ability together with convergence speed.Other than that,the model was compressed and pruned to reduce its internal parameters and shorten the inference time.The training process used the Focal+Dice loss function to increase the weight of the foreground areas and the loss on difficult samples.The validation performed binarization on predicted results and quantitatively compared them with doctors’ annotations.The functions above helped locate the best image slice in the scanned video rapidly,allowing the system to automatically measure the ONSD following the standard and label the result on the image.Results:1.Preliminarily established the technology of automated ONSD measurement via ultrasound.This model automatically selects the best frame for measuring the ONSD from the video collected in the earlier stage scanning from different angles.2.The initial model based on SD-UNet automatically picked the best frame for ONSD measuring.The preliminary model’s Dice coefficient was 0.8723±0.1814,with a 0.8448±0.1485 sensitivity,0.9917±0.0075 specificity,0.8647±0.0875 with F1 score,and achieved a 45 frame-per-second(FPS)video transmission rate.The optimized model hada Dice coefficient of 0.8817±0.1708,sensitivity of 0.9052±0.075,specificity of 0.9893±0.0074,F1 score of 0.8761±0.0631 and video frame rate of 34 FPS.3.Software engineers successfully integrated the model into the bedside ultrasound machine.The machine automatically identified the best measurement frame in the scan video and marked the optic nerve sheath diameter 3mm behind the eyeball.4.The prototype also supports manual image frame selection.By clicking the ‘Measure’ button on the user interface,the model automatically measures and marks the ONSD on the selected frame.Conclusion:This research established an ultrasonic automatic recognition and measurement technology of optic nerve sheath diameter based on deep learning model successfully and provided a simple,objective,non-invasive,and fast intracranial pressure monitoring program for the clinic.Part 3 Automatic measurement technology for lung ultrasound score in acute and critical patients based on deep learning modelObjective:The aim is to develop an automatic measurement technology for lung ultrasound score in acute and critical patients based on deep learning model,to realize intelligent recognition and automatic quantitative evaluation of lung injury.Methods:From January 2019 to December 2019,emergency physicians used bedside ultrasound to conduct pulmonary examination in the emergency intensive care unit of the Second Affiliated Hospital of Zhejiang University School of Medicine.Fixed ultrasound-trained physicians used Mindray M9 portable color ultrasound machine to perform ultrasonic examinations and data collection following the 12-zone ultrasound protocol.A team of experts then subdivided the traditional ultrasound score system for lung injury(0-3 points)and formed a new elaborate scoring standard(0-7 points).The team obtained 5610 lung ultrasound images for all different scores.According to both traditional and new standards,another experienced physician scored the ultrasound images and sorted them into corresponding subfolders.The final training and test dataset contained 4489 and 1121 images,respectively.Res Net-50 was chosen as the fundamental model for establishing the automated lung ultrasound scoring technology.Besides,to enable real-time application on portable ultrasound machines,different lightweight models are generated through transfer learning.Meanwhile,rebalancing branches are introduced to solve the problem of uneven sample distributions.Knowledge distillation was accomplished to improve the accuracy of lightweight models.Continuous model training,verification,and testing were carried out to finally realize the automatic scoring of lung injury based on traditional and new standards.This research then compared the performance of different lightweight models under the two scoring criteria.Results:1.Preliminarily built AI-assisted lung ultrasound diagnosis model based on Res Net50.The diagnostic accuracy was 93.75%,the precision were 0.94,0.77,1.00,0.99,respectively for the classical standard(0-3 points);The diagnostic accuracy was 94.28%,the precision were 1.00,0.97,0.83,0.88,0.83,1.00,0.67,0.98,respectively for the new standard(0-7 points).2.Generated various lightweight models from ResNet50 model via transfer learning for the traditional standard(0-3 points).The result is as follow: The accuracy of the Mobilenet V2 model was 93.19%;the accuracy of the Mobilenet V3-small model was 92.30%;the p accuracy of the Mobilenet V3-large model was 92.56%;the accuracy of the Shufflenet V2 model was 91.34%.The Mobilenet V2 model had the best performance in this study.Generated various lightweight models from Res Net50 model via transfer learning for the new standard(0-7 points).The result is as follow: The accuracy of the Mobilenet V2 model was 90.09%;the accuracy of the Mobilenet V3-small model was92.30%;the p accuracy of the Mobilenet V3-large model was 91.38%;the accuracy of the Shufflenet V2 model was 91.34%.3.After further optimization,the accuracy of RESNET 50 model for classical lung ultrasound score was 93.75%,the classification accuracy of 0-3 score was 0.94,0.82,0.93,1.00,the accuracy of new refined lung ultrasound score was 93.22%,the classification accuracy of 0-7 score was 0.95,0.84,0.81,0.93,0.94,0.95,0.90,0.89;The accuracy of Mobile Net V2 model was 92.26% for classical lung ultrasound score,and the classification accuracy of 0-3 score was 0.96,0.82,0.95,0.97.The accuracy of Mobile Net V2 model for new refined lung ultrasound score was 91.77%,and the classification accuracy of 0-7 score was 0.95,0.91.0.80,0.80,0.94,0.94,0.89,0.94.4.The optimized Mobile Net V2 model was integrated into the software platform to achieve rapid and automatic scoring of real-time bedside lung ultrasound images.Conclusion:This research developed an automatic measurement technology for lung ultrasound score in acute and critical patients based on deep learning model successfully,and realized intelligent recognition and automatic quantitative evaluation of lung injury. |
PDF Full Text Request