Intelligent Detection Of Scoliosis Based On Anatomical Landmarks Of 3D Ultrasound Images

Scoliosis is a common 3D deformation of the spine,among which adolescent idiopathic scoliosis is the most common type of scoliosis,leading to great threats to the health of teenagers.At present,the clinical gold standard for diagnosis is the Cobb angle based on X-ray images.However,frequent and long-term exposure to X-ray radiation can be harmful to the health of adolescents and even increase the risk of cancer.Therefore,in order to avoid radiation damage,non-invasive and radiationfree 3D ultrasound systems have been widely used in scoliosis diagnosis in recent years.Even though many automatic detection algorithms have been developed,existing methods are often designed based on two-dimensional projection images.As a result,they cannot obtain three-dimensional information of the spine to comprehensively analyze the 3D deformity of scoliosis,causing great limitations in clinical use.On the other hand,the key to achieving 3D assessment of scoliosis is to obtain the 3D spatial positions of anatomical landmarks on a spine.Even though artificial intelligence algorithms for landmark detection tasks have obtained promising results in various medical applications,accurate landmark localization in ultrasound spine images with an ambiguous structure and a poor signal-tonoise ratio is still a great challenge and related research is lacking.Therefore,to accomplish the 3D diagnosis of scoliosis,two popular artificial intelligence algorithms for landmark detection are modified in this paper to make them more suitable for the localization of landmarks in spine ultrasound images.A complete procedure of the generation of 3D spine curve and the assessment of scoliosis is further proposed based on the position of the detected landmarks.Also,a freehand 3D ultrasound system is developed and verified based on the proposed detection algorithms,with the ability of analysis and visualization of 3D spine curve.The main work of this paper is how to design artificial intelligence algorithms to automatically detect the landmarks of a spine and develop a complete procedure for scoliosis assessment.Specifically,it includes:(1)In this paper,a novel reinforcement-learning-based landmark localization algorithm is proposed to detect spinous processes in spinal ultrasound images.On one hand,the FPN(Feature Pyramid Networks)structure is used to replace the "convolution-pooling" structure,which is commonly used in existing agent networks,enhancing the agent’s sensitivity to positional information and representation capacity.On the other hand,a similarity prediction mechanism is advanced.In this way,the agent network has the perception of the target area and is able to select the predicted point locations with obvious spinous process structures.(2)In this study,a novel heatmap regression network is designed to achieve accurate detection of spinous processes.This method notices the important influence of the Gaussian standard deviation for heatmap generation on the network performance,and uses a reinforcement learning optimization framework to automatically search for suitable parameter values,which significantly improves the localization accuracy and decreases the maximum error.At the same time,a similarity estimation branch is added on top of the heatmap regression network to realize the similarity estimation between the area around the predicted position and that around the target position.(3)Combining the predicted positions of spinous processes and similarity estimation values obtained by the landmark detection algorithm mentioned above with the 3D position information of the image obtained by the electromagnetic locator,an automatic procedure is carefully designed to generate three-dimensional spine curve and measure spine curvature angles,including similaritybased adaptive threshold for landmark selection,removal of confusing points caused by transverse processes,removal of outliers,curve fitting and inflection point detection,visualization of 3D models of the spine,etc.In the quantitative analysis of the coronal plane,the 3D curves obtained by both landmark detection algorithms achieve highly correlated and consistent results with the gold standard Cobb angle and the manually annotated curvature angles on the basis of coronal projection images.In the qualitative comparison with the coronal plane and the sagittal plane,the plausibility of the 3D curves are further verified.(4)On the basis of the landmark detection algorithms and scoliosis assessment procedure,a freehand 3D ultrasound system for spine imaging and scoliosis assessment is developed in this study.The system possesses functions of data acquisition and storage,scoliosis assessment and visualization,3D reconstruction and visualization,etc.It can provide clinicians with more comprehensive and intuitive information to assist diagnosis,and has great clinical significance and application potential.Overall,compared with existing scoliosis detection methods,the method proposed in this paper has two significant advantages.First,by leveraging artificial intelligence landmark detection algorithms,spinous processes in the spine ultrasound images are localized with high accuracy.Second,a complete scoliosis assessment procedure is proposed based on the landmark detection results.In this way,a relatively accurate 3D curve of the spinal spinous process is obtained,which can provide physicians with more comprehensive 3D deformation information of the spine for diagnosis,which has important clinical meaning and promising prospects.