Research On Three-dimensional Ultrasound Automatic Scanning System Of Spine Based On Convolutional Neural Network And Image Segmentation

Scoliosis is a spinal deformity disease that seriously harms the physical and mental health of adolescents.Early correction and continuous monitoring are the key to treatment.The mainly way to detect Scoliosis is X-ray,but X-rays are radiation-sensitive,which is not conducive to continuous monitoring of the disease.Three-dimensional ultrasound can obtain the three-dimensional structure of the spine by scanning the spine.It has the characteristics of no radiation,easy operation,low cost,etc,which is an alternative to X-ray.In the various scanning modes of three-dimensional ultrasound,the mechanical automatic scanning using the mechanical device to control the movement of the ultrasonic probe has the characteristics of precise positioning,full automatic and repeatability,and is suitable for the detection of scoliosis.In a three-dimensional ultrasound automatic scanning system based on mechanical automatic scanning,accurate spinal scan path planning is the key to obtaining accurate three-dimensional structure of the spine.Generally,to complete the planning of the spine scan path,we can obtain the color image information in front of the probe by using a device such as a camera,and then use the image segmentation algorithm to segment the color image to extract the target region of the spine,and finally plan the optimal scan path in the target region of the spine.With the continuous development of deep learning,the image segmentation algorithm based on convolutional neural network has better performance in feature extraction and segmentation accuracy than traditional methods.Therefore,based on convolutional neural network,the paper did the following research for the three-dimensional ultrasound automatic scanning system:(1)In the spinal scan path planning algorithm,three convolutional neural networks which are FCN,U-Net and Deeplab,were built to compare the effects of three networks on segmenting the spine region in the back view of color map taken by depth camera.At the same time,combined with the multi-view information of the depth map,the four-channel method and the dual feature extraction network structure method was used to integrate the color map and the depth map information taken by depth camera to improve the segmentation accuracy.Finally,the segmentation result is post-processed using the segmentation result determination method based on human contour and the least squares curve fitting method to obtain a more accurate spine scan curve.(2)Using the moving least squares deformation method to propose a data enhancement method for deforming the contour of the human body with the spinal line as the control curve,fitting a variety of scoliosis and increasing the diversity of the data set,thereby improving the generalization ability of the network..(3)Based on the medical ultrasound instrument Sonix RP,six-degree-of-freedom robot,depth camera Kinect,industrial computer,a three-dimensional ultrasound automatic scanning system was built.With an improved spinal scanning path planning algorithm embedded into the mechanical automatic control program,accurate scanning and imaging of the spine is achieved.