Research On Robotic Arm Assisted Ultrasound Scanning Algorithm Based On Visual Information

Ultrasound examination is widely used in medicine.Doctors scan the patients by holding ultrasound probes and use ultrasound images for diagnosis.However,there are many problems with the manual operation.For example,the quality of the ultrasound image scanned is affected by the doctor’s operating experience,there is no unified scanning process,and the scanning task is heavy.These problems limit the clinical diagnostic capabilities of ultrasound scanning,so it is necessary to use the robot to complete the ultrasonic scanning autonomously.However,the current ultrasonic scanning robot needs to be manually programmed to perform the scanning task.When the environment changes,the manual programming method fails,it is difficult to meet the actual needs.This paper presents a method of robotic arm-assisted ultrasound intelligent scanning with the help of visual information,which allows the robotic arm to acquire control strategies through learning and autonomously complete abdominal ultrasound scanning.This paper divides the abdominal ultrasound scanning process into two stages: the first stage is to guide the ultrasound probe from the initial position in space to the reference point of the patient’s body;the second stage is to guide the ultrasound probe to scan the patient’s abdomen.Aiming at the difficulty in obtaining labeled data in the first stage,a research on the control strategy of the robotic arm based on reinforcement learning was carried out;aiming at the difficulty in designing the reward function in the second stage,a research on the control strategy of the robotic arm based on imitation learning was carried out.This paper mainly carried out the following research work:First,training the neural network to detect the targets of interest in the patient’s surroundings.The surrounding environment of the patient is analyzed to obtain four targets of interest.Aiming at different target sizes,a single-step target detection network structure is constructed,and network training is completed.The network is tested on the test set data,and the AP values of various targets are all above 0.99.Secondly,using reinforcement learning methods to obtain the control strategy of the robotic arm in the first stage.Combining the detected targets and the joint information of the robotic arm,the state,actions and rewards are designed;a simulation environment is built,the model is trained under the Actor-Critic algorithm architecture and the control strategy of the robotic arm is obtained;With 500 randomly generated target points in the target area are tested,the robot arm reached the target 500 times.Finally,using imitation learning methods to obtain the control strategy of the robotic arm in the second stage.Aiming at the temporal correlation between images during the scanning process,an LSTM unit is introduced,and a model is constructed in combining the convolutional neural network with the LSTM unit;the model is designed as the visual layer and the motor control layer,which map the input image to the joints angle increment of the robotic arm;for the two types of images,the scanning process is refined,the camera image is used at the beginning stage and the ultrasound image is used at the latter stage,the training of the two models is completed;two models were tested on the test set,of which the model guided by the camera image,the root mean square error of the joints angle increment of the robot arm is 0.0387(rad);the model guided by the ultrasound image,the root mean square error of the joints angle increment of the robot arm is 0.0224(rad).