Research On Key Technologies Of Human-machine Cooperative Medical Assistant Robot System Based On Gesture Recognition

The outbreak of Corona Virus Disease 2019 has brought huge challenges to the global public health system.Medical staff are at great risk of being infected in all aspects of contact with patients.The heavy workload makes the medical staff work day and night in the front line of anti epidemic.Therefore,we urgently need a human-machine collaborative medical assistant robot system that can work remotely to reduce the work intensity of medical staff and the risk of cross-infection.This paper is committed to realizing a natural,convenient and efficient way of human-machine interaction.It conducts in-depth research on gesture feature extraction,dynamic gesture classification,and closed solution optimization problem of manipulator inverse kinematics.A set of human-machine collaborative medical assistant robot system based on gesture recognition is built.The specific work content is as follows:1.A gesture feature extraction method based on multi-sensor data fusion is proposed.First,in order to ensure the integrity and accuracy of the acquired gesture data,the Leap Motion and Kinect sensors are used to simultaneously collect data,and initially extract gesture feature data such as fingertips and palms.Secondly,the improved iterative closest point(ICP)algorithm is used for data registration,and Kalman filtering is used to fuse the registered gesture feature data.The experimental results show that the motion trajectory after fusion is smoother,and the fusion value will be biased towards the observation value with higher reliability,even if the data collected by a single sensor is wrong,the system can still operate normally.Finally,considering the characteristics of the human-machine collaboration system designed in this paper,a dynamic gesture database is established by using the fused gesture feature data.2.A dynamic gesture classification algorithm based on BLSTM-BLS is proposed.Combining the bidirectional long short-term memory neural network(BLSTM)suitable for processing long-term series data and the broad learning system(BLS)with incremental learning as the core,the combined classification model BLSTM-BLS is used for dynamic gesture recognition.After experimental verification,the recognition effect of BLSTM-BLS on ten gestures is significantly better than that of LSTM,and the overall recognition rate can reach 98.84%.Compared with other classification models,the recognition accuracy of BLSTM-BLS is 4.16% and 0.92% higher than that of LSTM and BLSTM respectively,and the training efficiency of BLSTM-BLS is higher than other combined classification models.3.A closed solution optimization method of redundant manipulator inverse kinematics based on NSAOA algorithm is proposed.Firstly,the kinematics of the WAM manipulator and the Power Bot car in the human-machine cooperation system are analyzed.Considering that the inverse kinematics solution of redundant manipulator with link offset structure is complex,the closed solution method is used to solve the inverse kinematics.Secondly,aiming at the problems of poor global search ability,easy to fall into local optimum and low convergence accuracy of Archimedes optimization algorithm(AOA),dynamic density factor,optimal neighborhood perturbation strategy and stagnant perturbation strategy based on Levy flight are introduced,and an improved Archimedes optimization algorithm NSAOA is proposed.Finally,NSAOA is used to optimize the inverse kinematics closed solution.After experimental comparison,NSAOA algorithm performs the best in the average error of the components on each axis,and provides the optimal solution with the smallest change anterior and posterior joint angles,and its convergence speed is second only to AOA,so the optimal solution of NSAOA helps to improve the accuracy,stability and real-time performance of the system.4.A set of Human-machine Collaborative Medical Assistant Robot(HCMAR)system based on gesture recognition is built.Firstly,based on the research on gesture feature extraction,dynamic gesture classification,and closed solution optimization problem of manipulator inverse kinematics,the HCMAR entity system is built by using the WAM manipulator and the Power Bot car,and the overall architecture and working principle of the entity system are given.Secondly,the entity system was tested,the manipulator and the car were controlled through gestures,and the tasks of manipulating the robot for distribution and temperature measurement were simply simulated.Finally,in order to make up for the shortcomings of the lack of visual feedback of the entity system and avoid unnecessary losses caused by frequent use of entity robots for training,a set of HCMAR simulation system was built using 3D Max and Unity,and the functions of disinfection and distribution are tested in the scene of the simulation system.The robot system has good isolation and stable remote working ability,which can reduce the workload and work pressure of medical staff,reduce their working time of medical staff in high-risk wards,and thus reduce the infection risk of medical staff.