The Interactive System Design Of Nursing Robots Based On Vision And Speech Recognition

As the aging problem is getting worse,how to provide better care services for the elderly and patients has become an urgent problem for the country.Nursing robots provide new ideas for this problem.This paper designs a interactive system for nursing robots based on vision and speech recognition.The nursing robot can perform corresponding interactive actions according to the user’s voice commands and human pose,such as picking up,beating back,and supporting.In this paper,the human pose estimation is based on the ZED2 binocular camera,which mainly includes monocular 2D,monocular 3D and binocular 3D human pose estimation.The improved recursive pictorial structure model can recognize the 3D absolute human pose with short-distance binocular cameras;secondly,this paper establishes a multi-scenario speech recognition system,which mainly includes a wake word detection system based on Snowboy,a continuous speech recognition system based on CMU Sphinx,and an isolated words recognition system based on HMM model,which can identify three commands: "pick up","beat back" and "support";then,this paper builds a ROS-based robot interactive system,which mainly includes startup node,speech recognition node,pose estimation nodes,and path planning nodes.Also,a motion simulation is carried out for back-beating action;finally,this paper conducts experiments in reality,which mainly includes experiment platform construction,camera calibration,the evaluation experiments of the isolated words recognition system and the improved recusive pictorial structure model and the interactive experiment of the beating back action,which invalidates the validity of the interactive system.This paper innovatively applies human pose estimation and speech recognition to the nursing robot interactive system.In terms of 3D human pose estimation,this paper has made improvements to the recursive pictorial structure model and conducts a lot of experiments.Compared with the original method,the improved recursive pictorial structure model has an accuracy increase of 16.94% on the Human3.6M data set and an increase of 34.54% on the ZED2 data set,invalidating its superiority and effectiveness.The nursing robot interactive system designed in this paper can effectively care for patients according to the speech commands and human pose,which is innovative.The interactive system can not only be applied to nursing robots,but also can be applied to other robots with interactive scenes,which is with generality.The disadvantage of this paper is that it does not carry out nursing experiments for real elderly people.In the UR robotic arm experiment,only an open-loop position control experiment is carried out,while the strength of the back beating is not controlled.