Research On Intelligent Demonstration Learning Of Humanoid Robot Arm Based On Visual Information

The emergence of demonstration learning promotes the improvement of human control mode of robots,makes robots more intelligent,and makes robots complete more complex tasks.Visual information is not only the main way of human perception of the world,but also the main source of information for human-computer interaction.It has the advantages of intuition,convenience and sufficient valuable information.Kinect somatosensory camera is inexpensive,easy to use,and can obtain very rich data information.Therefore,the research of demonstration learning based on Kinect visual information is of great significance.This paper is based on the demonstration learning of intelligent operation tasks of robots in complex environments,which includes the following four aspects:Firstly,a redundant 7 DOF inverse kinematics method based on elbow constraint is studied,and the angle of human arm joint is calculated according to the data of human skeleton points collected by Kinect.Data set for Mask R-CNN object detection and Data set for intelligent task demonstration learning based on TL-CNN-LSTM model are constructed.Secondly,Faster R-CNN with high accuracy of object detection is introduced to complete static object detection,it completes the use of a rectangular box to represent the position and size of static objects.The GMM model is used as the main detection model,and then a time-based and space-secondary GMM-Vibe model is built with the algorithm of Vibe model,which achieves more accurate performance of dynamic object detection.Based on the shortcomings of static and dynamic object detection,Mask R-CNN is introduced as a static and dynamic multiobject detection algorithm.by adapting the FPN network and the head network part of the model,more accurate multi-object detection is realized simultaneously.Thirdly,A TL-CNN-LSTM model is built based on the depth neural network.Firstly,Spatial Softmax is introduced to extract the abundant plane position information contained in the color image.Secondly,the full-connected network is used to extract the three-dimensional space position information contained in the depth image.Then,multi-information fusion is used to enhance the expressive ability and robustness of the model.Finally,transfer learning Mask R-CNN feature extraction layer reduces the scale of the model,improves the performance of the model,and completes the intelligent task demonstration learning.Finally,the proposed object detection algorithm is verified by experiments,and the feasibility of the algorithm is analyzed according to the experimental results.An experimental platform for robot intelligent task demonstration learning is built,and the TL-CNN-LSTM model is tested and the feasibility of the experimental results is analyzed.The task demonstration learning experiments under different initial states are designed to verify the adaptability of the model to the main operator and to different initial states.