Research And Application Of Visual Information Reconstruction Method For Manipulator Operation

The skill teaching of manipulator has always been one of the research hotspots in the field of artificial intelligence.Current skill teaching method mainly constructs a virtual space through three-dimensional reconstruction technology for manipulator to simulate and train.However,the current reconstruction method has complicated calculation process,large reconstruction error,high time cost and poor environmental adaptation.In addition,current reconstruction method also has some problems,such as the different size of the object constructed in the virtual space and the real world,and the different viewing angles between human and manipulator,which makes it difficult for the grasping skills in the virtual space to be transformed into the real operation of manipulator in the real world.To solve the above problems,a visual information reconstruction method for manipulator operation is proposed in this thesis.The main research contents are as follows:(1)Aiming at the problems of cumbersome calculation process and poor environmental adaptation of current three-dimensional real-time reconstruction methods,we propose a visual space and operation space mapping method DL-VOM based on deep learning.Firstly,a single RGB-D camera is used to collect visual information,and then the convolutional neural network is used to extract the features of visual information.Finally,the extracted feature values are used to calculate the three-dimensional position information of the corresponding manipulator operation space by fully connected layers.Experiments show that this method only need a single RGB-D camera and the convolutional neural network to complete the mapping,which is simple and fast,and the error is about 15 mm.(2)Aiming at the problems of large reconstruction error and time cost of current three-dimensional real-time reconstruction methods,we propose a three-dimensional real-time reconstruction method DL-3D based on DL-VOM monocular vision.Firstly,the visual information collected by a single RGB-D camera in real time is extracted through Mask R-CNN network,and then the extracted visual information is mapped by DL-VOM method.Finally,a cluster center distance limited outlier adjustment method BCC-Drop is proposed to reduce the error,and a symmetrical virtual space is built through Open GL.Experiments show that the reconstruction speed of this method can reach 16 fps,the absolute error of reconstruction is about 5.73 mm,and the relative error of reconstruction is about 5.23%.(3)Aiming at the problems that the grasping skills shown in the current virtual space are difficult to be transformed into the real-world operation of the manipulator in the real world,we propose a novel grasping strategy and establish a man-machine cooperative manipulator skill teaching system based on DL-3D.In this system,we record the state changes of virtual objects to guide the motion planning of manipulator in the real world.Experiments show that compared with the traditional system,the placement error of the system is reduced by about 1.7%,the placement angle error is reduced by about 2.1%,and the time consumed is reduced by about 3.6s.