Research On Robot Target Recognition And Slip Detection Based On Touch Vision Fusion

The home service robot senses the external information by carrying multi-modal sensing equipment,and then completes the humanized service tasks such as integrated human-computer interaction and tool operation combined with decision-making.This Dissertation focuses on the problem of target recognition and slip detection in the process of tool operation of home service robot.Based on the tactile and visual modal information in the process of robot grasping operation,and combined with the deep learning theory,an efficient robot target recognition and slip detection framework is constructed to improve the perception ability of robot based on touch visual information.The specific research contents are as follows:Firstly,aiming at the problem of low accuracy and accuracy of service robot recognizing household objects based on tactile modal features,a target recognition method based on spiking graph residual convolution neural network SNN-Atten-Res GCN based on tactile modal features is proposed.Firstly,the representation information of tactile time series is trained by using the graph residual network Res GCN model.Secondly,the attention mechanism in the deep learning model is introduced to fit the local features of the graphic structure of tactile data.Then,the target recognition features are obtained by fitting the reconstructed tactile graphics through SNN spiking neural network.Finally,the network features are decoded by Vote voting layer to detect the category of household goods.Comparative experiments on household goods container data sets verify the effectiveness of SNN-Atten-Res GCN algorithm.Secondly,aiming at the problem of low detection accuracy and accuracy of the deep learning method using tactile modal features in the slip detection task,a CNN-MogrifierBIGRU target slip detection algorithm based on tactile modal features is proposed.Firstly,the tactile time series data features in the process of target capture are extracted through the pre training network CNN,and then the current time input and the previous time hidden layer output are fitted through the Mogrifier coupling module.Finally,the bidirectional gated cyclic unit network BIGRU is used to compare the feature sequence and make a decision to judge whether the family object slides.Comparative experiments on household goods container data sets verify the effectiveness of CNN-Mogrifier-BIGRU.Finally,aiming at the problem of the first mock exam accuracy of target slip,a novel multi target collision detection algorithm based on hybrid multi-head-self-attention mechanism MMSA is proposed.Firstly,the pre training network CNN is used to extract the information of visual feature map and tactile feature map respectively.Secondly,the visual feature map and tactile feature map are encoded into preliminary tactile visual fusion features according to the position.Then,the hybrid multi head self attention mechanism is introduced to fit the internal autocorrelation of preliminary tactile visual fusion features to generate tactile visual fusion feature sequences.Finally,the tactile visual fusion feature sequences are compared by Mogrifier-BIGRU to generate slip detection tags.A comparative experiment is carried out on the visual tactile data set of household goods to verify the effectiveness of MMSA algorithm.