Touch Gesture And Emotion Recognition Based On Deep Convolutional Neural Network

Touch plays an important role in interpersonal communication,which can not only convey the intention from the gesture information but also the emotional state of the person.In order to enhance the intelligence of human-robot interaction(HRI),it is necessary to realize the robot’s tactile intelligence.Therefore,the ability of recognizing human touch gestures and emotional states accurately for robots is a prerequisite for HRI,which is of great significance.Current research on gesture recognition and emotion recognition has problems such as low recognition accuracy and differences between subjects,and there is no public touch emotion dataset.In response to these problems,this thesis created a dataset: Touch Gesture Emotion dataset of Tianjin University(Touch GET),and focused on gestures and emotional states recognition.The main work is summarized as follows:(1)The touch gesture and emotion collection experiment was designed and the Touch GET dataset was constructed.In the touch collection experiment,the ‘Tian Tian’service robot and the electronic skin was used as the experimental hardware platform,and the experimental questionnaire and the touch collection process were designed.A total of 15 subjects’ touch data,gesture and emotion tags were collected in the Touch GET dataset,including 5765 data samples.(2)A Spatiotemporal Separable Convolutional Neural Network(SSCNN)for feature extraction of the touch data was built.The training and testing performance of different networks was compared using the Touch GET dataset,and the results showed that the gesture and emotion recognition accuracy of SSCNN proposed in this thesis was better than others.Using a user-dependent test mode,the proposed SSCNN yielded the accuracies of 90.25% and 70.84% for touch gesture recognition(ten categories)and emotion recognition(twelve categories),respectively.Using a cross-subject test mode,the proposed method yielded the accuracies of 83.44% for gestures recognition(ten categories)and 60.26% for emotions recognitions(four categories).In addition,the spatiotemporal channel adjustment factor was added to the proposed SSCNN.By adjusting the factor,under the condition of ensuring the accuracy of gesture recognition,the amount of network calculation was reduced.(3)The Deep Domain Adaptive(DDA)method was used for the task of crosssubject recognition of touch gestures and emotions.Using the DDA method based on the multi-kernel maximum mean discrepancy and the correlation alignment discrepancy,the gesture and emotion cross-subject recognition experiment was carried out on the Touch GET dataset,and the accuracy rates were increased to 86.74% and 67.43%.The results showed that using the DDA method could effectively reduce the feature difference between training data and test data,and improve cross-subject classification performance.