Human Action Recognition Based On Robot Egocentric Perspective In Chemical Industry

According to the demand of robots for video recognition of production scenes in the chemical industry,the human action recognition algorithm oriented to robots’ egocentric perspective is studied.It can monitor human’s production operation action,remind human action errors,and prevent related chemical accidents.Usually,the video captured by the fixed position camera is from the third person perspective.Correspondingly,the video captured by the mobile robot is called robot egocentric perspective video.The motion of the camera is caused by the movement of the robot.The video captured from the egocentric perspective has the characteristics of big picture jitter,big illumination change,fast scene switching and so on.These characteristics pose a high challenge to follow-up tasks,such as action recognition.In view of these technical problems,a multi stream convolutional neural network is proposed.The model is composed of global stream convolution network,local stream convolution network and deep stream convolution network.The input of global stream is the whole video sequence frames,because there is a lot of overall information related to human motion in the background;the input of local stream is the human body target sequence frames detected by SSD method,which can effectively separate the background motion information from the human motion information,and can extract the human motion features more comprehensively;the input of the depth stream is the depth data,because the depth data is not affected by the change of illumination;In order to further enhance the temporal relationship between human action features,the paper replaces some of the convolution layers of C3 D network with Conv LSTM layers,which is the main network to extract human action features.In this paper,the attention mechanism of CBAM is improved,which can be extended from 2D convolution to 3D convolution,which can effectively improve the accuracy of multi stream convolution network in human action recognition.In order to verify the effectiveness of the proposed method,experiments are carried out on several egocentric human action video datasets,including the KTH dataset and self-made CIEA dataset.The results show that the proposed method has higher recognition accuracy than the existing methods.