Research On Spontaneous Learning Facial Expression Recognition Based On RealSense

Emotion plays an important role in people's life and can directly affect people's thinking,memory,creativity and behavior.The emotional state of the students in the classroom teaching environment will affect their cognitive activities.Positive emotions will promote learning activities,otherwise it reduce learning efficiency.In the traditional classroom,teachers face a large number of students,and it is difficult to timely perceive the learning emotional state of each student,which may go against the harmonious emotional interaction between teachers and students.The development of artificial intelligence technology has promoted the in-depth study of intelligent learning.The accurate identification of students' emotions in the intelligent learning environment is an important means to help teachers timely control students' emotions and optimize teaching.Facial expression is the main way of expression of emotion,and it is easier to obtain in the learning environment.Therefore,students' learning emotions in the smart learning environment are mostly judged by facial expressions.By summarizing relevant researches,it is found that the exploration of learning emotion started relatively late and developed rapidly in recent years.However,problems like the following still exist:First,different studies focus on different types of learning emotions and fail to provide a reasonable basis for classification.Secondly,there are few databases related to learning expressions,which makes it difficult to support study of algorithms.Third,most of the learning expressions are subtle and have a small degree of discrimination.There is a lack of feature extraction method for learning expressions,and the recognition rate is low,which worse for their application in the actual learning environment.Based on the above problems,this paper constructs learning expression database and proposes a fusion feature extraction method based on deep learning.First of all,based on the mechanism,educational function and types of learning emotion,the paper proposes five types of learning emotions that are common and have key educational functions:surprise,confusion,pleasure,tiredness and neutrality.Combined with the facial motion coding system,the key features of each expression were summarized.Secondly,a database of spontaneous learning emotion was constructed,including five learning emotion.A strict database recording standard was established.Intel RealSense SR300 camera was used to capture the natural emotion of the subjects in the learning state.And emotions were labeled by psychologists and subjects respectively to ensure the accuracy of expression labeling.Thirdly,a feature extraction method based on deep network feature,shallow texture feature and local geometry feature is proposed.A seven-layer convolutional neural network(CNN)was constructed for deep feature extraction.A complete local binary model(CLBP)was used to extract the shallow texture features.Four geometric features are defined in the eyebrows,eyes and mouth.And the fused features of the three are taken as the final facial feature data.The learning expression lasts for a short time and the facial changes are small.It is difficult to achieve a good recognition effect by using only one feature.Through the feature fusion of deep and shallow layer,local and global layer,the feature expressiveness and robustness are effectively increased.Finally,the validity of the proposed algorithm is verified through experiments.The method proposed in this paper is tested in spontaneous expression face database,Chinese Facial Affective Picture System and CK+database.Recognition accuracy rate reached 95.6%,87.6%and 96.3%.Comparing the recognition rate of single feature and fusion feature respectively,the results show that the feature fusion method in this paper is better than that of single feature,and able to effectively identify the learning expressions and basic expressions of Chinese students.By comparing the method in this paper with the experimental results of relevant studies,it is proved that the method in this paper can improve the recognition rate of basic expressions.This study lays a technical foundation for realizing the analysis of students' learning emotions and promoting the occurrence of intelligent learning in the classroom learning environment.