Research On Deep Learning Based Spontaneous Facial Expression Recognition Of Infant

Infant spontaneous expression refers to the natural facial expression of infants,which can convey intention and emotional information,and has been widely concerned by psychology,education and clinical medicine.Infant expression recognition technology has broad application prospects in smart home,health care,early education and so on.With the rise and rapid development of artificial intelligence,facial expression recognition based on adults has achieved remarkable results,while few studies focus on infant expression.The following challenges exist in the research of infant spontaneous expression recognition: infant facial skin is smooth,wrinkled texture is light,hair color is light and expression is weak,which makes expression features difficult to identify;infant spontaneous expression accompanied by head and limb movement,causing a large range of face angle and face occlusion;in addition,the research is also restricted by the lack of public data.At present,the research on infant facial expression recognition in natural scene is not mature.Focusing on the goal of infant spontaneous expression recognition,the following innovative work has been carried out in this paper:(1)a static infant expression database and a dynamic infant expression database were established.In order to solve the problem of lack of research data,this paper,based on the static face test paradigm,induced infant spontaneous expression in the parent-child interaction situation,and processed and annotated the facial expression data of 30 infant subjects.Based on this,a static infant expression database composed of 9,133 images and a dynamic infant expression database composed of 4,560 image sequences are established,which provide data support for this study.(2)A convolutional network for infant expression recognition is proposed,which combines face region adaptive learning and island loss strategy.The contribution of facial regions to facial expressions is different.An adaptive learning module based on attention mechanism is introduced to enable the network to automatically focus on and emphasize the features of infant facial regions with more discriminative expressions,and weaken the regions with low expression contribution.The strategy of fusion with isolation loss is used to maximize the distance between classes and minimize the intra class variance,which further focuses on the expression discrimination task and suppresses the interference of expression independent information.This method has small parameter scale,excellent recognition performance for infant's spontaneous expression,and is robust to face occlusion,large head posture and illumination changes.(3)A framework of dynamic infant expression recognition network based on multi cue fusion is proposed.Aiming at the complex problem of dynamic infant expression recognition in real scene,this paper simulates the processing method of human multi-level information comprehensive recognition,and determines three clues closely related to expression recognition.Facial key point motion feature modeling is used to extract structural features of facial flexible deformation caused by expression.The face region adaptive learning module is introduced into the temporal modeling of facial texture features,which is used to process the key texture features and timing dependence of infant expression.The joint modeling of local dynamic features focuses on the local features of five facial muscle groups.In order to make full use of the above geometric and texture,local and global expression information,the best weight combination of mesh search multi cue model fusion is used to make the cues complement each other and judge together,which effectively improves the performance of the network for dynamic infant spontaneous expression recognition.(4)An infant behavior analysis system based on expression recognition is developed.Combined with the needs of practical application,the method research is applied to the monitoring and analysis of infant expression behavior.The system can real-time detect and display the infant's facial position,expression status,statistical duration and proportion of various types of expression;after monitoring,it generates expression change map and summary report,and stores relevant data by classification for further evaluation by parents and researchers.The system has good recognition efficiency and accuracy,vivid and intuitive visual form,convenient user experience,and meets certain application requirements in infant monitoring,emotional ability clinical evaluation and other aspects,and has certain social value.