Similar And Dissimilar Representations Between The Human Brain And The Deep Convolutional Neural Network In Face Perception

The human brain has the ability to encode different facial information in a very short time,which can provide support for various cognitive functions and social behaviors.On the other hand,the deep convolutional neural network(DCNN)can achieve the level of human performances in many tasks.However,its encoding mechanism is highly different with the biological brain.Our study broke through the bottleneck of traditional research.We attempted to directly compare the internal coding mechanisms between the human brain and the DCNN model and explored the similar and dissimilar representations between both.Firstly,we designed and implemented two toolboxes,NeuroRA and PyCTRSA,which could be widely used in neuroscience domain based on Python.The former can be used to conduct representational analysis cross multi-modal neural data,while the latter can be used to conduct the cross-temporal representational similar analysis for electroencephalography(EEG)and magnetoencephalography(MEG).Secondly,we used neural decoding and representational similarity analysis(RSA)methods to analyze the temporal dynamic coding process of the human brain during face perception based on EEG.Our results showed that the human brain processes visual information earlier,and then processes more complex high-order facial information.Also,this study was the first study to directly separate the temporal coding differences between familiar faces and unfamiliar faces.Combined with the results of neural decoding and RSA,it was found that face repetition effect began to appear from about 300 ms,and this effect was stronger for familiar faces.Thirdly,we further explored the hierarchical representation of different facial information in the DCNN model during face perception based on RSA.The results showed that there were significant differences between pretrained VGG-Face model and nontrained VGG-16 model with random weights in the representation of facial information.With the full connected layer,VGG-Face model had a degree of within-type representational similarity for three types of faces(familiar faces,unfamiliar faces and scrambled faces),but randomly-weighted VGG-16 model hardly had this kind of representation.Surprisingly,we found that the DCNN model didn't specifically encode familiar faces and integral face information.However,for other facial information,VGG-Face model showed layer-by-layer representation enhancement,while randomly-weighted VGG-16 model showed layer-by-layer representation weakening.Finally,we constructed two popular repetition suppression(RS)models,and based on the model hypothesis,we modified the activations of the DCNN model.Then we conducted crossmodal RSA between modified DCNN models and the human brain.The results showed that the RS effect in face recognition was more likely caused by the fatigue mechanism,that was,the activation of neurons with stronger response to face stimulus was attenuated.In addition,the DCNN model had a certain hierarchical representation similarity with the human brain during face perception.In summary,our research achievements have provided many convenient toolkits for data analysis.Through the results of internal representations in the human brain and the DCNN model in the process of face perception,as well as the results of similar and dissimilar representations between them in our study,it not only suggests a new direction to explore the neural mechanism of face perception in cognitive neuroscience domain in the future,but also promote the establishment of the new model for face recognition in brain-inspired intelligence domain.