Re-identification Algorithm For Pedestrians Cloth-Changing Via Multimodal And Fine-grained Semantics

Person Re-identification(Person Re-identification,Re ID)has received widespread attention in recent years.Its purpose is to retrieve the same pedestrian in a cross-camera surveillance image database to overcome the field of view limitation of a fixed camera perspective.Most of the current related research is based on the assumption that the appearance of pedestrians’ clothing remains unchanged in a short period,but in real applications,it is very normal and easy for people to change or increase or decrease clothing.Therefore,pedestrian re-identification can be roughly divided into two types: pedestrian re-identification based on unchanged clothing and pedestrian re-identification based on clothing change.Although there have been breakthroughs in pedestrian re-identification based on clothing invariance when these algorithms are directly applied to pedestrians changing clothes,the effect will often drop sharply,and the existing methods for changing clothes are general.The performance is poor,and the information used is not sufficient.Therefore,to solve the above problems,this paper focuses on how to learn more human body information that has nothing to do with the appearance of clothing and is not limited to learning human body information on images.Therefore,a multi-modal and fine-grained semanticsbased re-identification of people who change clothes is proposed.The algorithm mainly includes the following two aspects:(1)Re-identification algorithm for people changing clothes based on multi-modal perception and fusion.The algorithm considers the importance of the image’s twodimensional feature representation and the human body’s three-dimensional structure feature.In this algorithm,firstly,a point cloud feature extraction network is proposed to learn the feature representation of pedestrians in three-dimensional space.Secondly,the human body semantic analysis map is used to locate the semantic part of the clothing in the pedestrian image and cover it to eliminate the influence of the clothing’s appearance.Finally,the obtained image features and point cloud features are input into the attention module to obtain a feature representation with rich information.(2)Re-identification algorithm of cloth-changing pedestrians based on finegrained semantic selection and masking.The algorithm considers extracting effective identity information from pedestrian parts while excluding irrelevant information such as clothing semantics.Firstly,the human body analytic branch(ISP)is used to obtain the pixel-level label of the pedestrian image and the prediction probability map of each part.Secondly,we can selectively erase the semantic information of the clothing area through the prediction probability map and the corresponding label,and then use the part prediction probability map to obtain the features of each part,and input these features in series into the attention module to learn an enhanced feature representation of human body parts.Finally,the loss function is used to constrain the distance between the enhanced image features and the features after erasing the semantic information about clothes,so that the model is more focused on the semantics information unrelated to clothing.