Research On Deep Learning Based Person Re-identification And Its Security

Person re-identification(Re-ID)technology is capable of using a large number of online interconnected multi-cameras to realize cross-domain tracking and matching for specific pedestrians.Therefore,it is widely used in the field of intelligent security such as criminal investigation,pedestrian tracking and human behavior analysis,which can provide technical guarantee for the public security and stability.In recent years,with the rapid development of deep learning in the field of person Re-ID,its recognition performance has become more effective,and various classic Re-ID models and algorithms have been produced as well.However,despite the good performance of these algorithms,the recognition effects under certain conditions still need to be improved,especially in the adversarial environment with adversary attack,the security of Re-ID model and the reliability of its recognition results will be questioned,which will greatly limit the application and expansion of person Re-ID in special scenes.To this end,this thesis focuses on two aspects of person Re-ID model algorithm and person Re-ID model security.Among them,the attribute feature learning and meta metric learning aim to study how to improve the recognition accuracy;the adversarial example attack and defense and the feature visualization aim to study how to improve the model security.The main contents and innovative contributions are summarized as follows:Firstly,aiming at the problems that existing attribute recognition methods are difficult to extract targeted features of specific attributes,and the dependencies between various attributes are not fully explored,an effective Re-ID model based on attribute attention and dependency is proposed.This approach constructs a joint network combining with attribute recognition and identity recognition.The improvement of recognition accuracy is achieved through the mutual assistance between attribute information and identity information.First,the Res Net-50 is employed as the basic module for feature extraction of the input image to obtain the last convolutional feature map.Second,for the attribute recognition branch,the attention mechanism is introduced to extract fine-grained feature vectors for each attribute,and then the Long Short-Term Memory(LSTM)is used to mine the dependencies between different attributes.After that,the output of LSTM is sent to attribute classifier.For the identity recognition branch,the feature map is processed using global average pooling and then is input into the identity classifier through the fully connected layer.Finally,the attribute classification loss is combined with the identity classification loss to achieve multi-task training and optimization.The experimental results show that compared with the benchmark method,the Rank-1 accuracy rate(Rank-1)and mean average precision(m AP)are increased by 0.83% and 0.91% respectively on Market-1501 dataset,and by 0.63% and 0.97% respectively on Duke MTMC-reID dataset.It indicates that the proposed method can effectively improve the recognition accuracy and achieve a competitive performance.Secondly,aiming at the problems that person Re-ID based on metric learning has a poor generalization performance on the test set,and the metric distance between examples are difficult to achieve adaptive optimization,a person Re-ID method based on meta metric learning is proposed.This approach uses a metric-based meta learning to model the task,and simultaneously introduces an improved circle loss for metric learning to improve the adaptive optimization effect of the distance between examples.First,the learning process of person Re-ID based on meta metric learning is constructed by setting subtasks.According to the learning method of subtask by subtask,each subtask is divided into query example and support example,and then they are mapped to the vector space.Second,the loss between the query example and support example in this vector space is calculated,and adaptive update intensity for each similarity score is set using the improved circle loss function.Finally,the model parameters are trained through each subtask.The experimental results show that compared with the benchmark method,the Rank-1 and m AP value are increased by 0.4% and 1.4% respectively on the Market-1501 dataset,and by 0.9% and 0.6% respectively on the Duku MTMC-reID dataset,which proves the effectiveness of the proposed method with competitive metric learning performance.Thirdly,aiming at the security problem that the result of person Re-ID may be wrong caused by adversarial examples attack,by summarizing and sorting out existing attack methods from the perspective of attacker,a more comprehensive attack method is constructed,and two simple and effective defense strategies are proposed to defend against adversarial examples.These studies provide a useful exploration for improving the data security and robustness of the model.First,several existing generation methods of adversarial example are analyzed,and then a pedestrian adversarial example generation method based on feature vector constraints is constructed.The adversarial perturbation added to the query examples is updated by feature vector constraints,thereby the adversarial query examples are generated,and both non-targeted attack and targeted attack can be achieved.In order to defend against the proposed attack method,two simple and feasible defense strategies are proposed.Only by a simple preprocessing before inputting examples to the model,the attack effect of adversarial example can be destroyed to restore the retrieval results to a certain extent.In addition,qualitative analysis shows that the proposed defense strategies have certain applicability to general attack methods.Fourth,aiming at the problem that the person Re-ID model is difficult to provide a specific basis for the recognition results,thereby limiting the interpretability and trustworthiness of the model,a pedestrian feature visualization method based on saliency map is proposed,which uses saliency map to demonstrate the pedestrian features and explains the reason why person in different images can be matched.Specifically,this approach is aimed to generate saliency map according to the recognition results,and highlights important regions of input features contributing to the output results.The feature interpretation of recognition result is provided from the perspective of visual observation,which is simple and intuitive.First,the network is fine-tuned on the Market-1501 dataset to build the person Re-ID model.Second,the forward feature extraction and backward saliency map generation are conducted to design the feature visualization process aiming at recognition results.After that,the average gradients of each channel with respect to similarity score among input images are used as the channel weights.Finally,the last convolutional feature map is weighted channel-wisely to synthesize the saliency map.The experimental results show that the proposed method can not only enable the network to identify key features of pedestrian across different images,but also provide visual interpretation for the person recognition results to improve the interpretability of person Re-ID and foster trust from users regarding its decisions.