Language-based Person Re-identification

In recent years,more and more surveillance cameras have been distributed in all corners of the city,have been used to record and investigate criminal behaviors,and have made a huge contribution to the stability of social security.However,searching for useful information in massive image databases is still a complicated and huge task,which takes a lot of time and effort to search by human alone.Therefore,the person re-identification technology,aiming at automatically identifying and matching persons,has very important research significance.Person re-identification tasks can be roughly divided into two categories according to different retrieval methods: image-image and language-image person re-identification ones.Among them,the latter one has wider potential applications due to less restrictive query conditions.However,in actual research,due to the different ways of expressing information in images and natural languages,how to perform effective cross-modal matching is a difficult problem.This thesis aims to retrieve the corresponding images in the image dataset according to the given natural language information and realize the language-image person recognition.As a kind of language information,person attributes play an important role in the person re-identification algorithm by virtue of efficient information transmission.Many studies have demonstrated that it is difficult to learn a re-identification model with sufficient generalization ability by relying on the person identity information alone.Therefore,this thesis proposes a person attribute recognition network,which adds an attribute classification model to the traditional person re-identification system,thereby introducing more than twenty attribute labels such as age,hair,and clothes.The person attribute recognition network can accurately classify person attributes while predicting the identity category of persons.The experimental results show that the combination of attribute characteristics and identity information greatly improves the generalization ability of the model.It not only can effectively identify the attribute information of persons but also predicts the identity categories of persons more accurately.Compared with discrete attribute information,natural language descriptions are more suitable for users' expression habits and the acquisition cost is smaller,which is more popular with users.However,effective natural speech information mining,robust text and image feature extraction,and accurate cross-modal feature matching are the keys to solving the problem of natural language person re-identification.To address the aforementioned problems,this thesis constructs an effective natural language person search framework,which mainly includes three parts: a deep convolutional neural network to extract image visual features,a bidirectional long short-term memory network that encodes natural language information,and triplet loss function for matching image language embedding.In the natural language encoding network,this thesis introduces an attention mechanism module to assign different weights to different words to enhance the effectiveness and accuracy of extracting language features.When matching image language embedded features,this thesis introduces a mutual learning mechanism to promote better matching of image features and language features.The experimental results show that the natural language person search framework proposed in this thesis has good performance and achieves relatively good performance on the CUHK-PEDES public dataset.