Surveillance Video Structuring And Retrieval In Camera Networks

Video surveillance is an important technique for public security in cities.With rapid growing of the number of surveillance cameras and the amount of surveillance video data,traditional video surveillance methods based on human operators are getting more and more difficult to meet the needs.It’s desired to develop intelligent video surveillance systems equipped with advanced algorithms.The key problems of intelligent video surveillance are "Surveillance Video Struc-turing" and "Target Retrieval".To solve these problems,this dissertation presents three novel works on(1)crowd tracking,(2)multiple attributes recognition for images,and(3)group re-identification.These three works correspond to "Raw Data Acquisition for Targets" and "Target Understanding and Description" in "Surveillance Video Structur-ing",and "Target Retrieval based on Images",respectively.Crowd tracking generates video segments and motion information of each target for further analysis.Multiple at-tributes recognition describes images with attributes,which provides high-level seman-tic features for image-based retrieval,and also makes it possible to retrieve targets with natural language.Group re-identification complements current works on single person re-identification,which is a fundamental technique for cross-view human retrieval with only appearance cues(non-face).The main contributions of this dissertation are as follows:(1)Proposed a novel approach for crowd tracking based on group evolution.This approach integrates low-level keypoint tracking,mid-level.patch detection and tracking,and high-level group evolution into one unified framework.Instead of com-puting optical flows,tracking keypoints,or detecting pedestrians,the crowd is repre-sented as a set of distinctive and stable patches in this work.In the low level,keypoint tracking provides accurate local motions,which guides the detection of patches,and also organizes patches into groups.In the middle level,spatial constraints between patches are captured by the proposed hierarchical tree structure.In the high level,group evolu-tion guides updating of the proposed hierarchical tree structure through merge and split events.Extensive experiments show that:the proposed patch detection approach pro-vides important assistance for tracked targets;the proposed dynamic hierarchical tree structure could effectively capture spatial relations among targets;and the whole crowd tracking framework significantly outperforms state-of-the-art methods.(2)Proposed a novel approach for multiple attributes recognition of images based on spatial regularizations.This approach exploits both semantic and spatial re-lations between attributes in an end-to-end trainable deep convolutional neural network,with only image-level supervisions.Given one image,the proposed Spatial Regular-ization Network(SRN)generates attention maps for all attributes,and then captures the underlying semantic and spatial relations of attributes based on learned attention maps.Finally,the original attribute confidences from the basic network(for example,the ResNet-101 network)are regularized by adding confidences from SRN.Extensive experiments on multiple public datasets show that:the proposed SRN could effectively capture spatial relations of attributes,and such regularizations could significantly boost the performance of multiple attributes recognition for images.(3)Proposed a novel approach for group re-identification based on patch match-ing.Compared with single person re-identification,group re-identification faces new challenges,such as great intra-group occlusions,significant variation of relative po-sitions,and so on.This approach models group re-identification as a patch matching problem between two sets of image patches.First,discriminative "Salience Channels"are learned to filter out patch matches which have low appearance similarities or are not discriminative between two group images.Then,the resulting candidate patch cor-respondences are further explored by the proposed "Consistent Matching" process,re-sulting in the final similarity score of two group images.Extensive experiments show that:the proposed approach significantly outperforms state-of-the-arts;the proposed"Salience Channels" and "Consistent Matching" are complementary in improving the performance of group re-identification.