Research On Key Techniques Of Multi-camera Collaboration For Digital City Surveillance

As an important means of perceiving city running status and improving urban management efficiency,digital city surveillance is not only the guarantee of urban safety and stability but also the core factor of city development.It has important theoretical significance and practical value in many fields such as public security,industrial production,traffic management,community life and so on.However,with the increase of video surveillance requirement and the popularity of imaging sensors,traditional systems are limited by the independent monitoring mode of each camera.This leads to their deficiencies in omnidirectional visual perception and global information interconnection.Therefore,how to break the visual information barrier between cameras,realize complementary advantages and efficient collaboration of multiple cameras,and improve system monitoring ability is the key problem to be solved urgently for digital city surveillance.This dissertation puts the emphasis on multi-camera collaborative digital city surveillance,detailed analyzes two typical spatial layouts of multi-camera video monitoring system(i.e.,dense multi-camera array for single-point monitoring and distributed multi-camera system for multi-point monitoring).It aims at the improvement of single-point omnidirectional perception ability and multi-point global visual information interconnection ability,starts from the perspective of multi-camera fusion imaging,spatial alignment and target association to explore the internal relationship of different camera,and studies a multi-camera collaborative de-occlusion surveillance approach based on semantic feedback,a multi-camera spatial collaborative monitoring method based on common feature combination,and a multi-camera target collaborative surveillance method based on object-object interaction consistency.On the basis of the above works,this dissertation finally designs a new multi-camera collaborative video surveillance system.The main research contents and contributions are stated as follows:1.In order to solve the problem that current single-point surveillance system with limited perception ability has many monitoring blind areas when it observes crowded scene with serious mutual occlusion,a multi-camera collaborative de-occlusion surveillance approach based on semantic feedback is proposed by using dense multi-camera array.This method first mines the semantics of multi-camera visual information according to the characteristic of crowed scene that the front target may be the occlusion of target behind it,and labels the visual information whose semantic is occlusion.After that,a depth-by-depth semantic feedback strategy is used to filter multi-camera visual information at two layers.The first layer filters out the visual data which semantic meaning is occlusion.That can remove the interference from foreground defocused images.The second layer selects the optimal subcamera array that meets the baseline width and has the most effective target information.In this way,the negative impact from redundant perspectives is reduced.Then,according to the pre-calibrated multi-camera spatial parameters,this method fuses all filtered multi-camera visual information and realizes de-occlusion monitoring of the entire scene.Experimental results show that the proposed method can improve the omni-directional perception ability of single point monitoring through multi-camera collaborative visualization.2.Compared with dense multi-camera array for single-point monitoring,distributed multicamera system for multi-point monitoring has the problem of inconsistent imaging space and inability to achieve collaborative monitoring due to scattered camera spatial layout and unclear relative position relationships.Aiming at this problem,a multi-camera spatial collaborative surveillance method based on common feature joint analysis is proposed by using the mutual complementarity and mutual restriction of common visual information between different cameras.On the one hand,this method presents an inter-frame common feature joint analysis strategy for camera pose initialization.According to camera pose consistency between different frames,it performs complementary integration and redundant screening of inter-frame common features and achieves the success rate and accuracy improvement of camera spatial parameter initialization; On the other hand,this method introduces an intercamera common feature joint analysis strategy for camera spatial relationship optimization.Through the mutual restriction of common visual features between multiple overlapping cameras,it obtains a more accurate camera spatial relation.In this way,the proposed method estimates the relative position relationship of all cameras in distributed system and realizes spatial collaborative monitoring of multiple cameras successfully.Qualitative and quantitative experiments show that this approach can not only achieve good spatial cooperative monitoring performance under partial occlusion,large view gap and more visual interference,but also has better robustness than other methods.3.The distributed multi-camera systems for multi-point monitoring with limited visual characteristics or related information of surveillance scene are usually difficult to obtain the relative position relationship between cameras and cannot achieve collaborative monitoring in space.To address the problem that each camera independent work in such system causes visual information fragmentation of the same target,this dissertation proposes a distributed multi-camera target collaborative monitoring method based on object-object interaction consistency.Inspired by the invariance of line intersection point under perspective projection transformation,it exploits cross-camera consistency of the interaction between objects to mine target association relationship in different cameras,and then realizes multi-camera collaborative surveillance.The proposed method first extracts object-object interaction information in each camera,and describes them as interactive feature vectors.The interactive feature vector includes interaction time,position and local target information.Then,using the cross-camera spatiotemporal consistency of interactive feature vectors,the time synchronization parameters and spatial transformation relations of multi-camera system are calculated by wide-domain cross-correlation analysis.On this basis,this method finally utilizes local targets’ cross-camera correspondence in spatiotemporal aligned interactions to find the association relationship of the same target visual information in different cameras.After that,target collaborative monitoring of distributed multi-camera system is realized.The experimental results demonstrate that the proposed multi-camera target collaborative surveillance method does not need pre-calibrated multi-camera spatial parameters,and has advantages of both wide application range and good robustness.4.To meet the urgent demand of multi-camera collaborative work in current video surveillance,this dissertation designs and constructs a multi-camera collaborative video surveillance system on the basis of the above work.There are three modules in this system.The first is multi-camera monitoring data acquisition to collect the visual data of each camera synchronously; The second is multi-camera spatial relationship calibration to estimate the relative position relationship between cameras; The last is multi-camera collaborative video monitoring.This module utilizes multi-camera monitoring data and spatial calibration parameters to realize a variety of collaborative monitoring functions,including threedimensional fusion display,crowd removal monitoring and collaborative target tracking.This system takes full advantages of each camera’s monitoring ability,and achieves good omni-directional perception and global information interconnection performance through multi-camera effective cooperation.