Research On Human Action Detection In Surveillance Videos For Public Security Applications

In today's society,surveillance videos are more and more widely used in public places and play an increasingly important role in maintaining social public security.The detection and identification of behaviors or events in surveillance videos can better serve the surveillance systems.However,due to the complexity of monitoring scene and the diversity of behavioral activities,action detection technique in surveillance video faces higher demanding scenario requirements and more challenges.In this paper,we focus on human action detection technique in surveillance videos,especially for public security applications.We divide the actions into individual actions and group actions,and propose an innovative action detection algorithm for each of the two types of actions.For individual actions in surveillance video,we propose a multi-modal based individual action detection algorithm.Since the skeleton information is a better representation for behavioral gesture and is not interfered by the background environment,we decide to use both the skeleton information and the RGB image for detection.We design a novel way to represent skeleton by mapping each joint point onto the skeleton image.During the feature extraction process,this skeletal representation of mapping with RGB images is fused with the RGB image features through the attention mechanism strategy,which effectively reduces the interference caused by the skeleton detection error.What's more,such representation has a stronger semantic meaning.After feature extraction,the features of the two modalities are fused once again for the network training,which further improves the utilization of the two features.As for the group behavior in surveillance video,we propose optical flow guided group localization method and use it for group action detection.Considering the general characteristics of group behavior,we introduce the optical flow to detect the motion active regions in the video.Based on these detected regions,the individuals are clustered and analyzed.On one hand,such clustering method effectively solves the problem of target object occlusion and motion blur in videos.On the other hand,such motion center-based clustering method is more robust to dense and complex group behaviors.For action recognition,we utilize both image and optical flow acceleration map and design an action recognition network based on the classical two-stream architecture.To accelerate the feature extraction process,we also introduce the ROI-Pooling technology.Finally,based on our proposed action detection algorithms,we conduct related experiments on several mainstream databases,as well as our own collected surveillance video databases.We also compare our methods with several current state-of-the-art methods.The experimental results show that our proposed algorithm has higher detection accuracies and stronger robustness.