| Video classification mainly recognizes and classifies human actions in videos,and has a wide application prospect in the field of video understanding.The video contains rich spatial and temporal information.How to effectively extract the spatial and temporal features of the video is still the key technology in video understanding,and it is also the difficulty of video understanding.In order to improve the performance of video classification,this paper uses 3D convolutional neural network to study this problem.The specific research content is as follows:Firstly,for the spatio-temporal network,as the depth of the network deepens,the extracted features are more abstract,which makes the model ignore the problem of detailed information in the data.This paper proposes a deep space-time network with multi-level feature fusion.In this paper,three different pooling units are designed to extract and compress the features of each layer of the deep convolution network.Finally,the different levels of hybrid features are cascaded and classified.At the same time,different pooling units and classification network architectures are compared.Secondly,in order to make good use of complementary between C3 D and optical flow 2D network and reduce computing load of optical flow.This paper proposes a video classification algorithm for end-to-end fusion spatiotemporal two-stream convolutional network,which can combine the advantages of C3 D and self-learning end-to-end optical convolution network.For spatial streams,C3D-based Res Ne Xt-101 networks are used for video classification.The other branch uses an end-to-end time stream network,and the optical flow learning is performed by the TVnet network in real time,then the BN-Inception network is used for video classification for the stacked optical flow data.Finally,the video classification results of the time stream and the spatial stream are weighted and combined to form a final decision.Finally,in order to solve the problem of calculating the cost and storage requirements for decompressing video frame by frame and extracting optical flow information for temporal modeling between frames,this paper proposes Multi-information fusion based on convolutional neural network for Compressed Video Classification.The I-frames,the motion vector(MV)and the Residual(Res)are directly extracted from the compressed video for classification.The I-frames are the key frames in the compressed video and contain the complete image information,which are classified using the Res Net-152.The motion vector contains motion information similar to the optical flow,the residual contains the contour information of the moving object in the video,and the motion vector and residual are classified using the R(2+1)D network.Finally,the results of the three kinds of information are weighted and combined to produce classification results. |
