| Due to the application of image capture technology,and the development of Internet and Internet of Things technology,people can easily create and share videos on their computers or cell phones.In recent decades,as more and more people are involved in the process of video production and sharing,a huge amount of videos are uploaded to the Internet every day,which makes the number of videos on the Internet show an exponential growth trend.These videos have a wide variety of content,including news videos,film and TV videos,surveillance videos,video logs(vlogs),etc.The rapid growth of video data has not only enriched people’s social life,but also greatly broadened the way people get all kinds of information and learn about new things and the outside world.However,it also brings many urgent problems,such as great pressure and challenges to users in video storage,distribution and retrieval.In order to solve the above-mentioned problems of difficult video retrieval and inefficient distribution,video summarization technology has emerged and is gradually attracting more and more attention.The computationally generated summary video is a high summary of the original video,which retains the important information in the original video.A very important feature of video summarization technology is that it comes in a variety of forms.Video summaries can consist of either static video frames or dynamic video clips,or even moving targets from the original video in different spaces and times can be stitched together to generate entirely new video content.But no matter what form the video summary takes,its length and viewing time should be smaller than the original video,thus helping users to quickly browse the original video and get the key information in the video.Therefore,video summarization techniques that can extract important information from videos have become a hot topic in the field of computing in recent years.This paper makes a study for this problem by deep learning algorithm,and the specific work is as follows.(1)A video summarization model based on full convolutional sequence network is proposed,in which the model uses temporal pyramidal pooling to extract multi-scale contextual information in videos,which produces better results for videos with complicated scene changes and more shots.Conditional random fields with full connectivity are also used to label the video frame sequences,so that the model is enhanced in the classification details of video frames.Experiments on the TVSum and Sum Me datasets show that the model achieves better performance compared to the original fully convolutional sequence network.(2)An audio-video feature fusion method based on convolutional neural network is proposed to address the existing methods that only extract image features in the video but ignore audio features in the video when doing pre-processing of the input video.In order to achieve better representation and diversity of video summaries,generative adversarial networks are introduced to enhance the results.The method is experimented on TVSum and Sum Me datasets and the results are analyzed.The final results show that the multimodal fused features,which make up for the deficiencies of single image features,and the algorithm achieves better performance compared to current state-of-the-art methods.(3)A generative adversarial network model based on a full convolutional sequence network is proposed.The improved full convolutional sequence network is used as the generator of the generative adversarial network.The feature vector generated by the encoder part of the generator is then processed and mapped into a score that is used to determine whether the input summary video is a true summary video.We use this modified network structure as the discriminator of the generative adversarial network.The adversarial training of the generator and discriminator enables the generator to generate video summaries with diversity and representativeness. |
