Research On Video Object Segmentation Algorithm Based On Time Series Information Fusion

Video object segmentation aims at automatically separating the foreground and background areas in video,which is a technology that can accurately classify video at pixel level.According to whether the initial frame mask is used in the test stage,it can be divided into two categories: unsupervised and semi-supervised video object segmentation.Video object segmentation plays an important role in video coding,intelligent monitoring and humancomputer interaction.In recent years,with the development of deep learning,video object segmentation has also made significant progress.However,high-precision algorithms are still very scarce,because the learned model is difficult to effectively cope with changes in complex video scenes,such as severe occlusion,severe deformation,fast motion,and similar target interference.Reasonable and full use of the rich temporal information hidden in the video can help solve these problems.In this thesis,from unsupervised video object segmentation research to semi-supervised video object segmentation,the video object segmentation algorithm based on temporal information fusion has been deeply researched,and the results are as follows:(1)An unsupervised video object segmentation algorithm based on feature alignment and context awareness is proposed.In response to the problems of complex object motion changes between video frames and insufficient local semantic representation,two specialized modules are used to align multi-frame features and extract contextual semantics to make full use of the motion information between adjacent frames and establish semantic relevance.First,the alignment module aligns the features of the left and right frames with the middle frame as the reference to reduce the motion changes between adjacent frames and capture short-term temporal sequence information.Then,the context module uses neighborhood matching technology to model the feature similarity in the spatio-temporal domain,which can enhance the semantic representation of the spatio-temporal context and reduce the interference of background clutter.Finally,lots of evaluations have been conducted on three commonly used standard datasets DAVIS 2016,FBMS and You Tube-Objects,which fully verified the superior performance of this method.(2)A semi-supervised video object segmentation algorithm based on dual temporal memory is proposed.In order to further improve the segmentation results and handle multiple targets video conveniently,the annotation of initial frame is introduced and the video object segmentation is further studied through the semi-supervised route.Two special modules are designed to decouple the temporal information into short-term temporal information and longterm temporal information.These two video sequence features are extracted,saved and memorized separately to further solve the timing modeling problem in video object segmentation.First,the short-term memory module uses graph convolution technology to model the feature similarity between frames in a certain time window to maintain the visual consistency of short-term target features.Then,the long-term memory module uses a simplified convolutional gated recurrent unit to model the long-term evolution process of the target appearance in the video to capture global and stable features to overcome segmentation errors that may be caused by occlusion and mutation.In addition,all the extracted features are sent to a feedback multi-kernel fusion module,which allows the front layer of the network to fuse useful information from the back layer through the feature feedback mechanism and can further enhances the discriminative ability of the model.Finally,lots of evaluations have been conducted on three commonly used standard datasets DAVIS 2016,DAVIS 2017 and You TubeVOS,which fully verified the superior performance of this method.