Research On Semi-supervised Video Object Segmentation Algorithm Based On Spatiotemporal Memor

Video object segmentation is one of the most fundamental tasks in machine vision and plays an important role in practical applications.It can be divided into unsupervised video object segmentation and semi-supervised video object segmentation according to human participation.The purpose of this work is to study the video object segmentation task in semisupervised scene,that is,to predict the target mask in the subsequent frames according to the annotation of the artificially annotated target mask in the first frame of video.Currently,spacetime memory-based methods are widely popular due to their excellent performance.The basic idea is to build an external memory to remember the target object information in the history frame,then match the pixels in the query frame with the target object information in the memory,and then select the information that is conducive to modeling the target object in the query frame to complete the prediction.Although such methods can effectively deal with occlusion and deformation problems,they still suffer from the following two problems: 1)they cannot achieve a balance between accuracy and efficiency,that is,short-term memory is efficient but has limited performance;long-term memory has good performance but low efficiency.2)they cannot deal with the interference brought by similar semantic objects and are prone to false predictions.In order to solve the above problems,in-depth research has been carried out in this work.The main contributions can be summarized as follows:(1)To address the inability to balance algorithmic accuracy and efficiency,a semisupervised video object segmentation algorithm based on spatio-temporal compression is proposed in this work,which improves the efficiency of the algorithm by reducing the spatiotemporal redundancy in memory.In terms of time,the algorithm adaptively selects video frames with obvious changes for memory update.In space,the algorithm no longer stores all the pixel information in the video frame,but only stores the pixel information with low similarity to the existing pixel information in memory.In addition,the algorithm further reduces spatial redundancy by discarding old pixel information that is less used.Finally,the algorithm proposed a more efficient memory reading mechanism,with lower memory footprint and computational cost to achieve the same reading effect.Experimental results show that the proposed algorithm is effective in reducing spatio-temporal redundancy and improving algorithm efficiency.Finally,a large number of ablation experiments demonstrate the effectiveness of the proposed algorithm.(2)To address the problem of inability to deal with interference caused by similar semantic objects,a semi-supervised video object segmentation algorithm based on spatio-temporal scene learning is proposed in this work,which can avoid interference objects,eliminate target candidate regions and avoid wrong segmentation by utilizing surrounding scene information.Specifically,the algorithm encodes the surrounding scene information into dense knowledge vector,which is transmitted in the form of video sequence,and then combined with the output of the target appearance model to guide the target object modeling of the query frame.In addition,the algorithm introduces a knowledge updating strategy to update the knowledge vector,which can mark the distractors newly entering the scene or correct the knowledge propagated by mistake.In this work,the algorithm is tested on multiple benchmark datasets,and the results show that the algorithm can capture rich scene information,and this information can complement the output of the target appearance model.Finally,a large number of ablation experiments demonstrate the effectiveness of the proposed algorithm.