Research On Moving Objects Segmentation Based On Time Difference Image Pairs

Segmentation of moving objects is an important part of the intelligent monitoring system,which is also the basis for tracking the object in the later stage and analyzing the behavior of the object.It plays an important role in image stitching to remove the"ghosting" and unmanned driving.The traditional segmentation of moving objects usually depends on the color of the image,and the category of the object is unknown.With the introduction of the deep learning-based moving object segmentation methods,the moving object segmentation is raised to the class known level.Based on the current situation of moving object segmentation,this paper proposed a moving object segmentation algorithm combined with optical flow field,instance segmentation and motion status classification for a time different image pair captured by the same camera.We generated optical flow field through the network FlowNet2.0 and got instance segmentation result through the network MNC.Then we put the result generated by FlowNet2.0 and MNC together with the image RGB information into the classification network based on ResNet-34 to get the classification of the motion status of each object.Finally,we can get the moving object segmentation in the whole image pair.The main results of the research in this paper are as follows:(1)A pixel-by-pixel object level segmentation is performed on the moving object.In the past,the segmentation of moving objects stayed at the semantic segmentation level-that is,the motion regions and non-motion regions of the entire figure were separated from each other by pixels,and the category categories of each pixel in the full image were classified.However,Semantic Segmentation cannot separate moving objects in units of objects.In this paper,we uses a pixel-by-pixel segmentation method to get the object level segmentation with a hot spot search strategy.(2)Segment the moving objects on image pairs that are took with the same camera but not captured in the same time.In the past,the common moving object segmentation methods are usually based on a video image sequence to get the information of frames in the sequence as a reference.However,unlike video sequences,it is difficult to segment the moving objects of the image pairs.In this paper,the segmentation of moving objects is based on image pairs,which improves the application of moving objects segmentation,such as image stitching.(3)A new method of moving object segmentation based on deep learning is proposed.In the past,the moving object segmentation based on deep learning was segmented into semantic level.In this paper,we use deep learning to get instance segmentation,as well as getting the optical flow field to obtain the motion feature information of objects.Then a motion status classification network based on deep learning was introduced,with the input of image information,optical flow field and segmentation.Beside,we set up our motion status classification dataset,which can provide data support for later research on moving object segmentation.Experimental results show that the motion status classification network model trained in this paper achieves an accuracy of 94.1%in the test set of the real scene,and the classification accuracy is 70.6%for the difficult-to-handle small-scale moving situation.In addition,when classifying the motion status accuracy of various objects,the accuracy reached 84.7%,which support the strong universality of the classification network.Finally,after mapping the target back to the original image,the overall moving object segmentation results is compared with the frame-difference method.We can see our method can avoid the influence of noisy in the image,getting more information in the segmentation which can broaden the application.