Visual Salient Object Detection Based On Conditional Random Fields

Human visual system can locate the object in a complex scene rapidly,which is helpful for human to capture the most important visual information for further processing and save the computing resource of our brains.Additionally,it allows human response to the environment timely and effectively.While,salient object detection is focuses on how to simulate these human visual properties.In recent years,the human visual system has been well studied by neuroscientists,and based on this,computer vision scientists try to simulate human perception and design computational saliency model to detect and segment the most attractive object automatically which is used to prepare for the subsequent visual tasks.This dissertation mainly focuses on the study of salient object detection in static and dynamic scenes.In the complex scene,multi-objects scene and low contrast scene,the saliency maps generated by the existing algorithms have incomplete object regions,blurred boundaries and noisy background.Therefore,this dissertation mainly studies the image saliency detection,video saliency detection and applications of saliency detection.The main contributions are summarized as follows:1.A superpixel-based conditional random field saliency detection approach is proposed.Most existing CRF approaches set up the probabilistic graphical models with pixel-wise eight neighborhood grid-shaped graph,while our superpixel level graph handling can not only simplify the model but also promote the performance due to the superpixel level tworing with pseudo-background neighborhood system.It is intuitive and easy to interpret.As a result,the saliency maps generated by the proposed model have relatively accurate boundary and pure background regions.Extensive experimental evaluations on six benchmark datasets with pixel-wise ground truths validated the robustness and effectiveness of the proposed saliency model.2.A deep conditional random field neural network based image saliency detection approach is proposed.Most existing deep learning based approaches introduced conditional random field as a post-refinement stage,which is apart from the training of the whole neural network.Thus,the conditional random field can not help to fine-tune the net,additionally,increasing the complexity of training phase.The proposed approach fused the conditional random field into the network directly and constructed an end-to-end net which is benefit to bring the neighbor-constraint into the model.It will further enhance the completeness and restrain the background noisy.Experimental results indicate that the proposed method can generate better saliency maps.3.A novel video saliency detection method based on least square conditional random field is proposed.Different from the traditional video saliency detection methods,which mostly combine spatial and temporal features,we adopt least squares conditional random field to capture the interaction information of regions within a frame or between video frames.Specifically,dual graph-connection models are built on superpixels structure of each frame for training and testing,respectively.In order to extract the essential scene structure from video sequences,LS-CRF is introduced to learn the background texture,object components and the various relationships between foreground and background regions through the training set,and each region will be distributed an inferred saliency value in testing phase.Benefitting from the learned diverse relations among scene regions,the proposed approach achieves reliable results especially on multiple objects scenes or under highly complicated scenes.Further,we substitute weak saliency maps for pixel-wise annotations in training phase to verify the expansibility and practicability of the proposed method.4.A weakly supervised learning-based video saliency detection algorithm utilizing eye fixations information from multiple subjects and total variation-based conditional random field is proposed.Our main idea is to extend eye fixations to saliency regions step by step.First,visual seeds are collected using multiple color space geodesic distance-based seed region mapping with filtered and extended eye fixations.This operation helps raw fixation points spread to the most likely salient regions,namely,visual seed regions.Second,in order to seize the essential scene structure from video sequences,we introduce the total variance-based pairwise interaction model to learn the potential pairwise relationship between foreground and background within a frame or across video frames.In this vein,visual seed regions eventually grow into salient regions.Compared with previous approaches the generated saliency maps have two most outstanding properties: integrity and purity,which are conductive to segment the foreground and significant to the follow-up tasks.Extensive quantitative and qualitative experiments on various video sequences demonstrate that the proposed method outperforms the state-of-the-art image and video saliency detection algorithms.5.Based on the studying of salient object detection,we proposed a saliency guided irregular image mosaic approach.The images,generated by the traditional image mosaic methods,exist the sawtooth effect and have rough boundaries to due to the regular rectangular tile images.Hence,the proposed approach introduce the saliency information and irregular tiles to increase the quantity of mosaic tiles and restrain the sawtooth effect.Experimental results demonstrate the enhancement of visual effect.