Researches On Visual Saliency Detection Model And Its Applications

Human vision systems possess a natural ability to rapidly search and locate region of interest in various scene.This kind of ability helps human to process visual information efficiently in daily life.With the large amount of image and video data produced by Internet and social networks,how to select important information from these images and videos appears to be a significant question.Performance of image processing and computer vision tasks can be improved greatly by incorporating visual saliency mechanism into these tasks.There are two main advantages of incorporating visual attention into these tasks: first,it can help computer to deliver more computational resources to process and analyze the most important information in image and video,second,results of such tasks based on visual saliency can meet the demand of visual aesthetics.Visual saliency has tremendous contribution to various image and video processing tasks,such as image and video compressing,image retrieval and image retargeting,etc.Computational model of visual saliency aims to predict which parts of an image are most conspicuous to human eyes with the help of computer vision algorithms.The research attracts tremendous attention from researchers,and a large amount of theories have been proposed in recent decades.Research of this thesis is based on visual attention mechanisms,computational methods of visual saliency detection and its relevant applications have been investigated thoroughly.Some new ideas and algorithms of visual saliency detection methods and its applications have been proposed.The main work and contributions of this thesis are as follows:Firstly,two efficient saliency detection methods based on Hyper-complex Fourier Transform have been proposed.The main framework of saliency detection based on Hyper-complex Fourier Transform is as follows: a color image has been transformed to frequency domain using Hyper-complex Fourier Transform,then filtered in frequency domain,at last a saliency map is derived using inverse Hyper-complex Fourier Transform.The contribution of first proposed method is: different feature channels of image are treated separately instead of holistically,and different saliency map corresponding to each feature channel is derived using the framework just mentioned,then these saliency maps are combined adaptively into the final saliency map.The contribution of second proposed method is: an adaptive post-processing step of the framework above has been proposed,so that saliency detector can be adjusted adaptively according to size of salient object and salient objects of different sizes and shapes can be highlighted uniformly.Secondly,this thesis proposes a salient object detection method based on modeling background features.The main idea of this method is to extract salient object by first constructing background feature distributions and then separate salient object from these distributions.The main steps are: at first,background feature distributions are constructed with the help of clustering algorithms,which could fully exploit statistical information about background features.Then,a coarse saliency map is derived from calculating difference between image features and background features.In the next,a two-step refinement is utilized to enhance and optimize the coarse saliency map,thus producing the final saliency map.The two refinement steps are: edge-preserving super-resolution Gaussian blurring and geodesic distance up-sampling.This method is robust to image noise and can highlight salient object uniformly.Thirdly,this thesis proposes an image retargeting method based on visual saliency.Different from other image retargeting methods,this proposed method employs frequency domain saliency model to calculate importance map.This kind of importance map makes salient objects of different sizes pop out uniformly.At first,the image is partially resized using seam carving algorithm,and then resized using warping algorithm to get the final retargeted image.This proposed method incorporates advantages from both seam carving and warping algorithms,the results are more likely to meet the demand of visual aesthetics.Fourthly,this thesis proposes a salient object detection method based on fully convolutional neural network.Fully convolutional network(FCN)has shown it accuracy and stability in semantic image segmentation.This proposed method transfers its advantage in segmentation to salient object detection task.First,a semantic segmentation result is derived using forward propagation in fully convolutional neural network,meanwhile the deep features in the middle layers of the network are utilized to calculate a coarse saliency map.Second,the coarse saliency map is employed as a guidance map to transfer segmentation image to final saliency result.This method relies solely on pre-trained fully convolutional network,and is proved to be a convenient and efficient salient object detection method.