Research On Progressive Image Analysis And Its Applications

As one of the most commonly used form of visual content expression,digital image analysis and its related research have attracted much attention.Image analysis technology acquires meaningful objective information by detecting,extracting,expressing,and describing the target objects in the image.It is one of the indispensable foundations of visual understanding and applications.The research achievements of image analysis have been widely applied to many fields such as industrial manufacturing,digital entertainment,security monitoring,and intelligent transportation.Due to the lack of sufficient guidance information,the traditional image analysis methods designed for a single input image or a group of images have limited quality of results.The data-driven image analysis methods which are popular in recent years have solved these problems well.However,since such methods require a large amount of related images to be prepared in advance,the development of their applications is limited.In addition,the requirements from of applications also bring new challenges to image analysis.Combining with the research status and shortcomings of current image analysis methods,this thesis proposes a progressive image analysis scheme to meet the actual requirements of the Random-needle Embroidery application,and gradually explores and accumulates guidance information in image analysis.In this thesis,the related technologies are discussed in three aspects:image representation,image analysis modeling and image analysis process framework.This thesis makes an in-depth and systematic study on three different levels of specific analysis tasks including image classification,image segmentation and image parsing.The corresponding progressive analysis methods for these three tasks have been designed and implemented,which verified the feasibility of the proposed scheme.Regarding these issues,this thesis engages in the following work:(1)An iterative image classification method based on metric learning is proposed.The method improves the classification result by introducing user guidance,and supports the modeling and learning of user guidance using an iterative classification process.It combines multi-view image representation,metric learning,image clustering,online learning and user intervention in image classification.The method proposes to model the user’s classification guidance preferences as the classification perspective together with the category granularity.In order to learn the user’s classification perspective,the method models the user’s classification perspective as a weighting of multiple perspectives,and uses a multi-view online metric learning method to learn the user’s perspective preference.For the category granularity,this methods proposes a multi-centroid nearest class mean classifier,combining with the learned distance measurement of the user’s perspective preference,to online learn the classification granularity of the user.At the same time,the method uses image clustering strategy to mine the correlation information between images,and proposes to cluster images before user annotation.It chooses one cluster of images to the user for labeling,which greatly improves the classification efficiency.(2)An accumulative image segmentation method based on incremental learning is proposed.The method augments user labels based on a multi-level image representation,and uses an accumulative image segmentation process to explore the user interaction more efficient.This method integrates multi-level image representation,incremental learning,online learning and graph cut optimization in interactive image segmentation.For the scribble-based interactive method based on label propagation,it builds a three-level image representation using pixel,superpixel and over-segmented images.The user annotation on the pixel image is passed to the corresponding superpixel sample,and further passed to the neighbor superpixels based on the over-segmentation constraints.In addition,an incremental classifier is online trained using the segmented image after each segmentation.Because of this,the subsequent images can be segmented without any interaction or with just a small amount of interaction.These strategies greatly improve the friendliness of user interaction and reduce the interaction burden.At the same time,the method uses the graph cut algorithm to utilize the appearance consistency information of regions by analyzing the classification confidence of the pixels and the difference between adjacent pixels.Also,it eliminates the effects of superpixel overflow and classification noise effectively,and improves the accuracy of image segmentation.(3)A progressive image parsing method based on convolution neural networks is proposed.The method designs a stacked and a recurrent neural network structures to parse images from coarse to fine from multiple granularities,and uses a pre-defined multi-granularity parsing rules to generate training data for each granularity.The method proposes to parse images in details by a coarseto-fine multi-granularity image segmentation,and the coarse-grained segmentation result is used as a priori of fine-grained segmentation.It proposes to realize this process by stacking multiple segmentation modules in neural networks.The method proposes to add a skip-connection from the shallow layer of the network connected to the fine-grained image parsing module.By doing this,it can introduce shallow image coding information for the fine-grained parsing which provides necessary details for more accurate fine-grained image parsing.In addition,the method proposes to use an LSTM layer to simulate this type of stacked framework,which simplifies the design of the stacking network when multi-granularity analysis is performed.Finally,the method proposes to automatically generate multi-granularity training data by using pre-defined multi-granularity image parsing rules from coarse-to-fine grained using groundtruth data.Then the multi-level supervision is used to train the stack/recurrent neural network which reduce the requirements for training data.