Research On The Semantics Analysis For Scene Classification

Scene understanding is one of the problems for computer vision techniques to challenge with techniques of image segmentation,object detection and annotation,and scene classification,etc.Among these techniques,scene classification is the precondition to achieve scene understanding,which plays an important role in the visual applications such as video surveillance and robot navigation and decision-making.It has become an important task investigating the techniques of automatic scene classification in the fields of computer vision,machine learning and pattern recognition.During the recent years,with the rapid development of computing technology and image sensors,the way of collecting images has been remarkably expanded,motivating the field of vision.For example,the quantity of images stored by the popular image sharing website,Flickr,exceeds six billion,and the number of active users of Instagram has break through 100 million.Meanwhile,more and more devices have the capability of capturing images,making the smart devices popular and extending their field of application.Rich image data can provide informative resources of better quality,however,manually classifying the large amount of data cannot meet the fast increasing requirement of scene classification and does not coincide with the trend of device intellectualization.Thus it is necessary to investigate the scene classification methods to automatically annotate the category of images to store and achieve all kinds of visual applications.Existing scene classification methods includes classifying methods based on lowlevel visual features and inferential methods established upon semantic knowledge.These methods choose visual features and train visual classifiers to achieve classification tasks,which is effective on small scale sample set.The main drawbacks lie on the facts that there exists semantic gap between low-level features and high-level semantics understand by human,restricting the ability of describing images.At the same time,the knowledge base is merely constructed from semantic attributes,ignoring the importance of visual attributes.A theoretical architecture is proposed to solve these problems,including sample selection,extended bag-of-visual-words(BoVW)model with semantic hierarchies,and knowledge base construction with scene structure and visual attributes to achieve scene classification.The main innovative work is listed below.First,an automatic sample selection method is proposed from the perspective of human cognition to solve the problems of traditional strategy that the category distribution of samples is discarded,and the samples need to be re-annotated after the sampling process.By introducing the certainty measurement quantified by the BoVW model,the entropy-based active learning method can collect and annotate samples at the same time.During the iteration,the negative-accelerated learning theory in cognitive psychology is introduced to adaptively adjust the stop criterion.Samples of different categories are weighted according to the similarity measurement,and their weights are adjusted during the iteration to make the stop criterion more reasonable.Experimental results have proven that the proposed method can improve both the efficiency of sample collection and the effectiveness of classification.Then,a semantics extension method for scene classification is proposed to solve the problem caused by semantic gaps that the low-level features cannot effectively describe high-level semantics in images.The traditional BoVW model is extended by introducing abstract semantics of multiple layers to generate visual vocabulary.Semantics are transmitted from bottom to top to train classifiers of upper semantics to improve the descriptive ability of the BoVW model.The classification process is achieved from top to bottom by generating categories of the testing sample layer by layer.Experimental results have shown that the proposed method achieved better performance than other methods.After that,a hierarchical structure of indoor scenes is proposed to solve the problem of variation and similarities between different categories,which go against the principle of training classifiers.Similarities exists between scenes of different categories,and there is also dissimilarities in scenes of the same category.According to the cognitive principles of human and the features of indoor scenes,a hierarchical structure is proposed by automatically detecting and dividing the hierarchical structure to represent the indoor scenes.Compared with existed classifying methods,the proposed hierarchical structure can describe the indoor scenes more effectively and improves the performance of classification.Last but equally important,a knowledge base constructing method is proposed for indoor scene classification.Indoor scene classification is the precondition to scene interaction.Methods based on first-order logic ignores the commonly existed hierarchical structure and visual attributes.A knowledge representation and inference method based on Markov logic networks is proposed to overcome this shortcoming.The structure of indoor scenes and visual attributes are introduced to construct higher layer knowledge base to improve the descriptive ability of knowledge base.Experimental results have shown that the constructed knowledge base is more robustness,and more effectively in classifying the indoor scenes.To solve the problem of scene classification,research have been taken on sample selection,semantics-extended BoVW,structure of indoor scenes analysis and construction of knowledge base along with visual attributes.The proposed methods constitute the scene classification framework that is able to improve the performance of classification.