Knowledge-based image retrieval using spatial and temporal constructs

A knowledge-based approach is introduced for retrieving images by content using spatial and temporal constructs. It supports the answering of conceptual image queries involving similar-to predicates, spatial semantic operators, and references to conceptual terms, as well as temporal, evolutionary, and stream constructs. Interested objects in the images are represented by contours segmented from images. Image content such as shapes and spatial relationships are derived from object contours according to domain-specific image knowledge. Sequences of image objects are represented as streams for retrieving image (sequences) based on their temporal change.; A three-layered model is proposed for integrating image representations, extracted image features, and image semantics. With such a model, images can be retrieved based on the features and content specified in the queries.; A knowledge-based spatial temporal query language (KSTL) is also presented to express and process image queries with conceptual, spatial, temporal, evolutionary, and stream constructs. The implementation of KSTL via extending ODMG's object-oriented query language OQL (Cat94) is also presented.; The knowledge-based query processing is based on a query relaxation technique. The image features are classified by an automatic clustering algorithm and represented by Type Abstraction Hierarchies (TAHs) for knowledge-based query processing. Since the features selected for TAH generation are based on context and user profile, and the TAHs can be generated automatically by a clustering algorithm from the feature database, the proposed image retrieval approach is scalable and context-sensitive.