Content classification and retrieval of digital video based on motion recovery

Content based retrieval of digital video requires the ability to analyze, describe, classify and retrieve video sequences based on the properties of objects and their actions. While content analysis of images has been emphasized in previous research, relatively little has been done on the analysis of the contents of moving pictures. The primary difference between the outcome of analysis of still images and moving images lies in the extraction of object descriptions vs. activity descriptions. Thus, to go from the realm of still image repositories to video databases, we must be able to deal with motion. In particular, we need the ability to classify objects appearing in a video sequence based on the movements of each object, as well as other characteristics and features such as shape or color. In this dissertation we investigate the issues involved in the abstract representation of video indexing mechanisms and retrieval of video sequences based on motion characteristics of objects.; We introduce a dual hierarchy consisting of the spatial and temporal parts for video sequence representation based upon descriptions derived from motion analysis. This gives us the flexibility to examine selected frames at various levels of abstraction, and to retrieve the associated temporal information (say, object trajectories) in addition to the spatial representation. Our algorithm for motion detection uses the motion compensation component of video encoding schemes. The algorithm then computes trajectories for objects of interest.; Synthesizing the results of the above investigations, an abstract algebraic framework for modeling video information is presented. This model is the basis for a query language called VEVA (Visual Extension to VArqa) for dealing with the semantics of video sequences. The specification of the language for retrieval of video is based on the spatial as well as motion characteristics of objects. VEVA incorporates the representation of the visual concepts of the model into a unified iconic and character-based language.