Navigating media using virtual environments

Today, video is pervasive and everyone and everything seems to be equipped with video cameras. However, once the video is captured there are not many options for viewing it besides watching hours, days or even years of video from start to finish. In this thesis, we address this problem for tour-type video by developing an automatic toolkit for the creation of video albums augmented with interactive virtual environment technologies.; Virtual environment technologies permit navigation through complex surroundings based on intuitive actions like “go forward” or “go left”. The space is traditionally constructed using computer graphics models and enhanced with video, still images, and sound. While video is usually incorporated into these models, it is played as taken and generally as a dependent media; i.e., the navigation controls do not affect the video even if the video is a “real” walkthrough of the virtual space. Integrating the real-world media and the computer graphics model by registering both within a common virtual reality framework would allow navigation in the computer graphics model to control video of the corresponding location in the real world and vice versa. In other words, the user can “ride the video” in the virtual world and also use the virtual world to control the video.; Additionally, due to the underlying framework, new media of any type can be added, including other video tours taken at different times by various parties. The paths of each tour can be intertwined and intersect creating infinite synchronized, navigable paths of real and synthetic media in the virtual environment. This is especially important today as video is omnipresent. Efficient and intuitive indexing and control is required for data mining huge video libraries, telepresence and teleoperation, emergency management, entertainment and education.; In this thesis, a multimedia data structure and set of applications capable of supporting these navigation operations through video is presented. A mobile video camcorder is demonstrated that can record GPS enhanced video and process it for inclusion in the framework described. Next, an interactive multimedia virtual environment application is presented that allows navigation through the virtual world created in the previous steps. The capabilities of the system are demonstrated using video filmed in Jersey City and New Brunswick, New Jersey along with several examples that highlight possibilities for enhancements and future work.