Scene Understanding Using Internet Photo Collections

With billions of photos now online, much computer vision research has been devoted to using Internet photo collections for tasks such as visualization, learning object category models, and 3D scene reconstruction. While most of this recent work leverages the sheer quantity of online images, I present several approaches for using the distribution of images (and associated metadata) to extract structured information about 3D scenes. In essence, I use online photo collections as a proxy for human perception in aggregate, treating each photo as a statement about the world and not just a source of visual data;I present three examples of information extraction leveraging the distribution of online photos from the photo-sharing site Flickr. First, I demonstrate the selection of canonical views of objects and scenes via a greedy image clustering algorithm. Second, I show how scenes can be decomposed into individual objects by using a cue based on the field-of-view of large numbers of images. Finally, I extract scene-scale human movement patterns from the distribution of photo sequences. Based on these projects, I demonstrate applications to scene summarization, browsing, image/object tagging, and visualization.;What objects and views do people find interesting? What is an object? How do people move around while exploring a scene? How do people frame their photos? In this thesis, I suggest a new way to answer such questions about the world, and our perception of it, using Internet photo collections.