Vision Based Hierarchical State Representations for Mobile Robot Task Completion

In this thesis a methodology for hierarchical representations of images from a robot's stereo camera system is developed to provide near real-time state identification. Real-time state identification from images facilitates rapid detection of system state in order to complete a biologically inspired herding task. The methodology incorporates form discovery on a compiled feature dataset to develop a hierarchical tree structure based not only on feature similarity but also on the robot's function or role within a given task. Results show that different hierarchical representations can be derived from the same feature dataset dependent on a robot's role.;VGA resolution stereo cameras mounted on the robots capture scene information. Image features to be mapped to state are developed with discriminately trained, multiscale, deformable part models. Object detection rate is increased using a star cascade technique in which probably approximately admissible thresholds prune highly unlikely image regions. Object detection is initiated as a "top down" process by which the robot knows what it is looking for in a scene, given the task. Once objects are detected in images, features can be derived for use with the hierarchical tree representation to rapidly determine the robot's state. Experiments with the Pioneer 3 All-Terrain robots evaluate and test the method, verifying that the implementation performs tasks in realistic cluttered environments in near real time.