A model-based road sign recognition system

A road sign recognition system poses a real challenge for machine vision. It must recognize a wide variety of road signs under considerable variations in illumination and imaging geometry—all in real-time. This thesis presents a modular road sign recognition system relying on modelling for both detection and recognition. It divides into three main stages of processing. The first, concerned with detection, exploits the specific colors of road signs. The color constancy problem caused by the daylight illumination variations is addressed directly with a physics-based model supplemented by a calibration stage using real data. The second stage of processing, devoted to recognizing road signs in regions of interest found in the detection phase, involves a database containing more than 400 road signs arranged in a tree structure, and uses a novel correlation-based template matching technique relying on a bitwise encoding that accounts for both color labels and affine variations in the image formation process, and which also allows to build templates that are able to represent classes of objects. The content of the database used by the recognition algorithm is generated in a deterministic and automated manner by way of geometrical modelling of the image formation process starting with only model images of the road signs to be recognized. The recognition algorithm exploits color as a first logical classification step to direct the search for a road sign in the database, with the later finer steps being driven by correlation scores obtained from template matching. At the third stage of processing, a scene understanding module exploits constraints on the position of road signs along with the spatial relationships they must have in certain cases to other road signs in the image to filter out false positives. During processing, the system incorporates top-down mechanisms that use data fed back by partial recognitions, which allow to progressively gain more information about the identity of a potential road sign, thereby increasing detection and recognition robustness. Experimental results are presented which demonstrate the high overall performance of the system in a real task environment.