Invariant pattern recognition and neural networks

Signals or images with invariance property are common in pattern recognition and image processing when discrimination of signals or visual objects must be independent of where they appear in space or along the time axis. Examples of invariance problems are: a coin image from the scanner in an automatic coin recognition machine can have any orientation in a two dimension plane; handwritten zip codes can be anywhere or have any orientation on an envelope. Such recognition tasks handled easily by biological systems are a great challenge in the engineering field. The conditions under which various algorithms can apply usually are not well stated and there is no well-defined performance measurement to evaluate different algorithms for the invariance problem. In this thesis, we provide a framework for the invariance problem. We study various conditions in which different algorithms may apply. We introduce a new precise measurement to study the performance of different methods and focus on two basic problems: the shift invariance and rotation invariance problem. Based on solid foundations, we develop new algorithms that combine the advantages of traditional methods. The result is a significant improvement for the classification rate of the invariance problem.; The proposed algorithm is better than the traditional central moments for larger shift window and high noise levels. The proposed constructive neural network is better than the traditional backpropagation neural network since it avoids lengthy training and local minima during the non-linear optimization process. Seven algorithms are proposed for shift invariant problems with simulations that demonstrate the performance improvement when the input assumptions are satisfied. Based on these seven algorithms, rotational moments such as Zernike moments are introduced to deal with the shift-rotation invariant problem. Simulation for the Zernike classifier is provided to show that the proposed algorithm is significantly better that the traditional method. Some extensions such as finding the templates and higher order neural networks along with neural network prunning methods are studied.