Computer-generated fingerspelling for assistive technology

The focus of this research is to design, develop and evaluate a system for converting written English text to images of fingerspelling, a component of American Sign Language (ASL). This conversion is achieved by parsing and mapping letter pairs from English text to images of letter pairs. Geometries of the letter pair images were captured by a CyberGlove. The objective of this research is to utilize this computer generated fingerspelling system as an assistive technology on affordable, existing personal computers to aid in communications between those who are deaf/hard of hearing and those who can hear.; An important focus of the research is to create and determine enough realism in both the movement of the image and the quality of the image to enhance accuracy and reduce response latency. In order to achieve this objective, the system provides: (i) an effective method for capturing, editing, and transitioning between the signed letter pairs; and (ii) a rich display system consisting of different hand models (simple or realistic) and sizes (proportional or enlarged), motion realism, ( no motion, linear interpolation, or captured transitions), sign rate (slower or faster), and image rate (fewer or more frames).; The system was tested with deaf/hard of hearing subjects conversant with fingerspelling. Detailed analysis from several perspectives using accuracy of individual letters, letter pairs and whole word is presented. Of the forty-two (42) variable combinations tested, only one combination using linear interpolation, enlarged simplified hand model with standard speed and four frames per second—achieved a higher level of accuracy, which was statistically significant. Several other variable combinations offer comparable levels of accuracy. Indeed, several combinations with comparable levels of accuracy allow for effective presentation of letter pairs at the double rate. In addition, some of these combinations result in lower processor demands. These observations are important in view of the long-term objectives of developing a system for converting English text to signed English or ASL.