| Sign language recognition (SLR) can translate sign gestures into text or voice and provide a helpful tool for the communication between the deaf and the normal hearing society. Meanwhile, sign gesture recognition technology plays an important role in the field of human-computer interaction. From the practical point, a sign gesture translation system designed for the deaf should meet some basic requirements including portability, identifiable vocabulary of a certain size, high recognition rate, low cost, etc. The sign gesture recognition technology based on surface electromyography (sEMG), accelerometer (ACC) and gyroscope (GYRO) sensors has the potential to realize the practical sign language translation system.Although researches on sign gesture recognition technology based on sEMG and inertial sensors have achieved good progress, there is still a large distance between this technology and the practical application. On the one hand, only dozens of sign gestures have been involved in current researches, which cannot meet the need of daily communication. On the other hand, most of the current sign gesture recognition algorithms are realized only in user-specific condition, and the heavy training burden under large vocabulary recognition limits the practicability of this technology.Aiming to solve the problems mentioned above, this paper proposed a component-based vocabulary extensible SLR framework using data from sEMG, ACC, and GYRO sensors. In this framework, a sign word was considered to be a combination of some common sign components, and sign words recognition of a large-vocabulary was realized based on the classification of a small-scale gesture component. The experimental results demonstrated that the proposed framework can significantly reduce the user’s training burden in large-scale gesture recognition, which can facilitate the implementation of a practical SLR system. The main work can be summarized as follows:(1) Through comprehensive analysis of the changes of gesture components during the implementation process of sign gestures, a sign gesture segmented encoding scheme based on gesture components was proposed. Five gesture components including handshape, orientation, axis, rotation and trajectory, which can describe the characteristics of the sign gestures effectively, were involved in this study.(2) A vocabulary extensible sign gesture recognition framework was proposed based on the gesture component encoding scheme. In this framework, the subclasses of the gesture components were determined using the data from reference subjects to obtain the target gesture set code table firstly. Then feature extraction method and classifier for gesture components were design according to the signal characteristics of surface EMG, ACC and GYRO. Finally, large-vocabulary SL recognition with small training burden was realized based on the recognition and code matching of gesture components.(3) To verify the feasibility of the proposed vocabulary extensible sign gesture recognition framework, recognition experiments under different size of training sets were implemented on a target gesture set consisting of 110 frequently-used Chinese Sign Language (CSL) sign words and five subjects. With the smallest training sets (containing about one-third gestures of the target gesture set) suggested by two reference subjects, (82.6±13.2)% and (79.7±13.4)% average recognition accuracy were obtained for 110 words respectively, and the average recognition accuracy climbed up to (88±13.7)% and (86.3±13.7)% when the training set included 50～60 gestures (about half of the target gesture set). |
PDF Full Text Request