| Sign language is a natural and fully-formed communication method for deaf or hearing-impaired people.However,it is not well known in the common community,which makes it even harder for the deaf or hearing-impaired people to communicate with others.Unfortunately,most of the state-of-the-art sign recognition technologies are not able to provide a real-time,portable and affordable service in a daily-life envi-ronment because of the limitations of high energy consumption,expensive device costs or the long processing time.Smart wearable devices are becoming smaller,cheaper and popular.The smartwatch is one of the most popular wearable devices.Since the smartwatch is usually equipped with sensors like accelerometer and gyroscope and is worn on the wist,which makes it possible to identify user' s gestures by tracking the movement of user' s finger,hand and arm.We propose a real-time,robust,and user-friendly American sign language recognition(ASLR)system with off-the-shelf mobile devices.This system consists of the data collection system,the offline model training system and the inferring system which is deployed on a smartwatch along with a smart-phone.The smartwatch collects the sign signals and the smartphone outputs translation through an inbuilt loudspeaker.We first create a finger gesture dataset.The finger gesture recognition model achieves a recognition rate of 96%,which shows the mo-tion sensors embeded in the smartwatch can collect enough information about finger movement and hand shapes.WE crate a dataset consisting of 11680 samples about 103 common sentendes and 73 words.We implement a prototype system and run a series of experiments that demonstrate the promising performance of our system.For example,.the average translation time is approximately 1.1 seconds for a sentence with eleven words.The average detection ratio and reliability of sign recognition are 99.2%and 99.5%,respectively.The average word error rate of continuous sentence recognition is 1.04%on average. |
PDF Full Text Request