Research On Chinese Isolated Sign Language Recognition Based On Channel State Information

Sign language recognition is a technique of translating sign language into words or sounds through computer technology,which is of great significance to the normal communication between the deaf and the non-sign language people.The existing methods of sign language recognition are mainly the sensor-based and the computer vision-based sign language recognition,which hold high hardware dependence,high design and deployment requirements,high maintenance costs.In sign language recognition,isolated words are the basis and precondition of constructing continuous sentences.Based on the influence of sign language movement on the wireless signal transmission,this paper presents a method for identifying isolated words in Chinese sign language by using the channel state information(CSI).The main research content includes the following four aspects.First of all,a signal denoising method using discerte wavelet transform and the principal component analysis is proposed to remove the noise caused by the environment and the internal state transitions in commercial WiFi devices.Moreover,the interpolation method is combined to overcome the sampling defect caused by hardware imperfect and preserve the details of all the subcarriers.Secondly,the multi-stream anomaly detection algorithm is proposed to analyze all the CSI streams synthetically in sign language action segmentate and fusion.Furthermore,the time domain features are extracted to classify different sign language actions.Thirdly,by comparing with the existing CSI based action recognition classification algorithms used in isolated sign language recognition,extreme learning machine is utilized as the fine granularity activity classifier via wireless signals for the first time to meet the accuracy and real-time requirements.Finally,CSI data of different sign language actions are collected in both the empty conference room and the laboratory.The experiment results show that the recognition accuracy of the approach reaches 94.3% and 91.7% respectively in the two environment scenes.