Development And Algorithm Research Of Multi-Terminal Continuous Chinese Sign Language Recognition System

As the main way of communication for the deaf mute,sign language plays a great role in the communication of daily life.In order to solve the problem of barrier free communication between the deaf and the deaf,sign language recognition technology came into being.Nowadays,the research on sign language recognition has made great progress,but there is still a lack of corresponding functional systems to apply the recognition methods.The use of sign language recognition technology combined with a convenient and friendly client operating system to solve the problem of inconvenient communication between deaf mutes and listeners can help deaf mutes better integrate into society and enjoy life.Therefore,the research of sign language recognition and development of its corresponding system are of great significance.To solve the above problems,this paper proposes a deep learning based sign language recognition algorithm,and designs a multi terminal continuous Chinese sign language recognition system with the cloud server as the data processing center.The main work is as follows:（1）In view of the current situation that the relevant sign language data sets are difficult to obtain and there are few data samples,this paper designs a sign language data set production and data preprocessing system,which includes the functions of image acquisition,important feature interception and segmented information extraction in the process of sign language demonstration.The system can save the important feature image information and segment information demonstrated by the sign language user,and realize the production of sign language data set and the preprocessing of algorithm training data.（2）In continuous sign language recognition,the lack of segmentation information between the corresponding sign language actions of each word has a serious impact on the recognition accuracy.According to the characteristics of Chinese sign language,a continuous Chinese sign language recognition algorithm is designed and implemented by using convolution neural network and bidirectional long short-term memory network.The algorithm uses connectionist temporal classification as the objective function to forcibly align the output results with the tag sequence to detect the transition frames between the words in the sign language.This method uses the self-learning ability of convolution neural network to replace the traditional artificial feature extraction,and fully obtains the long-term and spatial information of context in continuous sign language through the bidirectional long-term and short-term memory network.Finally,the effectiveness of the proposed algorithm is verified by several groups of comparative experiments.（3）In view of the current lack of application programs related to sign language recognition,this paper takes the cloud server as the data processing center and designs a multi terminal continuous Chinese sign language recognition system according to the commonly used computer desktop programs,browser web pages,wechat applets and other user operating terminals.Upload the trained algorithm model to the cloud server and deploy it.The multi terminal interaction system takes the cloud server as the sign language recognition center,uploads the RGB image information obtained by the terminal through the Internet,and returns the results to the terminal device sending the data for display after the algorithm recognition in the server is completed.After experimental verification and system testing,the multi terminal sign language recognition system built in this paper can effectively recognize and display the sign language image information obtained by the terminal equipment,and has strong practicability.