Research On Virtual Human Sign Language Translation Technology Driven By Chinese Voice

In our country,there are more than 20 million hearing impaired people who are unable to communicate normally with people around them in oral.In order to make it easier for them to integrate into ordinary social groups,the virtual human sign language translation technology is studied in this article,so that deaf-mute people can communicate with ordinary people.In this article,speech recognition technology is used to recognize words spoken by healthy people,and then the recognized text is translated into virtual human sign language actions,so that deaf-mute people can communicate with normal people by understanding the virtual human's sign language.This subject has broad application prospects.For what mentioned above,three aspects of speech recognition technology,sign language translation technology and virtual sign language action synthesis technology are studied in this article,as follows:1.Speech recognition technology is the focus of this article.Aiming at the lack of probability estimation by GMM in traditional GMM-HMM,in this paper,based on the DNN-HMM model framework,a 1D-Goog Le Net-HMM acoustic model is proposed.The first step is to train the GMM-HMM model.Then the hidden state sequence decoded by the Viterbi algorithm is used as the annotation for training 1D-Goog Le Net,and finally the 1D-Goog Le Net is trained.The 1D-Goog Le Net-HMM model outputs the posterior probability of each frame corresponding to the hidden state,then calculates the observation likelihood,and then decodes the resulting state sequence.Finally,the algorithm is implemented in the Kaldi framework.The simulation experiments show that the speech recognition effect of this model is better than traditional acoustic models.2.In the end-to-end CTC speech recognition model,for longer audio recognition,it will cause insufficient decoding information.In this paper,an end-to-end speech recognition based on a sliding window attention mechanism is proposed.The sliding window attention mechanism is introduced to concentrate the "attention" of the decoder at each moment on the part of the encoder,and then use the CTC algorithm to calculate the loss between the encoder output and the label.The simulation experiments show that the speech recognition effect of this model is better than the general CTC speech recognition model.3.After speech recognition,the Chinese word order needs to be translated into sign language word order.Due to the differences in word formation and grammar between Chinese and sign language,Chinese vocabulary cannot correspond to sign language vocabulary one-to-one.Therefore,a sign language translation model with attention mechanism is proposed in this article.A Chinese-sign language corpus are built,and then the translation process from Chinese word order to sign language word order is completed by the LSTM encoder and decoder.4.In order to achieve the action demonstration of the virtual sign language word order,Maya modeling software is used to make virtual characters.The virtual human sign language animation is made with sign language vocabulary as the basic unit.A Web GL-based 3D Web application that combines speech recognition,sign language translation and virtual sign language animation are built into a complete system,and finally the Chinese voice to sign language translation process is displayed in the form of a web page,which expands the scope of its application.