Research On Language Model Of Teaching Scenes Based On Deep Learning

With the rapid development of big data and deep learning,there is a breakthrough in the research of speech and text-the basic features of language.Language is the most popular way of communication in education and therefore,the research of speech and text is very important for Education + AI.As a basic task,Language Model(LM)could be applied in many Education + AI situations such as speech recognition,photo searching,machine translation and intelligent speech conversation.Up to now language model has performed well in some areas like intelligent customer service,in which training data is sufficient.However,the academia has paid less attention to language model in education.There are two main reasons.One is data barrier.The accumulation of high-quality data in this field is insufficient.Another reason is that,data in teaching scenes has some special features.Firstly,the application of terminology in education,for example,verbal terms are used in the class and the combination of Chinese,English and numbers.Secondly,jargon in certain subjects(chemistry,physics,mathematics and so on)is used during teaching.Next,conversations usually happen in an oral form.The last is the comprehensiveness of education.Teachers often combine knowledge from other area to make students understand better.Consequently,it is imperative to train a high-performance language model,especially for teaching scenes.This paper focused on training the deep-learning based language model with educational data and complete adaptation research to the language model using common field models.First,we trained N-gram statistical language model and standard recurrent neural network(RNN)language model on the educational data,and compared the performance of them.Then we proposed the language model with SCN-LSTM text feature extractor.The SCN layers learned the feature of words relative location by skip connection,and LSTM layers were responsible for deep semantic feature learning of combined word locations and vectors.We compared the performance of them with different extractor on the testing sets.Experimental results showed that our method significantly outperforms models with other extractors and gains a huge improvement of performance under education conditions.Compared with traditional N-gram model and standard RNN model,the perplexity of our model has decreased by 36.9% and 33% in average.In addition,we compared our model with long short-term memory(LSTM)language model,convolutional neural network(CNN)language model and CNN-LSTM language model.The perplexity of our model has respectively decreased by 25.6%,26% and 5.4%.Finally,we completed adaptive research on our model for better generalization in the educational field.After adaptation,the perplexity of our model further decreased by 8% compared to SCN-LSTM language model.