A Study On Neural Network Language Model Based On Russian Military News

The language model is the main form of language knowledge.The research results of the language model will help to improve the readability and accuracy of the results of many natural language processing systems such as automatic summarization,speech recognition and machine translation.In order to intelligently handle the intelligence of the Russian army,the study of language model of the Russian army has important academic exploration significance.Moreover,with the maturity of deep neural network technology in recent years,the neural network language model shows superior performance to traditional language models in solving data sparseness and long-term dependence.Therefore,this paper uses the neural network-based training method to construct the military Russian language model to study the effect of the neural network language model in the military Russian field.This study attempts to mine the news text data in the Russian military field.The neural network-based training method is used to construct the military Russian language model.From the two aspects of perplexity and number of parameters,Comparing the performance of two military language models based on neural networks and traditional statistical methods,comparing the perplexity of two Russian neural network language models based on general domain corpus and Russian military news corpus on military Russian text,and computing pre-training language Model BERT's perplexity on military Russian text.In the experiment,the news text corpus collected from the Russian Ministry of Defense website were used as military Russian data sets,which were divided into train set,validation set and test set.Firstly,the traditional n-gram model and the long-term short-term memory structure(LSTM)neural network model are used to train the Russian language model on the train set,and the validation set's perplexity is prevented by the early termination method to prevent the neural network model from over-fitting.And then calculate the confusion on the test set.The experimental results show that compared with the traditional n-gram model,the number of parameters of the Russian military news neural network language model has increased by 93%,and the perplexity has decreased by 36.3%.Compared with the neural network language model of Russian corpus training in the general field,the model's perplexity for Russian military news is reduced by 92%.In addition,the Google open source pre-training language model BERT was used to calculate the perplexity on the Russian military news test set,and found that even with the Rubent model for fine-tuning Russian,the calculation of the perplexity in Russian military news is still not satisfactory.However,the military Russian language model based on LSTM is superior to the traditional model,indicating that the use of neural network-based training methods to construct military Russian language models can help to improve the effectiveness of various military Russian intelligent processing tasks.