Mongolian Named Entity Recognition Integrated Language Model And Attention Mechanism

Mongolian named entity recognition is a fundamental task for Mongolian natural language processing;its recognition effects will directly influence the performance of subsequent processing tasks or relevant application.Currently,the research work on the in-depth learning of Mongolian named entity recognition is relatively rare,meanwhile the performance of traditional machine learning method is unsatisfactory.In view of the excellent performance of the deep learning method in named entity recognition task of other language,I use BLSTM-CRF as the baseline model,and then combine the word formation characteristics of Mongolian for further improvement.I propose Mongolian named entity recognition framework based on language model and attention mechanism.The main work of this dissertation is as follows:Firstly,due to the scarcity of annotated corpus in the Mongolian named entity recognition,i make use of the language model to capture the linguistic features in the Mongolian texts,such as grammar and syntax.And the extracted linguistic features are introduced into the BLSTM-CRF model to augment the features learned from the limited annotated corpus.The newly proposed model was the MNER-LM model.The experiment showed that the average F value of the MNER-LM model has increased by 0.86 compared to the baseline model,and it has great robustness on insufficient training data.Secondly,in the input layer of the BLSTM-CRF model,the information expression ability between the Mongolian morpheme vector and the character level vector is unbalanced.So i propose MNER-ATT model which use attention mechanism to combine two feature vectors dynamically.It can enhance the information expression ability of the input layer of the model,and alleviate the impact of imbalance.In the experiment part,its F value has increased by 0.5 compared to the baseline model,and the feasibility of the MNER-ATT model is verified.Finally,i propose the MNER-LM-ATT model.In this model,i use attention mechanism to improve the coding layer of MNER-LM model,mitigated the influence of the asymmetry of feature vector information.Its average F value has increased by 1.13 compared to the baseline model.The experimental results indicate that the MNER-LM-ATT model could efficiently improve the performance of the system.