Research On Named Entity Recognition Based On Attention Mechanism

Named entity recognition is one of the key technologies of natural language processing,which can effectively identify various types of entities in information,and also plays an import role for other downstream tasks to extract main information.Traditional named entity recognition methods mainly include methods based on manual rules,unsupervised learning methods based on clustering,and supervised learning methods based on feature engineering.These methods place high demands on manual design rules or features,and the traditional models often cannot be directly transferred to new datasets or domains.The named entity recognition model based on deep learning can autonomously discover the features required for the task,and compared with the traditional method,the effect of model recognition has also been greatly improved.However,named entity recognition methods based on deep learning also have shortcomings: first,most studies only use a single word feature for model training,while ignoring the importance of other semantic features to enhance the recognition ability of the model;second,some researchers used the context information generated by the same word in the entire document to assist model prediction,but these studies ignore the problem of whether other contextual representations of the current word are beneficial to the current model training.This dissertation proposes improvements for the above two problems,as follows:For the problem of only using single word feature,this dissertation proposes a multifeature named entity recognition model based on attention mechanism.Firstly,the initialized character vector is concatenated with the length vector of each word and the position vector of each character in the word as the input of CNN to extract features.Secondly,the effect of named entity recognition can be improved by using attention mechanism to integrate affix features and part-of-speech features learned by multi-task.Then,aiming at how to effectively utilize the contextual information of the same word in the document,this dissertation proposes a document-level Chinese named entity recognition model based on the attention mechanism.First,we utilize the label embedding vector to classify the information in the document.Second,an attention matrix is constructed based on the cosine similarity between the classified information and the target label vector.Finally,document-level features are computed using the attention matrix and selected contextual information for model prediction.In this dissertation,the proposed model is experimentally analyzed on different datasets.For the multi-feature model,the experimental results show that the model achieves competitive F1 scores on the CONLL-2002 Spanish dataset,CONLL-2003 English dataset and Ontonotes5.0 English dataset,respectively.For experiments on document information selection,the results show that the model achieves competitive results on two Chinese datasets,MSRA and Resume,respectively.