Research On Named Entity Recognition Based On Attention Mechanism

The task of named entity recognition is an important cornerstone of many downstream tasks in the field of natural language processing,and its goal is to determine the boundaries of entities in text and accurately classify entities.With the continuous growth of computer computing power,the research on named entity recognition based on deep learning has developed rapidly,but there are few researches on difficult tasks such as Chinese named entity recognition and nested named entity recognition.For these scenarios,traditional semantic encoders are difficult to take into account both global and local information,cannot capture key information,and cannot effectively extract text semantics.The attention mechanism and Transformer architecture can globally model the text,capture key information,and improve the effect of named entity recognition.Therefore,this paper combines the attention mechanism and the Transformer architecture to carry out research on the task of named entity recognition.The main work of this paper is as follows:1.This paper proposes a feature fusion model based on the attention mechanism for Chinese named entity recognition.In order to obtain global semantic information and direction information at the same time,the attention mechanism is used to dynamically fuse the Transformer encoder and the bidirectional long-term short-term memory network(Bi LSTM).Specifically,the Transformer encoder is improved by using relative position encoding and modifying the attention calculation formula,then the improved Transformer encoder is used to provide global semantic information,and Bi LSTM is used to capture direction information.On this basis,combined with the attention mechanism to dynamically adjust the weights,the global semantic information and direction information are deeply integrated to obtain richer contextual features.Finally,the conditional random field is used for decoding to realize the sequence prediction of entity labeling.In addition,for the problem that traditional word vector methods such as Word2 Vec cannot represent the polysemy of words,the Ro BERTawwm pre-training model is used as the embedding layer of the model to provide character-level embedding,and the model can also obtain more contextual semantic information and lexical information.Enhanced entity recognition.The experimental results show that the F1 value of this method on the Chinese named entity recognition dataset Resume and Weibo datasets reaches 96.68% and 71.29%,respectively,which is better than the traditional named entity recognition method.2.This paper proposes a gated attention unit span classification model for the unified recognition of flat and nested entities.Based on the span approach,the model converts the traditional sequence labeling problem into a span classification problem.The BERT pre-training model is used to obtain the context vector representation,and the bidirectional long-short-term memory network(Bi LSTM)and the gated attention unit are used as the encoding layer to effectively extract the context features to obtain a high-quality span representation.Aiming at the problem that the span-based method ignores the internal information of the entity,the relative position encoding of the rotation is used to provide relative position information for the span representation,and to improve the position awareness of the attention mechanism on the interior of the span.Finally,use the attention mechanism to score each span,and use Softmax to classify after obtaining the span score matrix.The experimental results show that the F1 value of this method reaches 96.54% and 81.02% respectively on the Chinese non-nested data set Resume and the English nested entity data set GENIA data set,which proves that the gated attention unit span classification model has a good recognition effect.