Reinforcement Learning On Named Entity Recognition Task

Natural language is one of the main manifestations of massive data on the Internet,and it is the most common way for human to acquire knowledge.The unstructured and the semantic diversity of natural language itself lead to difficulty in extracting textual information into structured knowledge,which has become one of the key issues in Natural Language Processing(NLP).Among them,named entity recognition plays a fundamental and important role in Information Extraction(IE).Existing models for named entity recognition can be divided into statistical based method and deep learning based method.The former requires manually extracted feature templates to perform pattern matching,and the latter mainly performs end-to-end automatic tagging through deep neural networks.In recent years,the development of deep learning has greatly enhanced the expressive ability of reinforcement learning and it has achieved or even surpassed human level in games and control systems.Under this background,this thesis studies the application of reinforcement learning on named entity recognition task,and the main work is as follows:(1)This thesis models the process of named entity recognition as a Markov Decision Process(MDP).In order to model the globality of tagging,it uses neural networks to model the semantic information of state,which is different from previous work.Through the experiment results achieved by the policy gradient algorithm,the shortcomings of policy gradient in solving MDP is summarized(high variance and low experience utilization).Then the core problem of this thesis is proposed:How to solve the MDP model which is able to find the globally optimal tagging path?(2)To address the problem in(1),this thesis proposes a novel reinforcement learning based model,referred to as MM-NER.MM-NER is the first work to apply the Monte Carlo Tree Search(MCTS)enhanced MDP model on named entity recognition task.In MM-NER,a policy value network is designed to guide the tag assignment and predict the whole sentence tagging accuracy.The policy and value are then strengthened with MCTS,which simulates and evaluates the possible tag assignments at the subsequent positions and output a globally aware search policy.In inference stage,MM-NER enjoys the advantage of reducing time complexity to O(T|A|),which is more efficient than Viterbi decoding whose time complexity is 0(T|A|2).(3)To better solve the problem of polysemy in named entity recognition,the pre-trained Bert language model is introduced to further enhance the dynamic semantics representation of MDP state.The experimental results based on two named entity recognition benchmarks demonstrated the validity of MM-NER on named entity recognition task thanks to the exploratory decision-making mechanism introduced by MCTS.