Research On Named Entity Recognition Based On KL-HMM

With the development of information technology and the application of artificial intelligence,more and more attention has been paid to natural language processing related research.Named Entity Recognition is a crucial step in the early days of natural language processing.It recognizes that entities such as time,number,person name,location name and organization name in the corpus text play an important role in many fields of research.In the actual recognition process,a lot of text is not a closed set,but an open set.In an open set of text different translated words appear for the text recognized by the transliteration entity,resulting in a high error rate in the recognition.The thesis adds the mechanism of KL divergence-based into the model,learns the differences of transliteration entities that may appear in other corpus,and then calibrates the parameters.In addition,to prevent over-fitting of Kullback-Leibler divergence-based hidden Markov model,the thesis verifies and analyzes the adaptability of different corpus,and improves the efficiency of recognition.Finally,it verifies the actual problem,writes named entity recognition program,and tests data for comparison.The experimental verification index uses the comprehensive index F-measure of accuracy R and recall P.In the experimental comparison,the KL-HMM model is 15.44%more accurate in transliteration of person name than the original HMM model,the location name is increased by 29.21%and the organization name is increased by 25.67%.However,after each layer of parameters is calibrated,it will have a slight impact on Chinese entities.The experimental results improved by 1.71%,4.66%and 0.41%respectively in the person name,location name and organization name.F-measure by using the adaptive adjustment increased 0.38%,0.36%and 0.25%.It verifies that the KL-HMM method can improve the recognition performance of transliterated named entities well.