Research And Implementation Of Fine-Gained Entity Typing Based On Pre-Traing Model

With the development of the Internet,a large amount of unstructured data will be generated at any time.So it is necessary to use information extraction technology to store and use these data.Fine-grained Entity Typing is one of the key technologies for information extraction,providing support for such tasks as entity links,knowledge maps,and so on.The goal of the Fine-grained Entity Typing task is to categorize candidate entities into the correct categories based on the candidate entities and their contexts provided.Unlike traditional Named Entity Recognition,Fine-grained Entity Typing is a multi-label and multi-classification task.It extracts features closely related to the Fine-grained Entity Typing task,determines the quality of classification results,and also affects downstream tasks.Pre-training model has been widely used in many fields of Natural Language Processing since it was proposed by researchers due to its strong characterization ability.Pre-training model is mainly applied to specific tasks through feature extraction and finetuning.In feature extraction,this thesis uses the word vector of ELMo pre-training model to replace the traditional word vector.In fine-tuning,this thesis uses four pre-training models:BERT,XLNet,RoBERTa and LUKE to classify fine-grained entities,and fine-tunes the RoBERTa model by using entence-transformer and contrast-based learning.The main contributions of this thesis are as follows:1.A Fine-grained Entity Typing method based on pre-training word vectors and hyperbolic space is presented.In this thesis,ELMo word vector is used for Fine-grained Entity Typing task.Fine-grained Entity Typing tasks have too fine categories and hierarchies between categories.This can be solved by using hyperbolic space.Experiments verify that the performance of Fine-grained Entity Typing using ELMo word vectors and hyperbolic space is better.2.A Fine-grained Entity Typing method based on fine-tuning pre-training model is presented.Fine-grained Entity Typing is done by fine-tuning BERT,XLNet,RoBERTa,and LUKE.Fine-tuning RoBERTa based on sentencetransformer and comparative learning to obtain sentence embeddings with semantic information for Fine-grained Entity Typing.3.Experiments on UFET standard datasets demonstrate that both pretraining term vectors and fine-tuning based methods can improve the performance of Fine-grained Entity Typing.