Research On Event Extraction Technique Based On Pre-trained Language Model

Event extraction is one of the most challenging and widely used tasks in natural language processing,which can support the development of artificial intelligence application fields such as knowledge graphs,knowledge Q&A systems,financial risk monitoring,and public opinion monitoring.In recent years,pre-trained models based on Transformer such as BERT,T5,and BART can handle various tasks in the field of natural language processing well,but their effectiveness in tasks related to event extraction still needs to be improved.First,the existing dataset in the event extraction has a small size and significant data imbalance,making it difficult to train a model with good performance using conventional methods.Second,the event extraction task is more complicated to model,and the interdependence among event triggers,event arguments and event types make model extraction difficult.Finally,there is a certain gap between the pretrained model and the downstream event extraction task,and it is difficult for the existing methods to exploit the performance and potential of the pre-trained model.Therefore,to address the above problems faced by event extraction techniques,this paper carries out research on event extraction techniques based on pre-trained models as follows:(1)An event extraction framework based on BERT and event annotations is proposed.The framework consists of an event classification module based on a bilateral branching network and an event extraction module based on BERT and event annotation.The bilateral branching network-based event classification module uses reverse sampling and adaptive learning strategies to alleviate the data imbalance and improve the event classification effect.The BERT and event annotation based event extraction module uses BERT as the backbone network.It combines event annotations in the input to give the model rich prior knowledge,thus improving its performance.Experiments prove that the proposed method in this paper shows high extraction accuracy on the benchmark dataset.The experiments,such as simultaneous ablation and multi-event extraction,prove the method’s effectiveness.(2)A retrieval-enhanced event argument extraction framework based on BART is proposed.The framework consists of an example retrieval module based on S-BERT and graph convolutional neural network and a BART-based event argument extraction module.The primary purpose of the demonstrate retrieval module is to build a demonstration store and retrieve demonstration by stitching the semantic vector computed by S-BERT and the structural vector computed by GCN to ensure that the retrieved examples have both semantic similarity and event structural similarity.The BART-based event argument extraction model uses BART as the backbone network and incorporates examples in the input.The demonstration serves as a guide to the model in the form of hints,releasing the excellent analogy ability of the pre-trained model and thus improving the model performance.Experiments prove that the proposed method in this paper exhibits high accuracy of thesis element extraction on the benchmark dataset.Meanwhile,experiments such as few shot validity and retrieval similarity prove the effectiveness of each module.