Research On Few-shot Learning ERE With Self-attention Mechanism

In the context of the information explosion,a large amount of redundant and repetitive information floods people’s lives.How to extract high-quality and accurate information from a large number of unstructured or semi-structured information is one of the reasons for the rapid development of information extraction tasks.Information extraction is widely used in tasks such as question answering systems and digital libraries,and entity relationship extraction is one of its important subtasks.Traditional and deep learning-based entity relationship extraction methods have achieved high accuracy in practical applications,but they are mainly concentrated in specific fields.When faced with entity relationship extraction tasks that are not easy to obtain large-scale supervised training data sets,Traditional methods are gradually failing to achieve the desired effect.Therefore,this paper provides new ideas for the extraction of entity relationships under the condition of resource scarcity through the small sample learning method.At present,small-sample learning research is mainly focused on the image field.Aiming at the diversity of text in the entity relationship extraction problem,this paper proposes a prototype network model fused with self-attention mechanism to solve the over-fitting problem and noise influence of small-sample learning of text.1)This article uses Prototypical Networks to highlight the problem of entity relationship extraction with a lack of data resources.The prototype network model encourages the ability to learn fast learning from previous experience and quickly promote it to new areas.The prototype network has achieved good results in the task of small image samples.Similar to the ordinary prototype network,this paper uses CNN neural network to embed all instances into the support center,calculates the instance weight through the sentence-level attention mechanism,and calculates the prototype of each relationship based on it,and finally compares the feature-level attention score,cliff The degree factors are combined to classify the query instances by measuring the standardized Euclidean distance between the query instance and the relationship prototype.2)This paper proposes a prototype network that integrates the self-attention mechanism,which integrates the multi-head self-attention mechanism into the model pretraining process to highlight the key feature information in the sentence.Aiming at the problem that the small noise of the support set in the learning of small samples of text may cause a huge deviation of the relationship prototype.The self-attention mechanism effectively eliminates the offset points in the data set,making model learning more focused.The addition of the self-attention mechanism also effectively reduces the impact of noise,highlights the key feature information in the sentence and the important dimensions in the feature space,and increases the accuracy and robustness of the model.In the experimental part of the thesis,a comparative experiment on 8 models including the baseline model is conducted on the Fewrel data set,which verifies the effectiveness of the entity relationship extraction model proposed in this paper,which integrates the self-attention mechanism,and predicts the model by incorporating the selfattention mechanism.Compared with other models,the optimized model after training has higher accuracy on the task of entity relationship extraction.