Text Matching Based On Graph Neural Network And Attention Mechanism

Text matching aims to judge whether two texts express the same meaning or describe the same events.Text matching is a very important natural language understanding task,which is widely used in search engines,online customer service and other services.Due to the complex structure and rich semantics of text,and the complex interaction between text pairs,text semantic matching has become a challenging task.Therefore,aiming at the problems existing in the current text matching task,this thesis proposes a short text representation model,short text matching model and long text matching model.The research work carried out in this thesis is as follows:(1)Aiming at the problem that the current short text representation methods lack spatial information such as syntactic dependencies,which leads to the decline of short text representation quality,this thesis proposes a short text representation model based on heterogeneous graph and Graph Attention Network.This model first extracts the heterogeneous graph structure of the short text,then uses syntactic dependency,point-wise mutual information and context information to capture the spatial features of the text.This model also uses Graph Attention Network to obtain the embedded representation of heterogeneous graph nodes.Compared with the temporal modeling text representation methods,this model combined with spatial information can better extract effective features.It is superior to the temporal modeling text representation methods,and has achieved very competitive results in short text matching tasks.(2)Aiming at the problem that the existing short text matching methods do not consider the importance of the relative distance of words when modeling the semantics of short text,which leads to the problem of deviation in the modeling of the core semantics of text pairs,this thesis proposes a short text matching model based on distance-aware self-attention and multi-level matching.This model integrates the distance information between words in short text into the self-attention mechanism to focus on those words closer to each other.At the same time,the interactive attention mechanism is used to capture the interactive relationship between two texts,and then the multi-level similarity modeling is used to further improve the short text matching effect.This model achieves 86.8%,84.1% and 89.2% accuracy on short text matching datasets LCQMC,BQ and Quora respectively.This result is better than the existing published short text matching model,and its network parameters and time complexity are lower.(3)Aiming at the problem that existing long text matching methods are difficult to model the complex interaction between long text pairs,and difficult to fully extract the core topics,resulting in the poor performance of long text matching,this thesis proposes a long text matching model based on Merged Keyword Graph and multi-head Graph Attention Network.Firstly,this model proposes the method of modeling two long texts into the Merged Keyword Graph,which models the spatial structure of long text pairs.Then it embeds the local matching features of long text pairs into the merged keyword graph.Finally it uses the multihead Graph Attention Network to extract the features,so as to further improve the performance of long text matching.This model obtains 82.4% and 89.8% F1 values on long text matching datasets CNSE and CNSS,respectively.These results are better than the existing advanced long text matching models.In summary,the short text representation,short text matching and long text matching models proposed in this thesis have certain research value and application value,which can provide more efficient and comfortable services for applications based on text matching methods.