Research On Automatic Text Summarization Based On Two-level Neural Networks

With the rapid development of the Internet,various types of text data have increased dramatically,bringing amounts of information to users,include information overload.The amount of text information has far exceeded the limits of manual processing.Automatic text summarization can automatically extract a given document or multiple documents,refine and summarize the main points of the information,and finally output a short,readable summary,saving users a lot of reading time,to achieve rapid access to effective information and improved information service capabilities of information systems.This paper aims to extract the summaries in two stages according to the sequence-tosequence framework via neural network modeling.Firstly,the encoder is used to obtain the text content,and the text features are extracted through a large amount of data training,and the attention mechanism is constructed to improve the understanding of the text information,and then the decoder is used to extract the sentences to form a summary.We mainly completed the work as follows:(1)We propose a self-interactive attention-based summarization(SIAS)model.Based on the neural network model of the encoder-decoder framework,this paper extracts the document summary in two stages: firstly,the encoder is used to encode the document content,mining and extracting the text features,and refining the information contained in the document.Then we use the decoder to filter the extracted features and information,and extract important sentences with significant information as the summary.In order to obtain more abundant document information,we construct a vector representation of the document through a hierarchical structure,and use the attention mechanism in constructing the sentence representation from the word representation to increase the extraction of the relationship between the words within the sentence,thereby obtaining better document representation for extracting important sentences.(2)We propose an attentive encoder-based summarization(AES)model.We propose to improve the encoder via enhancing the ability of the extraction and understanding of the document information.In the process of obtaining sentence vectors from word vectors,a convolutional neural network is used which is more convenient to train and capable of extracting multidimensional features.We use attention mechanism to analyze the information interaction at the sentence level to obtain the interactive information between different sentences,find the connection between them,understand the meaning and distinguish the importance.After using a unidirectional recurrent neural network to construct a document encoder resulting in Uni-AES model,we consider that a bidirectional recurrent neural network can associate a sentence with surrounding sentences to obtain more features,then we construct Bi-AES model.In order to verify the validity of the proposed models,we validate and analyze the model on a public dataset constructed from CNN News.The experiments show that:(1)The SIAS model has a significant improvement on short documents,and when the generated summary is shorter,the effect is more prominent,which shows that the SIAS model can extract the key words from the document.(2)The AES model has a obvious effect on the summary of the shorter documents,which is more prominent than the SIAS model,especially the Bi-AES model.We obtain improvements of up to 7.41%,23.68%,13.03%,6.41% and 7.59% in terms of ROUGE-1,ROUGE-2,ROUGE-3,ROUGE-4 and ROUGE-L,respectively,over a relevant stateof-the-art baseline.When the generated summary is longer,the model can get better summaries and extract more diverse information.This shows that the interrelationship between sentences can help to understand the meaning of the document more comprehensively,and our attention mechanism also plays a role in mining the relationship between sentences.