Research And Application Of Related Techniques For Text Summarization Based On Deep Learning

The arrival of the era of Big Data is accompanied with the risk of information explosion.How to quickly and accurately obtain the required information from the Internet with mass of information has become an urgent problem to be solved.The automatic text summarization is an effective solution to information overload which can extract the important content from the text and generate concise description.In recent years,the rapid development of Deep Learning brings new ideas to Automatic Text Summarization,and the abstractive summarization method emerges at the right moment.The text generated by this method is more readable and easy to understand.At present,the text summarization method based on the deep neural network mostly adopt the encoder-decoder framework where the encoder generates the semantic representation of the source text and the encoder generates the continuous readable summary sequence.However,this method has some problems such as generating out-of-vocabulary words and redundant sequences,and even making insufficient original semantic representation.In order to solve the aforementioned issues,this thesis makes an exploration of the text summarization methods based on deep neural network,put forward a sequence-tosequence abstractive text summarization model based on a random beam search and also an abstractive text summarization model based on language model.The proposed methods are tested on different data sets and the experimental results has proved the effectiveness of the models.The main contents of this thesis are divided into two parts,as follows:(1)Designed and implemented a sequence-to-sequence abstractive summarization model based on a enhanced semantic and improved beam search.The main contents includes: A hybrid encoder structure,which captures the short-range context information of the original text through the gated convolutional networks to obtain the context semantic representation,and adopts a bidirectional recurrent neural network(BiRNN)to learn long-term dependency and temporal information;A Random Beam Search Algorithm,this method introduce randomness into traditional Beam Search Algorithm,which ensures the diversity of decoding sequence in that Random Beam Search Algorithm adopts k candidates randomly sampled in a dynamic confidence space instead of the k candidate items from top-k sampling in each decoding step;A source text keyword re-ranking algorithm,use the tf-idf weighting to score every word in the input document,combine with the list of attention distribution vectors to evaluate the quality of candidate sequence,instead of only choosing the one with the highest probability as the standard beam search does.(2)Designed and implemented an abstractive summarization model based on language model.This method abandons the traditional sequence-to-sequence framework,instead of modeling the abstractive summarization task as a language model task and explored the feasibility of the approach.The main contents includes: We use the decoderonly Transformer framework to model this tasks,fine-tunning on pre-training language model of GPT,then improved the mask method of Transformer,and finally made analyses and comparison on the results of the experiment.