| With the popularity of the Internet,automatic text summarization technology has been widely used in the fields of news headline generation,news summary generation,and search result preview.As an important branch of natural language processing,automatic text summarization refers to the automatic summarization and refinement of the main content and meaning of the specified document to form a summary.Among the many automatic text summarization techniques,the graph-based TextRank algorithm has attracted much attention because it does not require in-depth linguistic knowledge and a specific scope or language corpus,and has been widely used.On the other hand,with the development of online media,online news has become the mainstream medium for people to obtain information.With the increase in the number of Internet users browsing or participating in the review,online news may evolve into a social hot event.It is especially important to accurately determine the news popularity to help people get hot news and gain insight into social hotspot related information.In this paper,the TextRank algorithm is deeply studied and an improved TextRank algorithm is proposed.Firstly,the process of constructing language network for TextRank is improved,including:this paper improves the BM25 algorithm and compares it with word co-occurrence,TFIDF/COS and native BM25 algorithm.The improvement of the TextRank algorithm on Rouge1,Rouge2,and Rouge3 evaluation indicators verifies the effectiveness of the improved BM25 algorithm compared to the other three similarity calculation methods.Then based on the novelty of the abstract,this paper uses the redundant processing technique to correct the importance of the sentence in the TextRank calculation result,so as to obtain a summary with less redundant information and relatively independent sentences.Experimental results on actual data verify the effectiveness of this redundant processing technique.In order to judge the popularity of online news,this paper also proposes a heat calculation model.The model combines multiple elements of online news to measure the heat of news from multiple perspectives.Relevant elements include:news page views,comment volume,number of similar events,news release time,etc.,in which similar events are obtained using SimHash-based algorithms.On the basis of requirement analysis,complete the architecture design,function design and database design of the news automatic summary system based on event heat,complete the detailed design of each functional module,and verify the system meeting the design requirements through functional and safety test.Through news automatic summary system based on event heat,users can quickly understand hot news. |
