Research On Evaluation Of Academic Papers Based On Peer Review Text Analysis

In order to establish a scientific evaluation system,China advocates the use of evaluation methods with peer review as the core,bibliometrics as the auxiliary,which combines qualitative and quantitative evaluation methods.The peer review texts of academic papers are difficult to obtain and use.At the same time,the existing quantitative evaluation methods are inherently dependent on simple counting and lack of in-depth related research on text content,which brings difficulties to the integration of the two methods.With the advent of the era of big data,the open science movement has provided opportunities for exploring peer review,and combined with the rapid development of artificial intelligence technology,it has provided great convenience for the intelligent analysis of massive texts.In view of this,this paper takes ICLR conference papers and peer review data of the international conference in the field of computer as the research object,explores open peer review mainly,uses artificial intelligence technology to analyze its sentiment tendency and text effectiveness,and combines the existing bibliometric indicators to construct an evaluation of academic paper profiles.The main work of this paper is as follows:First,we mine the fine-grained sentimental tendencies of evaluations in peer review texts to judge the degree to which reviewers support or oppose the paper.The paper evaluation words in the peer review texts are intelligently identified by the opinion mining method,so the paper evaluation words are used as the sentiment words in the sentiment dictionary,and then the positive and negative sentiment scores of the entire peer review text are calculated.The results of the analysis showed that papers with higher recommendation scores by reviewers contained more positive sentiment and less negative sentiment.Positive and negative sentiment scores are used as peer review sentiment representation information in the evaluation of academic papers.Second,distinguish the effectiveness of the peer review texts in the review stage of the paper,that is,whether it has an impact on the final acceptance result of the paper,so as to judge whether the evaluation information in the peer review text has reference value.The review effectiveness of peer review texts is automatically idistinguished by defining the concept of review effectiveness and utilizing sentiment analysis methods.This paper finds that the negative review comments and the review comments corresponding to the accepted papers are more effective.Review effectiveness is used as peer review reviewer effectiveness information to be used in the evaluation of academic papers.Finally,with the paper profile as the carrier,the method based on comparative learning is used to evaluate the academic paper.The paper profile integrates peer review information,bibliometric evaluation information and paper topic information.The peer review information includes five indicators: peer review positive sentiment score,negative sentiment score,review effectiveness,reviewer recommendation score and text length.And the paper profile method has a good evaluation visualization effect.On this basis,a scientific,comprehensive and intelligent evaluation of the quality of academic papers is carried out through the paper profile classification method based on C2 L.