| Under the leadership of the Ministry of Education’s "Breaking the Five Focuses" action,the voices opposing "SCI supremacy" and "Evaluating Papers by Journals" are increasing.The number of papers continues to grow rapidly,bringing opportunities and challenges to researchers,journals and research institutions.The evaluation of journal papers has become an important link in scientific research.The evaluation results help researchers quickly find the right research in the mass of papers,and effectively grasp the knowledge system and research frontier.It is helpful for journals to reflect on the contribution and quality of the published papers,and to find the difference and develop the development path.It is helpful to support many decisions of research institutions,such as project application and evaluation,subject acceptance and performance appraisal.The academic evaluation after "Breaking the Five Focuses" is worth thinking about.There are many value orientations in paper evaluation.The evaluation oriented by paper contribution and quality is the main research direction at present,and the influence and innovation become the key of evaluation.However,in the influence evaluation,there is a lack of consideration of the strength of citations,and excessive reliance on external indicators such as downloads and citations of the evaluated literatures is easy to cause academic misconduct.The evaluation results are one-sided,and the accuracy of the evaluation needs to be improved.In the innovation evaluation,as far as the evaluation is concerned,the unique degree of subject words recognized by machine methods such as natural language processing is more the evaluation of the novelty of the paper.There is a gap between the novelty of the paper and the innovation of the paper.However,to evaluate innovative sentences,it is necessary to find innovative sentences in the full text,which is a complicated process.Peer review and other manual methods have the problems of strong subjectivity,high cost and low efficiency.Citation network based innovation evaluation method is low cost but easy to ignore the substantial difference of innovation knowledge.According to the above situation,this study attempts to evaluate the contribution and quality of the paper scientifically and effectively from a longer and deeper perspective.Fully reveal the performance of scientific research results so as to better guide scientific research work.When evaluating the influence of papers in the academic communication system,an evaluation index system of paper propagation is constructed.Referring to the theory of Niche Ecostate-Ecorole Theory,the paper collection is regarded as an ecosystem.The "Ecostate" of the evaluated documents is the influence measured by the use of papers,and the "Ecorole" is the influence measured by the use of citation literature,so as to objectively evaluate the strength of citation.Ecostate is the basis of Ecorole,Ecorole is the extension of Ecostate,combining EcostateEcorole to evaluate the propagation.Objective CRITIC method and expert subjective weight method were adopted to weigh each index of the evaluation index system of propagation.The empirical study found that compared with traditional influence indicators,the evaluation effect of propagation is better,including more information,higher differentiation and better sensitivity,and the evaluation results can be extended to a group of papers,that is,to complete the evaluation of journals and authors.When evaluating the innovation level of a paper,an innovation evaluation matrix composed of innovative nature and innovative degree is constructed.There are four kinds of innovative nature,namely,topic selection innovation,theoretical innovation,method innovation and conclusion innovation.There are five kinds of innovative degree,namely,pioneering type,original type,improved type,evaluation type and use type.With the help of citation network,matrix matching is carried out according to the semantics of specific citation sentences in citation literature,and quantitative evaluation is carried out by using five-part method and fuzzy comprehensive evaluation,which opens up the "bottom-up" evaluation path compared with peer review.The innovation evaluation has reached the evaluation effect of scientific,reasonable,objective and fair,and dynamic change.Taking the propagation and innovation into consideration,cluster analysis and Boston matrix are used to observe the correlation or divergence between the two,and distinguish the star paper with high propagation and high innovation,the cash cow paper with high propagation and low innovation,the problem paper with low propagation and low innovation,and the dog paper with low propagation and low innovation.It is suggested that we should accelerate the development of star papers,evaluate and investigate cash cow papers,encourage and support problem papers,and control and adjust dog papers.In addition,suggestions on the management and development of the four kinds of journal papers are put forward to the researchers and research institutions.This study is a preliminary exploration of a comprehensive,scientific and objective paper evaluation scheme.The research idea is different from the existing research.Theoretically,Niche Ecostate-Ecorole Theory is introduced to help academic evaluation research.From the perspective of the paper,a contribution-oriented propagation evaluation system and a contribution and quality-oriented innovation evaluation matrix are constructed,and the comprehensive performance evaluation of the paper is completed by combining propagation and innovation.In terms of methods,the data characteristics and method characteristics of 9comprehensive evaluation methods are reviewed,the CRITIC method and expert method are used to empower each indicator of the propagation evaluation system,and the innovation score of the paper is obtained by fuzzy comprehensive evaluation and five-point method,providing reference for the subsequent research to use appropriate methods according to the situation.This paper presents the process of using cluster analysis and Boston matrix to deal with paper classification problems..In conclusion,both the evaluation of propagation and innovation have achieved a good evaluation effect.The paper classification research has been innovatively carried out to achieve the performance evaluation combining quality.The evaluation results can benefit many parties,and it is hoped that this study can provide a new angle and a new scheme for future research on academic evaluation. |
PDF Full Text Request