The Feasibility Reseach Of Evaluation And Comparison Of Natural Science Papers In Different Disciplines

ObjectivesTo construct a new comprehensive evaluation system to assess the impact of academic papers in different fields of natural science,through systematic comprehensive evaluating methods,mathematical statistical methods and scicntomctric methods.This system makes the evaluation and comparison of the impact of academic papers in different fields of natural science more objective,fair and stable than thc traditional indices such as total number of other-citation and journal impact factor,so as to scientifically evaluate the impact of academic papers and provide reference for the assessment of research performance.MethodThis research use the representative papers of Nobel laureates in physiology or medicine 2001-2006 as benchmark,and the related articles in the same field as sample set,to inspect the comprehensive evaluation system based on comprehensive evaluation theory and bibliometric methods.This system consider different backgrounds of academic papers such as publication time,disciplinary development,research focus, technical means and so on to reflect the impact of academic papers comprehensively, using 8 indicators including total number of other-citation,maximum annual other-citation,journal impact factor,annual average other-citation,other-citation in three-year after publication,and indicators derived from last three indicators,that is, ratio of each indicator to its average in same year.The 8 indicators of sample sets were processed by mathematic methods,and Technique for Order Preference by Similarity to Ideal Solution(TOPSIS) and factor analysis were employed to obtain synthetical evaluation values.Results of the two methods were compared to that of traditional evaluation method using other-citation, impact factor and annual average other-citation,to verify the objectivity and justice of the comprehensive evaluation system.Besides,articles in the sample set were clustered by means of SOM neural network clustering and K-MEANS under the comprehensive evaluation system.The results were compared to that of TOPSIS and factor analysis to verify the stability of this system.ResultsResults of TOPSIS and factor analysis under the comprehensive evaluation system were compared to that of traditional indicators.When compared within the same field,rank of the former show positive correlation with that of the latter,not change the general trend of traditional evaluation;when compared among multidisciplinary fields,the new system can further raise the visibility of benchmark papers on the base of balancing the interdisciplinary discrepancy.Synthetical evaluation values obtained through TOPSIS and factor analysis present significant positive correlation,and show conformance with the clustering results of SOM and K-MEANS.ConclusionThis research avoided bias of indicators such as other-citaiton and journal impact factor in common use as evaluation criterion for impact of interdisciplinary papers,and construct a multidimensional comprehensive evaluation system more inclined to be fair, objective,stable and scientific.When applied on the assessment and comparison of papers within the same field,the synthetical evaluation value obtained through TOPSIS and factor analysis under this system not only showed conformance with the evaluation result of traditional indicators,but also raised the visibility of high impact papers,and in good accordance to the clustering results of SOM and K-MEANS.When applied on the comparison among different fields,the synthetical evaluation value avoided the bias and distortion of evaluation result of traditional indicators induced by the interdisciplinary discrepancy on the base of distribution similarity,besides,it can keep the general trend of traditional evaluation and further highlighted papers with high impact.This comprehensive evaluation system can be used as an important and scientific reference for comparison and evaluation of impact of interdisciplinary papers.