The Impact Of Subject Classification Scheme On Field Normalization Effects

In the actual scientific research evaluation process,the number of citations of a paper is usually used as an important index to measure the scientific research influence of the paper.However,there may be differences in the citation habits and citation evolution laws of different disciplines.The number of citations of papers in different disciplines cannot be directly compared.However,comparing the influence of papers in different disciplines is an unavoidable topic in the actual scientific research evaluation process.For example,in the promotion of professional titles,university rankings,and interdisciplinary evaluations,the comparison of the influence of papers in different fields will inevitably be involved.In order to achieve the goal of interdisciplinary comparability of the influence of papers in different disciplines,it is necessary to use discipline normalization methods to eliminate the impact of such differences.At present,the academic circle has proposed a variety of discipline normalization methods.However,the presentation of the normalization method’s effect is not only limited by its own factors,but also affected by many other factors.In order to facilitate research,existing and more commonly used databases or subject classification schemes provided by scientific research institutions are usually selected as the basis for evaluation.Different subject classification schemes have different granularity of subject division,which may produce different subject normalization effects.This paper aims to analyze the impact of different discipline classification schemes on field normalization effects,and compare the effects of different field normalization methods in different disciplines classification schemes.This study will compare the effect of the mean method、median method and Z-score method under the Web of Science classification scheme.And change the different discipline classification schemes to test the sensitivity of the three field normalization methods under the Essential Science Indicators(ESI)and Organization for Economic Co-operation and Development(OECD)classification schemes.The results show that the disciplinary classification scheme does not have a significant impact on field normalization effects,and the effects of field normalization methods under different discipline classification schemes remain basically unchanged.Judging from the CCDF distribution graph,the citation distribution after using the three field normalization methods is obviously closer than the original citation count,and the citation distribution of the three field normalization methods after changing the discipline classification scheme with different granularity is still roughly the same.When the top z% method is used to numerically test the field normalization effect,it is found that the effects of the three field normalization methods remain basically unchanged after changing the different discipline classification schemes,and show the following laws: When intercepting papers below 30% of the global top,the Z-score method is slightly different than the mean method,but both are better than the median method;30%-40% stage,Z-score method shows obvious advantages;more than 40%,the median method is significantly better than the other two methods.At the same time,through the analysis of the results,it can be found that although the effect of the field normalization method is basically the same after changing the different subject classification schemes,there are significant differences in the effects of the three field normalization methods under the same subject classification scheme.How will this difference affect the ranking results of Chinese university research results? What is the degree of impact? In order to explore the impact of the differences in the effects of these field normalization methods on the results of scientific research evaluation in universities.This study compares the changes in the ranking results of the influence of Chinese colleges and universities after the normalization of the mean method,the median method,and the Z-score method.The selected subject classification scheme is the ESI subject classification scheme.Because when using the top z% method to test the effects of the three normalization methods,the average deviation of the proportion of papers at each section under the ESI discipline classification scheme is smaller than that of the other two discipline classification schemes.And Web of Science,OECD disciplines Under the classification scheme,there is a one-to-many relationship between the journals where the papers are divided into disciplines.In the results of scientific research impact evaluation,a paper may be counted repeatedly in multiple disciplines,while a paper in the ESI discipline classification scheme corresponds to only one discipline.Which reduces the possible impact on the evaluation results.Based on the results of the two calculation methods for the overall influence of papers.When the overall influence of the paper collection is the mean value of the influence of a single paper,the agreement between the mean method and the Z-score method is slightly higher than the median method.And because the number of papers is not taken into consideration,some universities with a small number of publications but a high number of citations are ranked in the forefront of the ranking results under the three field normalization methods,which are biased against university with more posts.When the overall influence of the paper collection is the sum of the influence of a single paper,university ranking results are not sensitive to differences in the field normalization effects of different disciplines.The results of the university rankings under the mean method,median method,and Z-score method fluctuate but the changes are small.Under this calculation method the differences in the effectiveness of the three disciplines’ field normalization methods did not have a significant impact on the ranking results of universities’ scientific research influence.However,this method also makes some universities with a large number of publications an absolute advantage in the ranking results,and is biased against universities with low output and high citation.