Identification And Analysis Of Non-consensus Projects In Peer Review

Objective:To discuss the identification and analysis methods of non-consensus items in peer review.Methods:(1)literature research:to retrieve the database of wanfang database,China knowledge network database,and weipu database,and to consult relevant literatures related to this research,collect and organize indicators on the identification and analysis of non-consensus items,and use various indicators to complete the identification and analysis of empirical data.(2)expert meeting:adopting the form of expert meeting,demonstrating the correctness of identifying and analyzing the indicators of non-consensus items and discussing their application strategies,in theory and practical application,to explore the evaluation methods of identifying and analyzing non-consensus items.(3)empirical study:using the actual evaluation of a fund in Beijing to analyze the accuracy and science of the non-consensus project and to improve the application strategy,using empirical data to test the feasibility of the non-consensus project and to provide the basis for the improvement of further research.(4)data processing,index calculation,basic statistical description and statistical test were completed using SAS 9.4 statistical analysis software,and the chart production was completed by Microsoft Excel 2010 software.Results:(1)Identification of non consensus projects:This study uses peer review experts to build indicators to identify "non consensus items" in the project,and the identification index is standard deviation.When the standard deviation of the project is greater than that of the standard deviation(M+2S)of all projects,the project can be considered as a "non consensus project".(2)analysis of non-consensus items:this study analyzed the non-consensus items from the perspectives of experts and projects.Experts point of view,this study thinks that the cause of the consensus on the project can be summarized as the following:first,peer experts personal habits of grading,could be generating score too strict or too loose,lead to big differences with other evaluation experts,cause the consensus of the project,measured by system error.Secondly,when an expert and other experts have a large difference,it is easier to cause the project non-consensus,so the consistency of the expert rating is the cause of the non-consensus of the project,which is measured by the cumulative horizontal dispersion index.Thirdly,the experts’ preference or bias towards a certain type of project or a project may result in abnormal rating,leading to abnormal fluctuations in the expert’s score error,and may lead to non-consensus of the project,which is measured by the comprehensive dispersion rate.Fourth,the innovative project,the experts are more obvious in support and opposition to such projects,and it is easy to cause the difference of project score,which makes the project non-consensus,but the innovation of the project is difficult to be measured from the score perspective.Therefore,this study adopts the elimination method,and if it does not meet the above three conditions,we can think that the project is innovative and the manager can pay more attention.From the perspective of the project,this research USES the single item discrimination index index to analyze the single item evaluation index which is the most controversial in the non-consensus project,which facilitates the manager and the project applicant to find the dispute of the project.Conclusion:For the recognition and analysis of "non-consensus items" in peer review,the ultimate goal is to improve the quality of peer review and ensure the support of excellent projects.In this study,the construction of identification and analysis indicators for non-consensus projects can provide inspiration for managers in the process of actual project management,and help managers to realize the importance of expert evaluation behaviors in projects with research value,and avoid the loss of excellent projects due to improper evaluation by experts.The identification and analysis indicators of "non-consensus items"constructed by empirical data inspection are feasible.In the practical application of managers,the data can be further improved according to the continuous accumulation of data.