Research On The Scientific Foundation Project Evaluation And Selection Method Based On Evidential Reasoning Rule And It's Application

Science foundation system is one of the main channels to support basic research for governments. Science Foundation Project Evaluation and Selection (SFPES) is the key component among project management activities, and the level of project evaluation is vital to the effectiveness of scientific project management. The choice of SFPES method directly relates to the vital interests of a vast majority of scientific and technical workers, and it determines whether the science foundation can be used efficiently. The research of SFPES nationwide and abroad is a hot topic in the field of project management in recent years. SFPES can be formulated as a multiple attribute group decision making problem. Current research of SFPES does not reflect the importance and reliability of the information given by decision makers, which play an important role in the aggregation quality of decision information. Due to the limited knowledge of individuals and their partial understanding of the problem, people often find it difficultto make accurate evaluation in a quantitative format,and tend to use a series of linguistic grade sets to express their opinions on a certain attribute of an evaluation problem. Therefore,it is necessary to express experts' evaluation information as pieces of evidence and then information from multiple sources or experts can be aggregated together. In addition, how to measure the reliability of evaluation information and use it to adjust the original evaluation information during the aggregation, is an important scientific problem of solving complex decision-making problem that with the utilization of experts' knowledge and experience. As a result, it is of great theoretical and realistic significance to study the SFPES.This dissertation introduces Evidential Reasoning (ER) rule as the basic theory to SFPES, and analyzes the process of SFPES based on ER rule. It also discusses the generation of belief distributions and aggregation of multiple pieces of evidence.Specifically, we aim to combine ER rule, peer review, Bayesian theory, confusion matrix, multi-attribute utility theory and expected utility theory, then study the SFPES problem, where we solve the expression and aggregation of evidences with linguistic grades and measurement of reliability and then finish the ranking and decision of projects with decision theory. The main contributions and innovative research results are as follows:(1) The problem of SFPES based on ER rule is analyzed. On the basis of the existing practice of SFPES,we construct a comprehensive evaluation model based on ER rule and provide the logical relationship of the science foundation project evaluation information. The expression of experts, evaluation information is explored and the meaning and measurement method of weight and reliability are analyzed. In combination with aggregation of evaluation information on multiple attributes by multiple experts and choice of decision rules, the key issues of SFPES based on ER rule are analyzed.(2) Reliability parameters based on historical evaluation accuracy are defined, to analyze the quality of evaluation information by different experts and reflect the influence of information quality on the aggregation results. With the use of the confusion matrix, we define two parameter values of reliability under different conditions and choose corresponding parameter values according to two opposite recommendations by experts to adjust the original evaluation information. On the basis of analyzing reliability, the aggregation of experts' evaluation information based on ER rule is discussed, in order to improve effectiveness of SFPES decision-making.(3) A data driven evidential reasoning rule for solving SFPES problem is studied. By analyzing the relationship between likelihoods and belief distributions in Bayesian theory and ER rule respectively, we explore the mapping relationships between the evaluation grades and the final funding outcomes reflected by historical data and a new generation method of belief distributions for representing the linguistic evaluation grades. After aggregating evidences on multiple attributes from experts by using ER rule, the total belief degrees and the overall performance can be generated for ranking and selecting projects.(4) The method and decision analysis of SFPES based on experts' subjective linguistic evaluation grades. The belief distributions are provided directly according to the linguistic evaluation grades by experts. A utility based information transformation technique is explored to handle qualitative evaluation criteria with different evaluation grades and a decision method incorporating expected utility and comprehensive probability distributions is provided.(5) The decision model based on ER rule is applied to the project evaluation and selection in National Natural Science Foundation of China (NSFC), and the effectiveness and practicability of the proposed approaches are validated. In the context of NSFC peer review, we apply the methods mentioned above in the project evaluation of NSFC by using ER rule. According to the actual funding outcomes, we conduct an experimental study. In comparison to the existing method in NSFC, the experimental results show that the reliabilities of the review information provided by different experts have a significant influence to the selection of eligible projects and the ER rule based methods provide effective ways for aggregating peer review information in the NSFC and it is also capable of handling different forms of peer review information obtained.