| BackgroundThe progress of science and technology is a key factor in advancing our country’s economic and social development enhancing overall national strength and international competitiveness.General Secretary Xi emphasized that it is necessary to concentrate on developing core technologies,and to reinforce scientific research input and capability building in public health sector.Since the outbreak of COVID-19,investments in public health research projects have increased significantly,indicating that public health research projects are highly valued by the government.As a critical component of science and technology evaluation system,the evaluation of scientific research performance is an important approach to promote research management efficiency and the construction of an innovative country.In practice,the achievements and innovations of public health derived from research projects have an directly favorable effect on people’s lives and health,and it also influence the formulation and implementation of national public health policies.Therefore,it is significant to make the continuous improvement and perfection of the performance evaluation system of public health research projects.Although our country has made some active explorations and accumulated valuable experience in the performance evaluation of public health research projects,due to the late start,narrow coverage and the short development time,there are still some problems in terms of evaluation systems and application.Given the existing research data,the evaluation system is incomplete,and the evaluation method is not standardized,the apply of performance evaluation in the field of public health is relatively small,with limited breadth and depth,which restricts the improvement of the performance evaluation system of public health scientific research projects.In summary,the study of the performance evaluation model of public health research projects is not only a need for management and development of national science and technology but also a choice for the transformation of government functions to make up for the shortcomings of traditional evaluation.Therefore,establishing and improving the performance evaluation system for public health research projects can provide data theoretical and practical supports for research management,and it will give full play to the edge of evaluation in adjusting and optimizing the allocation of scientific and technological resources.The health interest of public would be better secured via promoting the quality and quantity of evaluations of public health research projects.Research purposesTaking the performance evaluation of public health scientific research projects as the object,this study constructs the performance evaluation system with the characteristics of public health scientific research projects,in order to provide implications and policy suggestions for promoting the performance evaluation of public health research projects in China.The specific research objectives are follows:（1）To construct the performance evaluation index system and evaluation model of public health scientific research projects so as to determine the evaluation criteria.（2）To make an empirical study on performance evaluation of public health research projects in universities in Shandong province.（3）To use qualitative research to analyze the results of performance evaluation and to supplement the existing problems of scientific research project performance.（4）To summarize the research findings and put forward the basic ideas and policy recommendations for constructing a performance evaluation system of public health scientific research projects.Data source and research objectAccording to the research purpose,the research data mainly include:first,the data of the index system comes from expert consultation materials of Delphi method;second,the empirical research data comes from the survey data of public health research projects in Shandong province,including the relevant data information of public health research projects of 6 universities in Shandong province from 2006 to 2020;third,the data of qualitative research comes from the interview information of public health project researchers in 6 universities in Shandong province.Research content and methodsUsing literature research,Delphi expert consultation method and Analytic Hierarchy Process（AHP）,to construct a performance evaluation index system for the performance of public health scientific research projects.The index weight of each level was determined by Analytic Hierarchy Process（AHP）and the index system was modified by the result of reliability and validity test.A comprehensive evaluation model based on the fuzzy comprehensive evaluation method of Data Envelopment Analysis（DEA）is constructed.The public health research projects of six universities in Shandong province during 2006-2020 are selected for performance evaluation,the constructed evaluation system is empirically tested,and the comprehensive evaluation results obtained from the samples are analyzed to find the performance problems of public health research projects.Data envelopment model is used to quantitatively evaluate and analyze quantitative indicators,and fuzzy evaluation model is used to qualitatively evaluate and analyze non-quantitative indicators.Finally,the quantitative evaluation results and qualitative evaluation results are comprehensively evaluated and analyzed to obtain the comprehensive performance evaluation results of the project.The influencing factors of comprehensive performance score of research projects are analyzed.According to the results of the empirical study,the qualitative interview was used to carry out targeted research.The purpose sampling method and qualitative research method were used to conduct semi-structured interviews with selected researchers of public health research projects in Shandong province.According to the principle of information saturation,a total of 13 researchers were interviewed,and their interview records were converted into written materials for encoding and induction,and problems in the performance of scientific research projects were extracted from the interview materials by content analysis.Research result（1）Construction of performance evaluation system for public health research projectsThrough three rounds of Delphi expert consultation and Analytic Hierarchy Process assignment,the performance evaluation index system was finally formed.Based on the framework of the structure-process-result model,the performance evaluation system has a total of 3 first-level indicators,12 second-level indicators and 44 third-level indicators.The three first-level indicators of projects are basic evaluation,progress evaluation and effectiveness evaluation.The basic evaluation dimension includes 5 secondary indicators,the progress evaluation includes 3 secondary indicators,and the effectiveness evaluation includes 4 secondary indicators.In order to reflect the accuracy and objectivity of the data,this study constructs a comprehensive evaluation model combined with a qualitative method and quantitative method.The method of combining Data Envelopment Analysis（DEA）and Fuzzy Comprehensive Evaluation（FCE）methods is used to construct a comprehensive evaluation model.（2）Empirical Study Based on Public Health Scientific Research ProjectThe average efficiency of the sample research projects is 0.410,the average pure technical efficiency is 0.464,and the average scale efficiency is 0.872.The proportion of valid DEA was 4.9%,and the number of projects is 8.The proportion of weak DEA was 6.1%,and the number of projects is 10.The proportion of invalid DEA was 89.0%,and the number of projects is 145.There are 151 projects（92.6%）without the best scale efficiency,among which 59 projects are in the increasing stage of scale returns.The fuzzy evaluation results showed that there are 161 projects（98.77%）in the best-level group and 2 items（1.23%）in the good-level group.The comprehensive performance evaluation score can be classified into four classes:best,good,moderate,worst.The number of projects with performance evaluation of best-level is 19（11.66%）,and the number of projects with performance evaluation of good-level is 46（28.22%）.The number of projects with performance evaluation of moderate-level is 98（60.12%）,which is the group with the largest number.There is no comprehensive evaluation belonging to poor grade.The average value of performance evaluation results of sample projects is 2.468 and the standard deviation is 0.62.The results of multiple linear regression showed that compared with female project leaders,the performance score of research projects of male project leader is higher（β=0.515,P=0.000）.There is a positive correlation between the age of the project leader and the score of projects（β=0.039,P=0.000）.Compared with the project leader with master’s degree,the performance score of the project leader with doctoral degree is higher（β=0.622,P=0.000）.（3）Qualitative research on public health researchersConduct in-depth interviews with the public health researchers from universities in Shandong province on the problems in the performance evaluation of scientific research projects based on their scientific research experience,and explored the reasons that affect performance in the practice of scientific research projects.The results of performance analysis of different grades of scientific research projects show that the primary influencing factor of the performance of excellent-level and middle-level are the number of achievements,and the cumulative scores are 13 points（31%）and 12 points（33%）respectively.The secondary influencing factor of excellent-level performance is scientific research infrastructure,with a cumulative score of 11 points（26%）.The secondary influencing factor of medium-level performance is human resources,with a cumulative score of 10 points（28%）.After summing up the results of the interview,it can be divided into four main themes:input of research and development resource,output of scientific-research,project management,performance evaluation system.And then,the associative themes are summed up under the core theme.By summarizing the content of the interview,the existing problems of scientific research projects are concluded.The study found some existing problems,such as lack of scientific research personnel and resources,shortage of professional team cooperation,poor research infrastructure and insufficient fund,focus on papers only,neglecting other forms of output,inadequate incentives for scientific research honor awards,the imperfect incentive mechanism and feedback mechanism,and the deficient interior management system and scientific research funds management.Main conclusionsBased on the Structural Process Outcome model,this study constructed the performance evaluation system of public health scientific research projects,including 3 first-level indicators,12 second-level indicators,and 44 third-level indicators.The weight of each indicator is determined.The method of combining Data Envelopment Analysis（DEA）and Fuzzy Comprehensive Evaluation（FCE）methods is used to construct a comprehensive evaluation model.Thus,the study complete the task of systematically building the index system,evaluation subject,models and criteria.The overall results of the performance evaluation of public health scientific research projects in Shandong province are medium-level,but the number of projects with outstanding performance results is less.The overall evaluation results of qualitative indicators belonging to more than good-level,but the number of projects with best-level performance is less.The overall evaluation situation of quantitative indicators is not perfect,and there are many projects with low utilization of resources.The primary factor of the performance in different levels of scientific research projects is the number of achievements.Although project management and other related systems are still perfect,problems that affect project performance are still found in practice.There is room for improvement in the allocation of scientific research resources,the construction of the talent team,the scientific research evaluation system,and the scientific research management mechanism.Countermeasures and recommendations:According to results of this study,our suggestions were as following:（1）To strengthen the construction of talent team and improve the ability of scientific research and innovation.（2）To increase investment in scientific research funds and raise the utilization ratio of funds.（3）To strengthen the construction of scientific research platform and concentrate the advantages of scientific research resources.（4）To strengthen supervision and management to ensure the efficient functioning of the mechanism.（5）To improve the quality of scientific research results and raise the technology transfer rate.（6）To strengthen the protection of knowledge achievements and promote the cooperation and sharing of resources.Innovations and LimitationsInnovations:（1）Based on the theoretical framework of the Structural Process Outcome model,this study constructs a performance evaluation index system with the characteristics of public health scientific research projects,which is comprehensively analyzed from multiple perspectives such as project investment,project process,scientific research management and results benefit.Based on fuzzy analysis theory,a comprehensive evaluation model is constructed by combining Data Envelopment Analysis（DEA）theory.（2）This study evaluated the performance of public health research projects,explored the existing problems and analyzed the reasons.It provides a theoretical basis for improving the performance level of scientific research projects and improving the performance evaluation system.Taking the performance evaluation of public health scientific research projects in Shandong province as an example,the performance evaluation system was studied by empirical and qualitative research.LimitationsDue to the objective impact of limited data availability,it is difficult to obtain data from public health research topics included in this study,and the sample size is small,so the stability of the model is slightly weaker.Secondly,the access to obtain data is limited,so the representativeness and extrapolation of the conclusions of the evaluation results are slightly weaker.Thirdly,because the benefits of public health research projects have the characteristics of long return periods and various results output forms,the performance evaluation index system cannot be completely solidified and needs to be continuously revised and improved. |
PDF Full Text Request