| Objectives and Significances1. Performance evaluation is required for government to fulfill its duty in disease control and prevention systemPublic service implementation is the symbol of a modern government, and how to increase its quality has now become the core of the governmental management. Performance evaluation can promote the government to take up its public service responsibilities fairly and efficiently.Disease control and prevention system (DCPS) has provided the pure public service to the society, and it is one of the government's major public service tasks. Nevertheless, in China, public service providing in DCPS is still lack of efficiency. The new medical system reform plan (2009) clearly declared that it was important to strengthen performance evaluation and improve efficiency and quality in DCPS to promote equalization of public health service. Hence, it is urgent to establish the DCPS, in premium quality and high efficiency, with scientific performance evaluation and management in the new era.2. DCPS's appraisal presentation needs a breakthrough in the field of research and practicePerformance evaluation is a quantified analysis of working process, results and efficiency heading to the completion. It was oriented in the beginning of the 20th century; in the fifties, the United States, United Kingdom, and some other countries were using working evaluation in the management of the governmental operations and public service departments. In the health field, World Health Organization (WHO), in year of 2000, first declared three standard appraisal targets:improvement of health conditions, people's responsiveness and fairness of health care financing. Also, the primary health care guideline and the concept of healthy city have been a great influence around the globe. Those ideas helped to stimulate the perfection of health scheme. However, in China, most of the researches were focused in the field of medical treatment, rather than in public health. Especially DCPS was lack of systematic studies, and few regulations were announced:Only the Ministry of Health promulgated the "National Health and Epidemic Prevention Station evaluation criteria" and implemented the grade assessment around the country in 1996. At the end of the 1990s, following by the health reform, many health and epidemic prevent stations were changed into centers for disease control and prevention (CDC), and their functions have gone through a major shift with greater responsibility; nonetheless, it still lacked of in-depth study of performance evaluation. Therefore, it is a scientific issue to create the evaluation indicators and models, which closely combined DCPS with Chinese characteristics and other countries'experiences and methods.3. Performance evaluation is an extension of the research of DCPS in ChinaStarting from 2003, Disease Control and Prevention Bureau of Ministry of Health has prepared a disease control and prevention research group, which focused on the long term development target in series of questions "What should be done? What does it require? How to proceed? And how was it done? ". These series of studies have obtained many supports and inquiries from several government ministries for further decision making. Currently, how to objectively and accurately evaluate the DCPS's performance has been the main topic of the system.Hence, through the reviews of updated international researches and practices, this research was specifically designed for the needs of the performance evaluation of DCPS in China. It clarified a series of inquiries in performance evaluation:What should be evaluated? How to comprehensively evaluate? How to scientifically and operationally implement the evaluation? What is the result after the practice in provincial institutions?Materials and MethodsFirst, the performance evaluation indicator system was established by applying system theory, Macro-Model of Health System, the hierarchical structure analytic theory, expert consulting and field reviewing; the comprehensive evaluation model is constructed, based on the principle of fuzzy comprehensive evaluation, analytic hierarchy process (AHP), focus group discussions and brainstorming. Second, combining inductive analysis of literatures and expert consultations, following the concept of business process management and information systems design process and steps, the implementation procedures and methods in the practice of performance evaluation were developed and the design idea of the management software was formed. Finally, based on the data from 32 provincial CDCs, by using the principle of "structure-process-outcome", crosswise and longitudinal comparison, gap analysis, cluster analysis, sensitivity analysis and regression analysis, the performance status quo of the provincial CDCs were evaluated and some suggestions and improvement strategies were made.Data-collecting methods included inductive analysis of literatures, expert consultations, focus group discussions, brainstorming, pre-trial, field reviewing and evaluation data collected from 32 provincial CDCs across the country. The experts came from Disease Control and Prevention Bureau of Ministry of Health, the provincial health department (bureau) officers. Institutions involved in pre-trials included 13 provincial CDCs as Beijing, Hebei, Shanxi, Liaoning, Jilin, Jiangsu, Zhejiang, Shandong, Guangxi, Sichuan, Yunnan, Shaanxi, Qinghai. Institutions participating in field reviewing included 3 provincial CDCs(Beijing, Jilin, Sichuan) and 4 municipal/country CDCs(Handan, Hebei; Yanbian, Jilin; Chengdu, Sichuan; Changping, Beijing).Main Results1. Establish the performance evaluation indicator system and comprehensive evaluation model of DCPSUsing scientific and accepted methods, this research has developed the research thought, procedure and methodology on the performance evaluation of DCPS as "logical thoughts, acknowledged methodology, operable procedures and accessible results":By applying system theory, Macro-Model of Health System and the hierarchical structure analytic theory, a evaluation framework has been constructed, including 5 aspects:social environments, working conditions, working processes, system results and health outcomes. On the basis of the framework, through three steps of "collecting indicators-screening indicators-defining indicators", with expert consulting and field reviewing, the performance evaluation indicator system was established. The definition and calculation of each indicator, interpretation of basic data, data collection sources and methods were clearly defined. The indicator system of disease control and prevention institution was made up of 8 categories including disease control and prevention, the disposal of public health emergencies, information management, health hazards monitoring and intervention, laboratory testing, health education and health promotion, technical guidance and applied research and integrated indicators, which included 34 projects and 77 indicators at provincial level,35 projects and 104 indicators at municipal level,35 projects and 101 indicators at county level.And then, with the guidance of the Analytic Hierarchy Process (AHP), the indicator weights were determined by consulting experts from the provincial health department (bureau) officers across the country. Through focus group discussions and brainstorming, combined with the distribution of indicator values, the evaluation criteria of each indicator was identified. Finally, by obtaining the original data, constructing membership functions and evaluation matrix, calculating the weight with matrix, a comprehensive evaluation model was built to calculate the performance scores, based on the principle of fuzzy comprehensive evaluation. 2. Develop the implementation procedures and methods in the practice of performance evaluation of DCPSBased on the broad research results and practical experiences, this research developed the implementation procedures in the practice of the evaluation. It should include the three following phases:Preparation (program developing, organization implementation, and data collection), Implementation (data reporting, quality control and validation check), Feedback (comprehensive evaluation, performance diagnosis and continuous improvement).Moreover, to ensure the scientific implementation of performance evaluation, this research has focused on three key points in the procedures as below:(1) Broad collections of evaluation data:build performance evaluation questionnaire which included 4 models,26 categories,50 subcategories and more than 3000 indexes; define the scope and route of data collection; establish a standardized data collection and reporting process.(2) Data quality control:check the data reporting error with data type, format, logic and limit; pick out the key indicators and institutions by means of logic judgment, limitation judgment, economic judgment and longitudinal comparison judgment; clarify the verification methods and processes of the on-site data.(3) Scientific diagnosis of the performance:based on the crosswise and longitudinal comparison, gap analysis, the multi-dimensional thinking was formed to diagnose the inadequacies in quantitative gaps from "overall-dimensions-indicator", and a standardized template formation of "Diagnostic Report of Disease control and Prevention performance evaluation" was created.3. Form the design idea of the management software of performance evaluation of DCPSThe first is the framework design and requirements analysis of the management software. Based on the implementation process and focus group interviews, this research has defined the framework of the software which included information reporting, verification & judgments, comprehensive evaluation, information search and system management. With the concept of business process management, the logical relationships between the aspects and the flow of information were clarified. The user types of the software were defined, including 6800 information reporting users,450 assessor users and 450 administration users. And then, this research has formed the structures and contents of main results databases of the software and implemented dynamic allocation between different roles of users and databases. After that, this research focused on the specific function module design such as evaluation data collection, data error check, picking out the key indicators and institutions, computing the values and scores of the indicators. Finally, the management software was developed based on the ideas above and tested through the field testing to ensure its operability and feasibility. In a word, the software provided a unified and convenient operating platform for data collection, data analysis, result evaluation and utilization in the practice of performance evaluation.4. Performance evaluation and improvement strategies of provincial CDCs(1) Overall evaluationIn 2009, a grade point of 83.2 and accomplishment of 82.1% in average were obtained from 32 provincial CDCs across the country.(2) AchievementsFinancing mechanisms for CDC's work have gradually improved. In 2009, the average financial input of provincial CDCs was 345 thousand Yuan per capital, an increase of 199.7% over 2005. The personnel and public funding were 54.2 and 27.0 thousand Yuan per capital, increased by 79.5% and 78.4% compared to 2005.The special prevention and control funding for people served was 2.23 Yuan per capital, an increase of 283.7% over 2005.From 2005 to 2009, the proportion of profit-oriented service revenue of total institution revenue has declined to 27.4% by 33.2% and the proportion of financial input has increased to 67.2% by 23.3%.Professional personnel teams have been strengthened. The average score of the overall human quality was 7.50, the proportion of bachelor degree or above was 57.9%, and the proportion of senior professional titles was 26.6%.Working conditions have been improved. The floor areas per capita were 66.8 square meters, the proportion of laboratory area of the total institution was 41.8%, and the configuration ratio of Class A device was 78.6%, increased by 5.0%,34.8% and 31.4% respectively over 2005.The control and prevention of major diseases has achieved remarkable results. The index of regulate disposal of epidemic outbreak was 0.932. The completing rate of communicable disease surveillance and vectors monitoring were respectively 91.3% and 99.8%.The coverage of standardized vaccination unit was 89.8%.The proportion of cold-chain in good operation was 94.4%.Immunization rates of five basic vaccines including polio, hepatitis B, diphtheria-peruses-tetanus (DPT), bacillus calmette-guerin (BCG) and measles have all reached the requirements of national immunization programs. The ratio of counties (districts) that the timely immunization rate of the first dose of hepatitis B vaccine reached the standard was 91.3% and the positive rate of HBsAg within children under 5 years old has decreased to 0.7%. The DOTS strategy has achieved full coverage in all countries and the detection rate of new smear positive case in 29 provinces has reached more than 70%.The Schistosomiasis infection rates of humans and snails in epidemic provinces have under control. The ratio of counties (districts) Iodized salt monitoring reaching the standard was 98.7%. The ratio of endemic fluorosis and arsenism regions that water improvement project monitoring has reached the standard was 96.6%.The capacity of disease control and prevention has been reinforced. The disposal of public health emergencies was in time and specification. In data reporting, disease-related information retrieval, data analysis, the incidence trend forecasting, the number of provinces reaching the standards were 28,25,27 and 22 respectively. Information utilization ratio was up to 97.6%. The rates of food contamination monitoring and drinking water monitoring have reached the standards in 31 and 24 provinces respectively. The conducting rate of Class A laboratory testing projects was 84.3%, the rate of laboratory safety management was 100%, and the coverage of laboratory quality control was 85.6%. The accomplishment of health promotion activities was 97.8%, the knowledge awareness of key diseases control and prevention in target population was 73.6%, and the index of health behavior intervention was 0.916. The average time for on-site professional staffs guiding works in primary institutions was 25 days per capital, and the coverage was 97.7%.Disease control and prevention works have been widely recognized. The satisfactions of employees, the public and the relevant departments were 98.6%,98.6% and 96.5%.(3) Ongoing issuesFinancing mechanisms for CDC's work remained undeveloped. The growth of financial investment input was still lagging behind that of socio-economic, maintenance funding was still insufficient and the proportion of self-financing of total institution revenue was still high.Working forces were insufficient and human resource structure was unreasonable. In 2009, the average number of staffs in the post was 374, compared with a 5.6% decline over 2005. Only 7 provincial CDCs reached the standard in the proportion of bachelor degree or above and 9 reached that of senior professional titles. The proportions of staffs engaged in health risk factors monitoring and chronic disease control were only 10.6% and 2.5%.Equipment and facilities couldn't meet the needs of working configuration. Only 13 provincial CDCs reached the basic construction standards,10 CDCs'configuration ratio of Class A device reached the requirements, and 19 CDCs'proportions of equipment in normal operation were over 95%.The control and prevention of major diseases and chronic diseases needed to be strengthened. The ratio of counties (districts) that the coverage of HIV voluntary counseling and testing points reached the standard was 72.6%. Among the registered TB patients, the percentage of patients in system management was 85.6%. The accomplishment of parasitic diseases monitoring and control was only 55.2%. The coverage of medical institutions above the county level carrying out the cause of death reporting was only 81.1%. The ratio of counties (districts) that the work of setting up health records of residents reached the standard was just 30.6%. The ratio of counties (districts) that the management of chronic diseases patients reached the standard was just 41.7%.The capacity of disease control and prevention was not adapted to the development's needs. Only 15 provincial CDCs reached the network construction standards, and 20 CDCs' conducting rate of Class A laboratory testing projects were over 85%.The average rate of emergency supplies reserves was 68.6%, only 1 provincial CDC reached the standard. The accomplishment of health hazards monitoring and intervention was 75.5%, and the ratio of counties (districts) carrying out the projects of dietary guidance was just 35.8%. The average rate of job skills training was 84.6%, and the passing rate of continuing medical education was 85.8%. The average papers published by professionals were 0.32 per capital, and the accomplishment of research applying and research award-winning were only 67.7% and 65.9% respectively.(4) Strategies and recommendationsIn the future, the provincial CDCs should pay more attention:1) To establish an appropriated, long-term and stable financing mechanism.2) To create a quality scaled, optimize structured and qualified personnel team.3) To increase the power of control and prevention in major and chronic diseases.4) To reinforce the capability of DCPS.Explorations and InnovationsThis research has developed the thought, procedure and methodology of the performance evaluation by applying system theory, Macro-Model of Health System, the hierarchical structure analytic theory, the principle of fuzzy comprehensive evaluation, AHP, focus group discussions and brainstorming. And by using the methodology, the research has explored the indicator system and comprehensive evaluation model of DCPS. All the above provided a scientific basis and technological support for the performance evaluation of DCPS, and were adopted by the Ministry of Health and put into practice across the country.In order to solve the complexity and unfairness issues in the process of performance evaluation, this research has developed the management software by using the concept of business process management and information systems design process, which provided a unified and convenient operating platform to ensure the practice of performance evaluation. The software was also adopted by the Ministry of Health and widely used in all levels of CDC throughout the nation now.Moreover, by using the principle of "structure-process-outcome", crosswise and longitudinal comparison, gap analysis, cluster analysis, sensitivity analysis and regression analysis, this research has created the idea of scientific diagnosis of the performance and a standardized template formation of "Diagnostic Report of Disease control and Prevention performance evaluation". Also, it was the first time to comprehensively evaluate the performance status quo of the provincial CDCs, find out the key control points to promote the performance and finally make some suggestions and improvement strategies in resource distribution, building capability and duty responsibility. |
PDF Full Text Request