Development And Application Of Index For Performance Evaluation Of Clinical Departments

ObjectiveTo establish a scientific, reliable and practical index for performance evaluation of clinical departments, as an important basis of choosing excellent personnel and assessing the director's annual term.Subjects and dataForty two subjects, who were from the directors of clinical departments, the hospital leadership, Medical Service Department and the specialists in Economic Management Division in Nanfang Hospital, were investigated for Delphi method. The principles for selecting the experts were subjects with intermediate or above professional title, and the proportion of clinical department directors and the medical service administrators was no less than 70%. The data source came from the database of medical information system of Nanfang Hospital in the years of 2008 and 2009.MethodsItem pool was composed of:①relevant literatures from Chinese Academic Journal (1981～2009) with keywords of "clinical departments" and "performance"②relevant literatures from oversea③statistical index for the top hospitals which were set by the Ministry of Health. Two rounds of Delphi method were applied to conduct the questionnaire consultation survey for experts, and the Kendall coordination coefficient was used to evaluate the consistency of experts. The consistency reliability was evaluated by Cochran's a, and the construct validity by factor analysis. The double entry of data was implemented by the use of Epidata3.1. Sorting and sub-filing each clinical department, evaluating their performance were based on TOPSIS method, RSR method and KPI method respectively. Data was analyzed using SPSS 15.0 software.ResultThe first round of expert investigation:There were 40 copies of first round expert consultation questionnaires sent to the directors of clinical departments and the major manager of clinical, teaching and research departments. Among those,39 copies were took back,3 copies were invalid. The response rate was 97.5%, and the effective rate was 92.3%, senior professional titles account for 63.9%. In the questionnaire of the first round investigation, the items' coefficients of variation ranged from 17.30% to 71.24%, the coordination coefficient was 0.529, (χ2=323.529, P＜0.001). The above information meant that the opinions of experts were relatively concentrated except for some individuals'.After the first round of expert investigation, the index was modified. The bed turnover times and occupancy rate were deleted, and the efficiency of medical work and quality of medical work were consolidated into medical work status. The items of teaching, scientific research and personnel training were upgraded to the domains. Four items of completion of teaching tasks, continuing education credits, teaching assessment and teaching accident were added in teaching. Three items, the amount and quality of research fund, research paper and the research accident (academic fraud, multiple submission, laboratory accident, etc.), were added in the index. The second round of expert investigation:The index was constructed by 5 domains and 23 items. At the same time, experts' familiarity extent, judgment basis and influence extent were added. In addition, a questionnaire consultation was conducted, regarding the assessment criteria of each item. The 32 copies of second round expert consultation questionnaires were sent out with 30 copies feedback and 2 copies invalid. The response rate was 93.8%, and the effective rate was 93.3%, senior professional titles account for 86.2%. The items' coefficients of variation ranged from 32.0% to 41.4%. The minimum value of weighted means of items was the completion of the Guangzhou Medical Insurance, being 7.66%, illustrating that all experts recognized the index in this research. The maximum value of coefficients of variation was discharge from hospital, being 50.7%, indicating that experts had the most controversy against the weights distribution for this index. Experts' authority level coefficients ranged from 0.70 to 0.79, this indicated that experts were very familiar with all the items and each of the expert authority coefficients was equivalent. Coefficients of variation of the domains ranged from 32.0% to 41.4%, coefficients of variation of the items ranged from 17.4% to 50.7%. It was clear that in the second round of expert investigation, the volatility of the experts' evaluation of index was smaller than that in the first round investigation (coefficients of variation of the items ranged from 17.30% to 71.24% in the first round). Five domains' coordination coefficient in the second round investigation was 0.593 (χ2=66.365, P＜0.001) compared with 0.348 (χ2=37.558, P＜0.001) in the first round. The items' coordination coefficient was 0.717 (χ2=441.395, P＜0.001) compared with 0.529 in the first round, indicating that the opinions of experts were relatively concentrated. In view of the above information, we decided not to conduct the expert investigation anymore.After two rounds of expert consultation, the structure of index was set up with 5 domains (medical work, economic management, teaching, scientific research and personnel training) and 23 items (out-patient visits, discharge from hospital, average length of stay, completion of the Guangzhou Medical Insurance, medical quality, quality of care, quality of a sense of the hospital, medical complaints and safety, patient satisfaction, business income, balance, average of hospital charges, drugs charge ratio, completion of teaching tasks, continuing education, teaching assessment, teaching accident, research funds, research papers, research accident, internal training or study abroad, academic degree education, and recommendation and the selection of high-level personnel). The four items of binomial variables with the values either 0 or 100, namely teaching tasks, continuing education, teaching accident and research accident, would be the supplement in the index according to experts' opinion. So the index was finally determined with 5 domains,19 items, and 4 suspending items.The definition and assessment criteria for the index:The standard performance was scored 100 points. The score based 100 points was added (subtracted) by 5points if the performance was increased but less than 10%,10 points if it increased by over 10% but less than 30%,20 points if it increased by over 50% for positive (negative) items. Inversely the score was subtracted (added) by same points if the performance was decreased for negative (positive) items.Applications of the Index:The overall standardized Cronbach's a of the index was 0.415, the internal consistency was some reliable. There were significant difference (F= 182.987, P＜0.001) between the means of 19 items, and it clarified a good distinguishing. We performed factor analysis for construe validity. KMO value were 0.520, partial correlation was weak. In the Bartlett ball test, P<0.001, the data was suitable to conduct the factor analysis. The first 5 common factors'cumulative contribution rate was 52.73%; factors were not very consistent with the other factors. However, it was close to the biggest factor coefficient.We applied TOPSIS method to get the best three departments sorted by Ci were department of Hui Qiao, department of obstetrics and gynecology, and department of gastroenterology, and the bottom three departments were psychological department, traditional Chinese medicine department and radiotherapy department. The assessment by RSR method showed that 17 departments performed well and 5 departments poorly. The best three performance departments defined by the highest KPI score were department of obstetrics and gynecology, department of gastroenterology and dermatology department.The bottom three departments were psychological department, traditional Chinese medicine department and radiotherapy department. Compared with other two methods, KPI method was more practical and convenient for application.ConclusionWe initially developed the Index for Performance Evaluation of Clinical Departments with the structure of 5 domains,19 items and 4 suspending items. The index is acceptable with its reliability and validity. The index could be used for choosing the excellent personnel and assessing the director's annual term.