Study On The Establishment Index System For Evaluating Systemic Quality Of Master Student Supervisor

Cultivating quality is substantially vital for master student education. The personal qualities of master student supervisor, on the other hand, directly affect the cultivation quality of master students and appear to be the key factor for improving the quality of master student cultivation. Currently, there is a great gap between the master student supervisors in our country, which is largely due to the unreasonable evaluation of the quality of master student supervisors. In most of domestic colleges, master student supervisors will last for life long once singled out, which will naturally lead some of them to lose their enthusiasm to go forward and do much better work in their fields. To ensure high systemic quality for the master student supervisors, it is therefore necessary to evaluate their overall quality regularly.The present score evaluation system mainly focuses on the academic situation of master student supervisors, of which the scientific and research outputs constitute the major part and the quality of master student supervisor, particularly the systemic quality do the minor one. It can be clearly seem that it has emerged as a bottleneck to influence the cultivation quality of master students.In the hope of revealing the shortage and providing more scientific evidence to pick out and depose the master student supervisors, we employ a series of ways in the present study to establish an index system to evaluate the overall quality of master student supervisors, which include systemic analysis, documental research, Dephirule, class analysis and statistical analysis. This system will surely be helpful for the master student supervisors to check themselves and make them to work more actively and positively.Establishment of index systems for evaluating the overall quality of master student supervisorFirstly, we referred to some existing systems and consulted experts in a limited scope to obtain the primary suggestions and later, collected some first-class indexes to evaluate the overall quality of master student supervisor. The preliminary indexes are composed of 8 first-class indexes, which include personal thoughts, mental state, knowledge quality, innovating quality, guiding quahty, research quality, administrative quality as well as information quality, 16 second-class and 47 third-class ones. These indexes cover almost whole aspects of the overall quality and ability of master student supervisors. We inquired experts repeatedly who were endowed with convincing experience in master students cultivation, guidance and administration as well as some professors, mentors to pick out the evaluation indexes and determine the content and weight of these indexes. The first-turn inquiryOf 38 answered inquiry tables, 20 experts thought that the set-up for index systems was far scrupulous, which could hamper to emphasize the evaluation of master student supervisor and to evaluate them more objectively due to the crossing effects between indexes. Additionally, we also obtained a variety of suggestions from the experts. For example, experts were unanimously skeptical about the importance of ideological quality and they considered that this index was too subjective and hard to be evaluated. It seemed necessary to reduce its weight or even delete it. Furthermore, we should not be too critical about the personal physical status. The concepts about the second-class indexes of basic knowledge and related knowledge as well as the third-class indexes of noumenon knowledge and conditional knowledge are not as clear as expected. The experts, on the other hand, widely regarded that it was of great necessity to combine administrative ability, informative ability and other relatedquality and to stress the importance of knowledge quality, innovative quality, research ability as well as guiding ability. Experts also suggested that the feedback information should be added about the social judgment and working situation for the graduated students to serve as one item to evaluate the systemic quality of supervisors.Based on the experts' suggestions from the first-turn inquiry, we modified and even deleted some of the primary indexes, hi the revised version, we added a new first-class index, basic quality, which covered three second class indexes, knowledge quality, innovative quality and personal quality. Two other second-class indexes, ideological quality and physical quality were combined together and renamed as instinct quality, which was used as a third-class quality and worked as a branch index for the personal index, just as academic scores and active study did. Because the basic quality is known to be the basis for the supervisors to develop their activities, including teaching, researching and guiding students, we add it into the first-class indexes. We also merged the informative ability into the innovative quality and administrative ability into the guiding ability. According to the experts' suggestion, we added the standard of graduated students as a new index in the revised version. Because there were great changes in the first-class indexes, we had to modify or combine the second and third-class indexes. To emphasize the importance more clearly, we shortened all of these three-class indexes and the first-class ones was decreased from 8 in the original copy to 4, and the second-class ones from 16 to 10 and the third-class ones from 47 to 23. The second-turn inquiryOf 25 experts who completed inquires, all of them agreed on the set-up of first-class indexes and 24 agreed on that of the second-class indexes. Except for 88 percents agreement ratio in administrative ability, the other 3 indexes all received an agreement ration more than 92 percents. So, it can be easily seen that the experts had no dispute on these indexes and all of three class indexes are widely confirmed. The grade content in the third class index appeared more scientific and practical after modified on the basis of experts' suggestion.In the present inquiry, we determined the weight of these indexes. For the weight of first-class indexes, we quantified it by using expert's inquiry. The experts were asked to use Satty relative importance table and grade analysis to test the relative importance by 9 grades from 1 to 9. Later on, the weight was calculated using standardized summation after transferring the importance into judgment matrix. For those of second and third-class indexes, we asked experts to judge each of them with a value from 1 to 5 according to their relative importance. After summing the experts' results, we analyzed the focusing extent of their suggestions by calculating their means and full-score frequency and the tuning extent by calculating the variance and tuning coefficient of indexes. Under the identification by the low limit of 3 in means and 0.1000 in full score frequency and the upper limit of 0.2500 in variance, 10 second-class indexes were covered. When the limit of means was set at more than 3.5, full score frequency at more than 0.1 and variance at less than 0.25, we found that 22 third-class indexes were included except that scores and patent were deleted. In this turn inquiry, the tuning coefficient for second-class indexes is 0.19, for third-class ones is 0.161, indicating that the experts' suggestions were highly in harmony with each other. The index weight coefficient was calculated by using portion assignment, that is, we first summed the mean scores of second and third-class indexes respectively and after that, we calculated the weight coefficient by dividing the score of each index by the whole scores.The final version of evaluated index system for master student systemic quality is composed of 4 first-class indexes, that is, basic quality, teaching and researching ability, guiding ability and the level of graduated students. In addition, 10 second-class indexes and 22 third-class indexes are covered. Reliability analysis of expert inquiryAs generally known, the reliability of expert inquiry will directly determine whether the index system is accurate and scientific. Therefore, it is necessary to take measures to ensure the reliability. In the present study, we have fully considered the selection, activity and authorization of experts to minimize the loss of reliability.1. Selection of expertsOf all 42 selected experts, 40 ones own the title of associate professor or much higher position. Of these, 12 Ph.D candidate mentors and 13 graduate student supervisor were included. In addition, 28 persons work as the administrative leaders and 11 as the leader in their subject. The selected experts covered the leaders in different universities in charge of graduate student education, the administrative leader of the first military medical university, the department directors of our two affiliated hospital and of the department of basic medicine. Most of them, characterized by outstanding specialized knowledge and prompt thinking and judging ability, have enough experience in the selection of master student supervisor and in the cultivation or related academic research of graduate student and are widely thought to be able to provide comprehensive ideas through different sights.2. Active coefficient of expertsThis index is always represented by reclaiming rate. As general, 50 percent reclaiming rate can be regarded as a minimal ratio and it is reasonable if this rate is higher than 60 percents. In the first-turn inquiry, we sent out 42 copies of inquiry tables and 38 ones were reclaimed with the reclaiming rate reaching as high as 90.5 percents. In the second-turn inquiry, this rate is 89.2 percent and 25 tables were reclaimed out of 28 sent ones. It was noteworthy that in each turn, experts filled out the table carefully and paid sufficient emphasis and care on our research.3. Authorization of experts' viewThe authorization coefficient of experts' view was calculated in a grade manner by summarizing the familiarity and their self-evaluation on the proofs of index judgment. The higher authorization coefficient is, the more valuable experts' view is and the more reliable the result. It is acceptable only when authorization coefficient is higher than 0.70. This index of the 25 experts in second-turn inquiry varies from 0.60 to 1.00 and that of 78% percent experts is higher than 0.8. The overall authorization coefficient equals to 0.8467, which ensures the reliability of indexes. Innovation and Features1. Attaching importance to the guidance of evaluationThis system covers not only the summarization and evaluation of supervisor's past work, but their future development.2. Objective, Scientific and ReliableThe present study used a quantitative method to balance the errors induced by people's subjective experience during the qualitative tests. We employed the layer analysis to qualify the first-class indexes and used variance and tuning coefficient to weight the second and third-class indexes. In the content of indexes, we tried as possible as we can to avoid some qualitative words such as good or better. We have controlled the subjective effect to a relatively small limit and made the evaluation more objective and scientific.3. Erasing the misleading region in previous evaluation index systemsThe present study have not only reflected an overall demonstration of supervisors' knowledge, quality and ability, but also paid importance to the possible influences of supervisors' innovative quality, scientific research ability and guiding ability on the cultivation quality. Moreover, we regarded the level of graduated students as a standard to evaluate supervisor and combined the evaluation to supervisor with the quality of graduate student cultivation.4. Establish a computer-based evaluation systemIn the present study, we established a computer-based system to evaluate the systemic quality of master student supervisors. This system simplifies the procedure of evaluation and increase the versatility, which will help to evaluate master student supervisor smoothly and facilitate its generalization and utilization. Simultaneously, this system will provide some possibility to evaluate on-line.