Effect of multiple perspectives on assessment of teacher effectiveness: A Data Envelopment Analysis

The purpose of this research is to provide key stakeholders in higher education a way to understand teacher assessment and its contribution to teacher effectiveness in a multiple criteria context. Do the perspectives of teacher effectiveness held by administrators, faculty, and students affect the ranking of relative efficiencies of selected undergraduate faculty? This study will validate Data Envelopment Analysis (DEA) as a means to measure teacher efficiency without imposing production functions or prices. DEA is compared to, what is termed here, a weighted approach. The weighted approach is a variation on a more traditional quantitative approach: weighting criteria and applying the weights to performance measures. During the 2003 spring semester 10 full-time undergraduate faculty at Hawaii Pacific University (HPU), a private liberal arts university in Honolulu, Hawaii administered a paired comparison rating sheet (one each for the instructor and all students in the class) to weight the criteria, and a student questionnaire (one for each student in the class) to assess instructor characteristics and student learning. A total of 10 faculty, 132 students, and 6 administrators participated. Using Kendall's coefficient of concordance, agreement between all three raters of instructor characteristics ranks measured 0.59. Agreement between all three raters of student learning outcomes measured 0.72. Weights from all three raters were applied to the students' assessment of instructor characteristics, the input, and student learning, the output, to calculate relative efficiencies. Students' assessment of instructor characteristics and student learning were also used with DEA to calculate relative efficiencies. Rankings of instructor's efficiency using both methods and multiple raters were tested for inter-rater agreement using Kendall's W. The agreement between rankings was very high, with an overall Kendall's coefficient of concordance of 0.95. Although these results point to no significant difference in relative efficiencies based on the weighting of instructor, student, and administrative perspectives, the use of DEA as a valid and practical method for measuring relative teacher effectiveness was confirmed.