Assessing and addressing random and systematic measurement error in performance indicators of institutional effectiveness in the community college

Over the past quarter-century, there has been an increased call for colleges and universities to better demonstrate their institutional effectiveness. This quantitative study assessed the existence and degree of random and systematic measurement error contained in performance metrics used under an accountability system for community colleges to determine the stability, consistency and validity of the indicators to measure institutional effectiveness. A binary graphic display was also employed to visually represent the impact of improving intermediate outcomes results (momentum points) such as student persistence to predict subsequent success on later, terminal outcomes of student goal achievement. Test-retest reliability and internal consistency results for the performance indicators were strong, suggesting that the measures are stable and internally consistent and that random measurement error was minimal for the measures. Multivariate correlation analysis revealed that the performance measures did contain systematic error due to socioeconomic factors irrelevant to the construct of institutional effectiveness. Residual analyses were conducted to identify over and underperformance controlled for the presence of systematic error and to more accurately understand how and why institutional differences arise and to assess how those colleges who perform less well can be brought about to achieve better.