| The dissertation aims at calibrating a work performance assessment instrument to support continuous Improvement of Cooperative Education curricula. The calibration is pursued by determining a validity envelope for the instrument, by developing a theoretical framework for the uncertainty aggregation, and by testing the performance of the instrument in a real world environment. The instrument is further tested against the hypothesis: "It is possible to generate statistically valid data using the specific assessment instrument". The literature review focuses on continuous improvement of academic organizations, naming competition between institutions, declining societal resources, and outcomes-based accreditation philosophies as being the main drivers of the present development. The literature review covers research pursued with regard to the consistency and the relevance of the assessment instrument under scrutiny. The initial analysis compares assessment data obtained in business, technology, engineering, and design environments, showing that the instrument is most suitable for curricular development of mandatory engineering or engineering technology co-op programs. As a result, the Civil and Environmental Engineering program is chosen for further scrutiny. The dissertation continues to discuss the development of a theoretical framework for uncertainty aggregation of results. The developed model aggregates type A (random) uncertainty and type B (systematic) uncertainty, using a combination of statistical analysis and measurement theory. The underlying uncertainty levels are approximated using a test in which assessors assess mutual students. The results show an interrater reliability of 63%, a systematic uncertainty of 1/3 of a unit (on a one to five Likert scale), and that outcome uncertainty is dependent on filed returns (n values) and the amount of participating assessors (k values). The model is applied on the Civil and Environmental Engineering Program. Results show that whenever specific boundary conditions are met, the instrument produces data significant from both a statistical and a measurement uncertainty perspective. |
