In the Service of the Stakeholder: A Critical, Mixed Methods Program of Research in High-Stakes Language Assessment

The three studies presented here represent a two-year program of research that critically explored one case of high-stakes language assessment---English proficiency assessment for teacher certification in Quebec, Canada.;This program of research has been critical in that (1) it has integrated social and political values into the test validation process; (2) test stakeholders, including the test takers themselves, have not only been consulted but have determined the direction of the research program to a great extent; and (3) the conflicting views and competing interests of the stakeholders have been embraced and have enriched the research program.;These studies will make contributions to the field of language assessment, and in particular, in better understanding how all elements of the subjectively-scored assessment situation interact.;Because of its critical approach, these studies have demonstrated a responsible and progressive approach to researching the assessment of language proficiency for professional certification. In addition, the use of mixed methods designs in all three studies has been somewhat innovative and will add to the emerging field of mixed method research.;The first study was an examination of the final administration of a writing test used for this purpose at one Quebec university before this test was replaced. In this study, mismatches were revealed in stakeholder perceptions of the task to be produced in this assessment. The second study examined the pilot administration of a new replacement test. It focused on the socio-political environment of the test---namely, how the perception of high or low stakes by raters affected scoring. Results from this study suggested that test stakes---to all stakeholders, including raters---is a worthwhile focus of study. The third study examined the first official administration of the new test, and focused on rater behavior from a socio-cognitive perspective, suggesting that information on decision-making style may provide insight into variability in rater scoring.