Scoring L2 Chinese Speaking Performance: Linking Scores to Candidate Performance

The major objective of the thesis is to link scores to candidate performance in scoring speaking performance for Chinese as a second language (L2 Chinese). To this end, the thesis has been designed comprising three coordinated studies, namely, Study One, Study Two and Study Three. Study One employs Traditional Scoring for obtaining traditional scores, Study Two develops Confidence Scoring for producing confidence scores and Study Three compares Traditional Scoring and Confidence Scoring.;In Study One, the relationship between traditional scores and candidate performance was examined. Seven features---representing four major categories in the L2 Chinese speaking construct---were employed. Speech samples of 66 candidates on an L2 Chinese speaking test (i.e., 9 minutes' speech length for each) were analyzed in terms of the seven features, with correlations and standard multiple regression being employed. Results indicated that, first, each of the seven features was significantly correlated to the traditional scores, producing large or medium effect sizes; second, 79% and 77% of the variance in the scores could be explained by the features involved in two regression analyses respectively. Study One therefore provides empirical evidence for linking scores to candidate performance to validate the assessment of L2 Chinese speaking proficiency.;Two problems were, however, identified in Study One when linking scores to candidate performance in using Traditional Scoring--- indistinction between adjacent levels and overlap between scales. To address these two problems, Study Two therefore proposed a new approach, Confidence Scoring, leading to raw confidence scores between two adjacent levels applied to three rating scales. Since raw confidence scores had to be transformed to an exact confidence score for score interpretation and use, membership functions and rule bases were applied and a Confidence Scoring Algorithm was developed. A pilot study was subsequently conducted in Study Two to try out Confidence Scoring. An initial comparison was also made in the pilot study between Traditional Scoring and Confidence Scoring. Results of scoring dependabilities and correlations with Key Message Points (KMPs) indicated that Confidence Scoring outperformed Traditional Scoring.;In Study Three, a mixed methods study was conducted to provide a more comprehensive picture in comparing Traditional Scoring and Confidence Scoring. Quantitative score data comprised traditional scores and confidence scores from five raters. Qualitative interview data encompassed five raters' perceptions on the scoring process using Traditional Scoring and Confidence Scoring. The analysis of quantitative score data indicated that, in relation to Traditional Scoring, a closer link between scores and candidate performance was established through Confidence Scoring. The investigation of qualitative interview data found that Confidence Scoring was based on and developed from Traditional Scoring. More importantly, while Confidence Scoring embraced Traditional Scoring, it provided a more flexible way of acknowledging and incorporating raters' confidence in scoring speaking performance.;The contribution of the thesis therefore rests on conceptual understanding, empirical evidence as well as methodological innovation---in linking scores to candidate performance---in the context of validating the assessment of L2 Chinese speaking proficiency.