The Adjustment Of Subjectively Rated HSK[Advanced] Composition Scores

Compared with objective test, subjective test has some insurmountable obstacle, i.e., subjective test is always scored by two or more than two raters , which results in inconsistent scores. That means a score does not always presents the ability of examinees who take the test, so raters inconsistency is the main reason for rating discrepancy which needs to be rescored. To guarantee a high validity of subjective test, a high reliability is required first. Researchers had done a lot over the past years in this regard. But none of these methods can really solve the problem of subjectivity during the rating. In view of the facts mentioned above, we attempt to use the uAdj scheme introduced by Longford in 1995 to tackle this problem.There are two purposes in our study. One is to use the uAdj scheme to assess the reliability in subjective rating and to adjust scores given by outliers in HSK(advanced)compositions. This method which is based on the generalizability theory and variance component analysis can evaluate the variances of true score, severity of rater and that of inconsistency (rater-by-examinee interaction) ,and it can also make an in-depth assessment of the ratings of every rater and each pair of raters who scored the same examinees. So it can provide an overall assessment of the rating process. The other is to compare two schemes, namely, uAdj scheme and sAdj scheme since there are some problems while using the sAdj scheme.For what we got from uAdj scheme, we can see clearly that the benefits of the adjustment is noticeable and that uAdj scheme can achieve better results than sAdj scheme, which means sAdj scheme is much more conservative in nature , but we at the same time still found that uAdj scheme can not completely solve all the deficiencies sAdj scheme has, on which we will conduct further research and studies in the years to come.