A New Experimental Scoring Method Of TEM4

The present scoring method used in TEM4 dictation is the conventional count-and-deduct approach which is mistake-based and negative. The method recognizes the mistake into two levels:major mistake and minor mistake, then assigns two corresponding score levels. The final score is arrived at by counting the wrongly spelt or omitted words in each sense group, then adding up the corresponding score values and deducting them from the total score. The method is somewhat problematic. For one thing, it attaches great importance to language discrete points and the accuracy of duplication of the text dictated and ignores greatly the import of discourse meaning, which runs against the relevant test syllabus and specifications. Although it is basically objective and relatively highly reliable, it sadly lack in construct validity. For another, the scoring approach is dichotomous in essence and hence cannot forecast what students will produce in the assessment. Thirdly, the impact of using the approach on language teaching and learning is mainly negative and obstructive.This paper suggests a new experimental marking scheme——partial credit scoring method. The new method not only adopts partial credit scoring based on determining three standards, but also reserves traditional scoring method (with semantic groups for scoring standard). When rating the materials, we establish three criteria, i.e., integrity of the discourse meaning, accuracy of grammar and correct spelling. We use unified multi-level scale to rate the papers and based on the semantic group all the same.Our experiment data comes from a model TEM4 of Shanxi Agricultural University Foreign Language School, second-year students,28 test takers on April,20 in 2010. In terms of test content, test items, test quantity and test weight, etc, this simulation test is consistent with a real TEM4. And dictation is one item of the model test. It is 152 words long in the essay, include 8 sentences and 15 semantic groups. After the test, we had all the papers copied, then the papers were divided into two sets. Raters awarded their score using traditional TEM4 method for one set, then we got six sets of scores we called old scores, while the other set of papers was rated again according to our new experimental marking scheme——partial credit scoring method. We got the other six sets of new scores. In order to test the integrity of the rating scale and whether it can reflect the ability and target involved the test syllabus, we calculated correlation coefficient between the new composite result and other scores of each items (SPSS 13.0).According to the new experimental results and the data analysis, the new experimental marking scheme can be adopted to measure examinees'dictation ability accurately. In despite of such research results, the new method also needs further modification and validation, such as point description. In comparison with two methods, the conventional count-and-deduct approach has its own advantages:it simplifies many complex problems, simple and convenient, rapid and cost-effective. What is more, raters can be highly reliable to be a marker with proper training. But the advantage of the new experimental marking scheme is more fair and effective, can ensure the validity of the whole test and represent the multiplicity of ability standard in test syllabus. Owning to the rise of average scores of examinee, the total scores of TEM4 will be improved. In addition, the experimental partial credit scoring system is a cumulative-score approach, which is a positive effect in English teaching and exam preparation.