| This study seeks an innovative testing system in which trained students participate in a writing test as a complementary rater, specifically in the EFL context. The study has three research goals: (1) to investigate the psychometric reliability of self- and peer-ratings in a second language (L2) essay-writing test in Korea, (2) to explore students' rating process, and (3) to understand student's perception of rating tasks. The rationale of this research is that the Korean EFL students whose major is English education should conduct the rating tasks that they will encounter in their future career. Each of 104 Korean EFL learners at the intermediate level sequentially rated three sample essays, his/her own essay, and his/her partner's essay after he/she was provided with the same benchmark as two experts would refer to. Each essay was rated with a 5-scale in terms of 4 writing features (Organization, Content, Language Use, and Holistic feature). After students finished the rating tasks, they filled out a questionnaire. The students' ratings then were compared to expert ratings, using Multi-facet Rasch model. The quantitative research was accompanied by the qualitative study. Eleven students participated in semi-structured interviews. For the internal and external validity of the research, the researcher took use of three types of triangulations: data triangulation, methodology triangulation, and investigator triangulation. Results showed that peer raters were so lenient that they showed unpredictable rating patterns. From the qualitative analyses, students actively utilized a variety of rating strategies. In addition, they showed the positive attitudes toward the rating tasks. Most interviewees, however, viewed that they were not able to be an expert rater because of their linguistic incompetence. It was concluded that the collaborative rating procedure fosters students to activate high-mental function and a social skills, raising students' critical awareness of their social context. |
