EFL Raters' Evaluation Of Chinese Students' English Writing In The NMET: The Analysis Of Rater's Psychological Conformity Effect

Research on EFL rater reliability and variability of writing evaluation has aroused much attention from different perspectives in language testing. And many linguists and language researchers have made a lot of efforts to explore the rater behaviors in the writing evaluation of the yearly National Matriculation English Test (NMET). Meanwhile, psychological issues of the EFL raters are also popular in the psychological field abroad. However, the psychological issues of the EFL raters, especially the NMET writing raters have long been escaped by the language testing researchers in the mainland China. Since NMET is a large-scale high-stakes test, whose results would be used to make important decisions for the test-takers in the entrance of the universities and colleges. Therefore, to ensure reliability and fairness is of great importance. The purpose of this paper is to probe into the psychological influence which impairs raters'fairness by tracing the NMET rating process through an experiment among 15 raters in an inland province in 2009, and to examine the psychological influence of the NMET rating on individuals, groups and the society at large.This paper gives introduction to the relevant literature on rater reliability, rater variability and bias in the field of language testing. Then building on the previous studies which have demonstrated that the individual psychological behaviors of people are subject to social factors, this paper reviews certain kind of psychological effect -- conformity effect, by which an individual's attitudes, beliefs, and behaviors are influenced by other people. While evaluating the English writing task in the NMET, to some degree, raters are frequently characterized with the informational social influence and the normative social influence, both of which are the phenomena of the conformity effect and these are short of empirical studies in the field of psychology.The present study was carried out with a combination of qualitative data and descriptive statistics. Data analysis involved coding, recording and decoding of the responses from the questionnaire and the verbal report, and descriptive statistics were used to show the results of data analysis. And 15 raters were selected randomly from the entire pool of writing raters and an experiment was implemented among the 15 raters. And with these, this paper gives adequate support that either normative social influence or informational social influence upon individual raters will be greater among individuals forming a group than among an aggregation of individual raters who do not compose a group in the NMET rating group. And normative social influence to conform to one rater's own judgment from another as well as from oneself will be stronger than normative social influence from oneself. And this paper gives further support that the more uncertain the individual rater is about the correctness of his or her own writing evaluation, the more likely he or she is to be susceptible to both normative and informational social influences in making his or her evaluation. And at the same time, the more uncertain the individual rater is about the correctness of other raters'writing evaluation, the less likely he or she is to be susceptible to informational social influence in making his or her evaluation.The results of this paper indicate that the two phenomena of the psychological conformity effect, both normative social influence and informational social influence, will affect rater reliability and result in rater variability to some extent. Though a few doubts remain, the findings have implications for assessing the quality of large-scale rater training and improving the rating reliability. But still, there is much scope for improvement.