Gender and Occupational Knowledge: The Effects of Rater Variability on the Assessment of ESP Oral Performance

The overarching aim of the research is to find out empirically the effects of rater variability on the assessment of ESP oral performance, and provide empirical evidence for the possible interaction between individual rater variables. It also sets out to investigate whether the impacts of rater variability on the assessment of ESP oral performance are consistent across various types of oral task.;The data for this research were collected from 360 rater participants. Based on the finance or accounting qualifications they held, the 360 participants were categorized into three occupational knowledge groups: good mastery group (raters with an accredited senior qualification), basic understanding group (raters with an accredited junior qualification), and no understanding group (raters with no accredited qualification). The research showed that both raters' gender and their level of occupational knowledge would affect the assessment of ESP oral performance, depending on the specific testing context. The quantitative analyses of ratings of ESP oral performance revealed that in the assessment of ESP oral performance on each type of oral tasks, significant effects of the two rater variables would be identified only when particular rating scales were used. Meanwhile, the quantitative results also showed that the rating scales that would result in significant effects of the two rater variables on the assessment of ESP oral performance varied with the task type. These findings indicate that the effects of raters' gender and their level of occupational knowledge on the assessment of ESP oral performance are context-specific. The specific testing context can be defined by two factors: the type of oral task that is used to elicit the ESP oral performance and the rating scale that is used to assess the performance. The interaction between raters' gender and level of occupational knowledge in the assessment of ESP oral performance was also found to be context-specific, depending on both the rating scale and the task type. Both the effect size and the effect direction of the interaction were found to vary with the specific testing context.