Influences of information acquisition and method of rating, and in-role versus extra-role behaviors on rater accuracy, halo, type and amount of search

This study extends cognitively-oriented performance appraisal research by addressing two primary questions: (1) does the manner in which rating scales are organized affect the type of ratings made? and (2) what behaviors do raters consider relevant when rating job performance? To study the first question, a 2 by 2 factorial design was used, where method of rating (by person, versus by dimension) and prior knowledge of format were manipulated as between-subjects variables. The second question was addressed by creating performance dimensions which captured either in-role or extra-role (i.e., organizational citizenship) behaviors. The performance levels of six ratees were experimentally manipulated as within-subjects variables, using three levels of in-role and two levels of extra-role performance. Subjects were 116 supervisors from a large university, randomly assigned to between-subjects condition. A computer simulation was devised, which asked raters to search for performance information, and then make ratings for each ratee. Hypotheses were tested using overall performance ratings, halo, accuracy, type of search, and amount of search as dependent measures.; Method of rating had little impact on overall ratings but had a large impact on two measures of halo. Halo was lowest when rating by dimension, and when subjects had no prior knowledge of format type. Results were mixed for the effects of these variables on rater accuracy. In general, accuracy was better when rating by dimension, but effect sizes were small and inconsistent. No effects were found for these manipulations on type or amount of search.; Level of in-role performance, level of extra-role performance, and their interaction each explained statistically significant amounts of rating variance. Halo was increased for ratees exhibiting high levels of extra-role performance, but contradictory results were obtained for two accuracy measures (stereotype and differential accuracy). Finally, level of in-role performance had some impact on amount of search by ratee.; Overall, results from this study provided partial support for the first research question, and strong support for the latter. A proposed interaction between method of rating and level of extra-role behavior was not supported. Implications are discussed.