Generalizability Analysis On Rater Reliability Of Job Evaluation In QH Company

Job evaluation is the base of compensation design. As subjective score its reliability also is rater reliability. Because job evaluation should be effected by evaluator characteristics, evaluatee characteristics, environmental conditions, and organization conditions etc., the results of job evaluation cover varied errors. It is very meaning to distinguish diversified error sources which we can decrease to improve rater reliability. But Classical Test Theory (CTT) can not tell apart varied error sources. In the contrast, Generalizability Theory (GT) can solve it.As a new test theory GT can decompose error by variance analysis technology. It is helpful to find error sources and control errors. The unique reliability view of GT can also guide us to optimize decision. Two types of evaluators are always incumbent employee and job analyst in job evaluation. The research is using GT to analyze that which type of evaluator is better in rater reliability and how to improve the evaluation reliability. The research contains two studies. The model of study one is p×i×r, in which p is position, i is compensable factor, and r is rater. The model of study two is f×r, in which f is compensation factor, and r is rater.Through analyzing data from 559 evaluation questionnaire, we found that job analysts are better on total rater reliability and factors rater reliability, and more consistent in rating loose and tight standard and cognizance of position. Meanwhile we also watch the great impact of compensable factors on rater reliability and lower reliability of problem solving.In decision study (D study) Generalizability Coefficient reaches 95%, when incumbent employees are nine and job analysts are six. With increasing raters Generalizability Coefficient improves, but increment decreases. On the basis of D study, the research submits two optimizing decision model under the limited condition of cost and reliability. These conclusions of the research should supply the valuable theoretic basis for improve the quality of job evaluation.