Using Multivariate Generalizability Theory To Analyze Interviewer Reliability Of National Public Servant Recruitment

Generalizability theory, taking a dynamic and predictable reliability view, provides a dependable reliability analysis for complex and real-life multi-dimensional measurement through some statistical tools, for example, experiment design and variance component model. With the development of psychology and education, most psychological and educational tests are multidimensional. In order to meet the need of the multidimensional testing reliability, Multivariate Generalizability Theory (MGT) emerges as the times require. This research explores the application of MGT, including MGT model, variance and covariance components' estimation and explanation, multivariate generalizability coefficient, weight selection and so on.This research illustrates MGT through interviewer reliability analysis of National Public Servant Recruitment. The research samples 114 interviewees of 5 bureaus from certain bureaus at the provincial level. Two studies are made simultaneously. Study One is Univariate Generalizability Analysis in which the five testing dimensions of National Public Servant Recruitment are regarded as a fixed facet, and univariate'generalizability analysis is used in each condition of this facet. Study Two is Multivariate Generalizability Analysis. The interviewer error and reliability of the compositive evaluation is analyzed. By comparison of these two studies, the advantages of MGT are becoming obvious.The results of analysis by MGT cover those by UGT. Besides, covariance components offer the additional information. Making use of the profile information of multidimensional testing, Study Two discusses the covariant relationship of the interviewers' consistency in the different testing dimensions and the generalizability of the testing dimensions' special composite. The interviewer reliability index for the test battery is estimated. The research also supplies the valuable theoretic basis for optimizing the testing design and standardizing the procedure in the National Public Servant Recruitment.