An Empirical Study On Reliability Of CET-SET From The Perspective Of Multivariate Generalizability Theory

College English Test-Spoken English Test (CET-SET) has been administered for more than a decade and it has achieved great success and attracted ever-increasing population. But there is very rare studies conducted on CET-SET. It is generally claimed that the subjective test, as CET-SET, maintains high validity but low reliability. So it is worthwhile of examining the reliability of CET-SET in current administering context.Combining variance analysis with CTST, GT has powerful function in estimating the reliability of various tests, especially the subjective test, which makes the specified errors distinction possible. Upon the variance and covariance components estimating of each facet in the current administering context, the reliability situation of current context as well as that of more other context can be estimated. MGT is the extension of UGT. It is much powerful in that it may examine the composite reliability of those tests whose measuring object consists of several variables.This thesis, in the framework of MGT, conducts an empirical study on the reliability situation of CET-SET both in current context and in more other context, which may provide information and suggestions for the test designer as well as test authority. The data used in this study is the raw scores of the CET-SET administered in November,2010 in one test district.This study mainly focuses on the reliability tendency of each scaling dimensions as well as the composite reliability tendency with the varying of rater number or scaling weights ratio. In addition, it attempts to check the reliability situation on each task (topic).The main findings are as follows. Firstly, the CET-SET maintains very good reliability in current administering context. Secondly, in the three scaling dimensions, all appear good reliability value in most of the testing rooms. Comparatively speaking, the Accuracy and Range scaling dimension possesses higher reliability in most testing rooms; the Size and Discourse Management scaling dimension comes second, Flexibility and Appropriacy scaling dimension comes last. Thirdly, the more the rater number is assigned to each testing room, the more reliable the CET-SET is, and considering the reality, it is reasonable to believe that assigning three raters to each testing room will achieve the best efficiency. Fourthly, although there is still room to improve the reliability from the scaling weights ratio facet, but no proper weight ratio may provide better reliability value than the current one, thus it is reasonable to employ the current weight ratio. Fifthly, all the reliability situations on the 4 tasks are acceptable; but task 2 is far inferior to the other 3 tasks in reliability. Even in some testing rooms, the reliability of task 2 is too low to accept. Hence this task should be examined and analyzed to prevent from similar tasks being used again in CET-SET test of future.Addition to the above five findings which match with the five research questions putting forward, two other affiliated findings are gotten. First, the high correlations between each other among the three dimensions indicate that there exists "halo effect" in the process of rating, which may cause source of error. Thus special training is suggested to avoid such phenomenon. Secondly, a tendency can be found with the help of Design 2 that with the time passing by or with the rating experience gathering, the raters may do more and more reliable rating in the test. This tendency should inspire the CET-SET test designer and administrator that a warm-up should be arranged before the CET-SET starts.The present study, exploring the reliability of CET-SET in the framework of MGT, can provide information and suggestions for improving the administering context of CET-SET, meanwhile, it may set example for the similar subjective test. Therefore, this study attaches great significance in both practical value and theoretical value.