A Study On Content Validity Of TEM-4 Listening Comprehension(2011-2015)

As a nation-wide large-scare criterion-referenced test for English majors in Chinese colleges and universities, TEM-4 has becoming better and approaching perfection day by day. It represents the scientific and authoritative in terms of English language testing, has made a significant contribution to English testing and promoted the quality of professional English teaching in China.Since TEM-4 implemented in 1990, the validation research about it without break, but the content representative of the factors influencing test score explanation is still not sufficient, yet most of the validity research still only used quantitative analysis method to analyze the test scores directly while ignored the test skills and response format. In addition, the current domestic research for TEM validity is not enough in listening comprehension. Considered with these causes, this thesis based on the unitary view of validity, used the listening comprehension content validation theoretical framework, and verified the TEM-4 listening comprehension content validity between 2011 and 2015.This thesis aims to check whether the TEM-4 listening comprehension samples can accurately measure the target domain, namely whether the test has high content validity. In order to achieve this goal, the author read a lot of related literature about listening comprehension and content validity, combined qualitative analysis method with quantitative analysis method, carried out the following research:First of all, this thesis starts from the definition of validity, hackles its connotation and denotation systematically, then summarizes the characteristics and insufficiencies of each develop stage. Then the author analyzes the related literature about content validity and listening comprehension, produces TEM-4 listening comprehension content validation theoretical framework based on the research results of Bachman, Palmer, Weir and Messick;Secondly, combining with the requirements of the TEM-4 teaching and testing syllabuses, using TEM-4 listening comprehension content validation theoretical framework, the author detailed analyzes the test conditions, rubric requirements, task characteristics of TEM-4 listening comprehension;Finally, in order to obtain better visual data for previous researches, the author designed two steps of empirical research to measure whether the test score real reflect the listening ability of test takers. On the one hand, the author use a questionnaire which has 20 items, collecting feelings of 120 subjects about conversation, passage and news broadcast; On the other hand, the author use “immediate introspective method” to find out the difference between test score and actual score through record and analyze answer format of 10 subjects.The research results show that the test conditions, test time, test tools of TEM-4 listening comprehension meet the requirements of teaching and testing syllabuses, and come to a complete system. However, participants reflects the way of receive radio signals during the test is not a good choice.The results of task characteristics show that:(1) length of phonetic materials is reasonable while length of questions and options has a higher demand than reading comprehension at same level;(2) the percentage of new words are moderate, but concentrate on the news broadcast part;(3) topics are various and close to daily life;(4) speed and accent of speakers are satisfy the requirements of teaching and testing syllabuses, and speed rate is tend to be raise up year by year.The analysis of questionnaire shows that the evidences of TEM-4 listening comprehension are different.The perfect ideal and more ideal evidences are:(1) content and familiarity degree of topic are ideal;(2) the general difficult degree of passage part is corresponding with the requirements of teaching and testing syllabuses.The less ideal evidences are:(1) speed of phonetic materials is different from the requirements in teaching and testing syllabuses;(2) the way of receive radio signals during the test is not ideal.The analysis of answer process of test takers shows that:(1) TEM-4 listening comprehension still has speculation phenomenon namely using some content non-related test-taking skills to obtain the correct answer;(2) the proportion of checking literal understanding ability is designed too much in the TEM-4 listening comprehension.The result of T-test shows that the test score of test takers is different from their actual score, and is always higher.In a conclusion, there is no intention to judge the test itself or challenge its authority. No matter choose which kind of test as the research paradigm, the most essential thing is not the right or wrong of conclusions in this thesis, but the theoretical level it has arrived, the research method it has used and the common sense it has concluded. The author especially expects this research can fully use the relative knowledge during the postgraduate stage, enriches the empirical research evidences of TEM-4 listening comprehension content validity and provides positive backwash effects to the professional English teaching and learning. Also give some help to the improvement of test papers design and a little contribution to the development of listening comprehension.