From Context-Adaptive Evaluation Framework To College English Evaluation Standards

Evaluation plays a critical role in language learning, learner development and also in social development. With the development of language learning theories, education theories and educational measurement, the constructive paradigm of evaluation is being accepted by more and more educational practitioners, and more and more evaluation systems and assessment methods have been proposed by different researchers. However, the criticism and complaints from different parts of society about college English education all seem to demonstrate the ineffectiveness of college English instruction and the helplessness of the increasing numbers of studies in learning assessment. The desire to seek resolution to the evaluation problems has triggered the present PhD study.Study of the development of evaluation, examination of theoretical and empirical studies, and analysis of the current evaluation practice in college English instruction in China in relation to the development of evaluation indicate that there is need to construct a context-adaptive framework for college English evaluation. Therefore, this study approaches the study from the axiological perspective to analyze the reality of evaluation, the reality of language ability, the reality of language learning, and the reality of stakeholders. Adopting the reflective and basic logical thinking methods, this study proposes a Context-adaptive Evaluation Framework (CAEF) through expert consultation.The framework is composed of three elements, namely, context, decisions, and actions. The context refers the environment in which evaluation takes places, including the macro-context, the social environment, and micro-context, the pedagogical and learner's context. Within the context-adaptive evaluation framework, context affects the decisions made in terms of functions, subjects, abilities and assessment methods, and impacts the whole evaluation process. For the first time, meta-evaluation is included as one of the constituents of an evaluation framework.To seek resolution to the current college English evaluation, this study goes on to examine college English evaluation with reference to CAEF to create society-adaptive evaluation standards on the basis of needs analysis of social requirements for college English. To create evaluation standards scales, intuitive and quantitative methods are employed to develop five four-level standards scale for college English evaluation. First, the standards scale is drafted through consultation of existing scales and benchmarks including CEF, ALTE, BETS, and HKWEB. Then through the IRT analysis with the Rasch model, the descriptors are calibrated and rearranged into different levels, which are then correlated with authorized scales and tests, like CEF, ALTE and IELTS.To make the standards scale relevant to the social demands, a questionnaire investigation is conducted in Beijing, involving 758 respondents from 500 enterprises and institutions. The data from the investigation are analyzed with SPSS 12.0. Statistics from the study indicates that English is not so frequently used as it is held to be, and the demand for listening/speaking is not increasing so rapidly as commonly held. Reading is still the most frequently used and most needed in enterprises and institutions. Based on the analysis of the requirements for listening, speaking, reading, writing, and translation, the standards scale previously constructed have been improved with the levels adapted to the different requirements of enterprises and institutions.The creation of college English evaluation standards based on CAEF demonstrates the role context plays in evaluation. Descriptors which are labeled at the same level in CEF and ALTE have to be placed at different levels in the Chinese context. Some of the requirements in College English Curriculum Requirements are proved too simple (even easier than Basic English requirements) to serve as evaluation standards for college graduates. The differences that have been found in the requirements between different enterprises and institutions justify the construction of society-adaptive evaluation standards.The development of evaluation standards scale has tested a methodology to create society-adaptive scales. It proves the feasibility and necessity of applying intuitive, qualitative and quantitative methods for scale creation. The descriptors in the standards scale can be adapted for school-based evaluation standards in terms of achievement levels at different stages of language learning. The society-adaptive evaluation standards not only provide reference for learning evaluation in college, but also for reference for recruitment testing in society. As a pioneer study, this study provides reference for the amendment to College English Curriculum Requirements not only in theory but also in methodology. However, the study is still limited by the scope and depth of study. Further study needs to be conducted on a wide scale to investigate the requirements of different enterprises, companies and institutions all over the country. Authentic materials and typical performances are needed to match with the descriptor to calibrate the item difficulty. Further research needs to be conducted to study the evaluation of culture, affect and attitudes, learning strategies and literacy.