| At present,the implementation of the goal of developing students' key competencies through specific subject areas has become the consensus of educational theory research and practical change.What kind of assessment framework and test items should be designed to develop students' key competencies? This is the core issue that we need to solve in academic assessment and also a key step to decide whether key competencies can be“placed in” specific disciplines.In this study,we selected six large-scale science achievement assessment projects such as PISA run by OECD,NAEP in the United States,PCAP in Canada,NCA in England,NAP-SL in Australia and NMSSA in New Zealand,summarizes the characteristics of their assessment framework and test design,compares the similarities and differences between each other from four aspects: the content framework,the performance description framework,the sample situation and the design of test items.It is intended to provide a reference for the construction and perfection of the science assessment project based on key competencies in our country.The research contents,methods and conclusions are as follows:In the first study,the comparative study method was used to study the content framework of each assessment project.The analysis was conducted from five aspects: the basic composition of the content framework,the academic ability dimension,the subject knowledge dimension,the emotional attitude dimension,and the contextual dimension.Conclusions are as following:(1)According to the design concept,the content framework is divided into two categories.One is the knowledge-oriented content framework represented by NCA,basic structure is “content domain + cognitive domain”;the other is the capacity-oriented content framework which is constructed by PISA and the remaining four developed countries and the basic structure is “subject competencies + subject knowledge + attitude(+ situation)”.(2)The competency framework of each assessment project mainly focuses on the understanding and application of scientific knowledge and scientific inquiry.The proposed capabilities includes explain phenomena scientifically,scientific inquiry,interpret data and evidence,scientific communication,participation And contribution,problem solving and scientific reasoning.(3)There are two ways to examine knowledge about the nature of science.One is to juxtapose it with content knowledge and take up a certain proportion of assessment;the other is to examine it as a unified topic and content knowledge.(4)Emotional attitudes are more focused on “attitudes toward science” and tend to be more intentional,mainly focused on “interest in learning science”.(5)PISA and PCAP classify background from both personal experience and themes.The second study focuses on the peformance standards of six international large-scale science achievement assessment projects,intends to summarize the characteristics of performance description from presentation form,description clues and the factors of level distinction and compare the similarities and differences of description clues and factors of level distinction for the same subject ability by using comparative research method.Conclusions are as following:(1)The presentation forms of performance standard can be divided into three categories,the first category is the overall performance standards described only by subject ability level;the second category is the combination of subject ability and scientific knowledge combined with multiple performance standards;the third category is based on the “threshold criteria”based on content standards.(2)The level of description of scientific inquiry begins with the process of scientific inquiry.The level of differentiation includes the quality and quantity of the proposed problem,the level and understanding of the control variable,the scope of the situation,the level of understanding of the assessment,and the quality of the evidence.Data processing(including comprehension)is listed by many items.Descriptive clues include the presentation of data and its transformation,obtaining information in data,interpreting data and drawing conclusions and data evaluation.(3)The level-discriminating factors of “explain phenomena scientifically” include cognitive processes,the complexity of the attributes or relationships of objects,the complexity and familiarity of context,the abstractness or systematicness of knowledge,etc.(4)the level description of problem solving is not based on the specific thinking process of solving problems,but more concerned with the complexity and authenticity of solving problems,and the strategies to solve problems.(5)NMSSA defines scientific communication as“description,interpretation and construction”describing clues including the contents of communication,the presentation form of communication content and the quality of communication.(6)The level description of the nature of science corresponds to the level of scientific inquiry.The level of scientific content is described in the form of “action verb + knowledge”,and the level difference is mainly reflected in the cognitive level.The third study focuses on the items' situation of six international large-scale science achievement assessment projects,intends to summarize the characteristics of items' situation from identify parameters,content parameters and dress parameters by using the method of content analysis and compare the similarities and differences of test situation in different assessment project by using comparative research method.Conclusions are as following:(1)Compared with PISA,the situation scope of the national assessment project is mostly limited to subjects and personal life,with little or no attention to the regional and global contextual concerns.The problem setting is closely related to the specific content knowledge,and more emphasis is placed on the understanding and application of knowledge.PISA pays close attention to regional or global issues or events,context topics mostly focus on “environment and nature” and “frontiers of science and technology”,most issues are directed at procedural knowledge and epistemology but do not involve specific content knowledge.(2)The situation design of all the assessment projects is mainly based on single subject areas.(3)The types of assignments are most common among “answer choice”,“provide explanation” and “description induction”.All contain certain open questions and the answer to the question does not involve scientific formulas or complex mathematical calculations.(4)The situation is comprised of many questions,which called a “item unit”,the presentation form of situation is a mixture of text,pictures,tables and videos.(5)The PCAP attaches the attitude to the paper test questions.(6)The background of the question paper,as a material describing the situation,may provide known information.The fourth study focuses on the test design ideas of different assessment methods,such as paper-pen test,computer interaction test,on-site operation test and teacher assessment,intends to summarize the characteristics of test design and compare the similarities and differences test design for the same subject ability or the same assessment method.Conclusions are as following:(1)Computer technology has been widely used in large-scale scientific achievement assessment,including providing dynamic animations and video backgrounds,rich questions,designing inquiry tasks and interactive tasks.(2)In NAP-SL online inquiry task students are placed in a role more like observers of a practical task rather than active participantsbecause of the task without student interaction.PISA and NAEP computer interaction tasks reflect the idea of “operating-conclusion based on data-making judgments or choices-illustrating the basis for judgment or choice”,but there are differences in the specific design between the two projects.(3)Both NAEP,NAP-SL2012 and NMSSA set practical tasks for a limited period of time,relevant operation manuals are released before the operation,the operation tasks are familiar to students.However,assessment task and the focus are different.(4)Teacher assessment is a unique evaluation method of NCA.Its essence is a kind of procedural evaluation within the school.(5)There are three mainly ways to evaluate scientific inquiry.The first is to see scientific inquiry as a collection of a series of sub-skills,assess a certain skill of scientific inquiry through non-inquiry tasks and inquiry tasks,the second is more concerned with the whole process of scientific inquiry,especially the ability to design programs,the third focuses on the process of simulating or real “doing”,requiring real operation of variables or selection of experimental equipment.(6)The assessment of “explain phenomena scientifically is based on the concept or principle of “recollection” and “recognition” and requires students to“describe” or “explain” or “predict” on the basis of this concept.(7)PISA and NAP-SL's assessment of“interpret data and evidence”has some similarities with NMSSA's “scientific communication”,this is demonstrated to capture and understand the meaning of information by means of texts,tables,pictures,maps,videos and even simulations,focusing on analysis,interpretation of data,deriving appropriate conclusions and identifying assumptions,evidences and inferences in scientific texts.Overall,the assessment of“competence” orientation has become a trend of international large-scale science achievement assessment,which construct the content framework and the performance description framework based on the subject ability,clearly classifying and defining subject ability and the academic performance level,paying close attention to procedural knowledge and the nature of scientific knowledge,assessing science achievement by combining single subject with a multidisciplinary,simple situation with complex situation.However,compared with PISA,the situation of the national assessment project are mostly limited in scope of subjects and daily life of individuals,with little or no attention to the regional and global contextual concerns,focusing on the memory and understanding of specific content knowledge.Based on the above research,,under the goal of developing key competencies,we should pay attention to the following points:(1)Construct a competency framework with exclusion,pay much attation to “science communication” and “understanding of data and evidence”.(2)Construct performance level description based on disciplinary competence and content knowledge and enriching the level of differentiation.(3)Combine procedural evaluation with the summative assessment and assess science achievement in a variety of ways.(4)Emphasis on the assessment of knowledge about the nature of science and design targeted items.(5)Infiltrate the concept of "study integration" and create meaningful problem situations for student development.(6)Apply information technology and develop computer-based assessment materials and test questions database. |
PDF Full Text Request