Font Size: a A A

Group-level Ability Assessment And Cognitive Diagnosis On English Reading Problem Solving

Posted on:2011-07-02Degree:DoctorType:Dissertation
Country:ChinaCandidate:Y CaiFull Text:PDF
GTID:1117330332465636Subject:Basic Psychology
Abstract/Summary:PDF Full Text Request
There are two schemes to realize group-level assessment. One is to realize individual-level assessment first, then to compute the means or similarities in each group which are regarded as group-level outputs. It requires that all items are answered by each individual. Another one is to realize group-level assessment directly without individual-level assessment. It requires that some of items are answered by each individual randomly. The individual and group level assessment could be realized through the first scheme, and the group-level assessment only could be realized through the second scheme. But there are some advantages with the second scheme. Such as saving the number of items that should be answered by each individual; saving the time, labor and resources; avoiding the error that might exist during the assessment of individual. So that it might perform effective and realiable group-level assessment through the second way.Nowadays the researches on group-level assessment of English reading problem solving(ERPS) was few, most of them were done through the first way overseas. But their findings of cognitive analysis might not be feasible under the context of China. Thus the main purposes of this paper were to develop a new group-level cognitive model through the second scheme and to perform group-level assessment under the context of China through both sachemes. In this paper, four studies were composed, their findings were as follows:The first one was a study on cognitive analysis of ERPS, whose purpose was to construct a cognitive model. Based on some English reading comprehension processing models, curriculum experts'analysis and English curriculum standards, 10 cognitive attributes have been found. With the analysis of test items(2007 English college entrance examinationof some province), only 7 cognitive attributes were left, four of them associated with text representation process, two related to response decision process, but the rest involved both processes. With Hierarchy Regression Analysis, it found these 7 attributes were independent and could explain 79.2% variation of items'difficult, which indicates that they were the main cognitive elements of ERPS.The second one was a study on the group-level assessment on ERPS through the first scheme. Under IRT framework, with rule space model, individual-level assessment was evaluated first, then the means of individuals'ability and each attribute master ratios(AMR) in each group were calculated as group's mean abilities and cognitive states. Study results showed all school's ability level was moderate; and all attributes were well mastered, except from attribute A5(0.6159), all AMRs were greater than 0.7. When all schools were classified to three types according to the rule of this province, the mean of ability and AMRs of them were different significantly. Furthermore, the correlation coefficients between school abilities, AMRs and mean scores of schools were 0.998 and 0.836 (mean), which indicated that the group-level assessment had a good criterion-related validationThe Third one was a study on developing a group-level ability and cognitive diagnosis assessment method/model. Based on 2GPLM, RSM and AHM, a new group-level congnitive diagnosis called CY_GCDM was developed. In this model, not only the abilities and cognitive states of group but also could report group's attribute mastery probabilities(AMP) same as the AMRs in the essence could be reported; four new cognitive diagnosis classification methods (BUG(A), BUG(B), DB (A), DB (B)) were proposed. With Monte Carlo simulation study, the feasibility and rationality of this model were verified. The diagnosis accuaracy of all suggested methods were better than BDM, and each of them had its advantage. In details, the roubustness and diagnosis accuracy of methods DB(B), BUG(A), BUG(B) were great, but the roubutness of method DB(A) was weaker than the others. As a whole, methods DB(B),BUG(A) were the most feasible.The fourth one was a study about developing a new group-level assessment method or model. Using method BUG(A) under CY_GCDM model, it found the findings were similar with the outputs with traditional method. The correlation coefficiency and average absolute error of group ability between both methods were 0.957 and 0.2318 respectively. And the correlation coefficient and average absolute error between AMR and AMP were 0.6939 (significant at level 0.05) and 0.157 respectively. All these showed the new method was technically feasible and reliable in real work. Its results would be consistent with these under traditional method.All these findings reflected that the new mehod based on second scheme was feasible and efficient. The estimation precision of GIRT was great, which was consistent with IRT. Furthermore the diagnosis accuracy with CY_GCDM model was also great. Although the results under the new and traditional method were similary, there were some advantages of the new method.First, the new method could saved the time, labor and other resources during the data collection and processing. Suppose the number of total items wasm and the answered number of each individual was k , then the data used in new method was only k/ m times of the data used in traditional method. Second, with matrix sampling design, the new method can deal with the missing data. Especially during collecting questionnaire data that were easy to become invalid, such as attitude questionaire, consumer psychology questionnaire etc, matrix sampling design can enhance the flexibility and efficiency of a survey. So the new method may be feasible to deal with this kind of data.Third, the information (such as ability and cognitive state of group, AMP) about group provided by CY_GCDM model was abundant which may be helpful to students parents, schools, and governments.
Keywords/Search Tags:English reading comprehension, Group-level assessment, Cognitive diagnosis, Item Response Theory, Rule Space Model, Attribute Mastery Probability, Attribute Mastery Ratio
PDF Full Text Request
Related items