Automatic Item Generation And Verification Of Probability Word Problems

The purpose of this study is to explore the cognitive attributes that affect the difficulty of probability word problems through verbal reports,and to verify and estimate the parameters of the cognitive attributes through Rasch model,LLTM model and LLTM + ? model,and finally to determine the item generation template through the determined cognitive attributes.The automatic generation of probability word problems is realized by using the template method of automatic item generation technology,and the effectiveness of the item generator is verified.In the part of review,we introduce the related research on the process of probability word problem solving and the new progress of automatic item generation technology.Considering the question of high cost of manual preparation of test in the real teaching environment,the insufficient number of the question bank leading to item exposure,item parameter drift and other issues,this paper proposes to generate a large number of probability word problems with known difficulty parameters based on template method.The main results are as follows:1)According to the thinking aloud materials of solving probability word problems collected from verbal reports,12 generic concepts and 6 related concepts were obtained by using the three-level coding system of grounded theory.The six related concepts are understanding of the key words of the question,addition operation of mutually exclusive events,multiplication operation of independent events,multiplication operation of dependent events and permutation and combination.The core concept of understanding the key words of the question is problem translation,and the core concept of the other five related concepts is problem integration.2)Rasch model,LLTM model and LLTM + ? model were used to fit and estimate the model parameters of the answer data of probability word problems based on the six cognitive attributes of the first study.Through the model comparison,it is found that the response data in the LLTM +? model does not fit better than the LLTM.The results show that three of the six cognitive attributes extracted from verbal reports have significant effects on the difficulty of probability word problems.Among them,permutation and combination attribute has the greatest impact on the difficulty of probability word problems,followed by multiplication of dependent events,and the smallest impact is understanding of key words of questions.After the data fitting of the LLTM model with the non-significant attributes removed,it is found that the model fitting is not significantly better than the LLTM model before the removal,so six attributes are retained for the template preparation of the third study.3)Based on the cognitive attributes determined in the second study,an item template with the difficulty distribution from-1.77 to 2.08 was developed.The automatic generation of probability word problem is realized by R language code + markdown text document mode.The reliability and validity analysis of the generative test showed that the internal consistency coefficient of the generative test was 0.76,and the replica reliability of the parallel test was 0.77,which indicated that the generative test had good consistency and stability.The correlation coefficient between the difficulty parameters of the generated items and those of the template items was 0.89,and the correlation coefficient between the scores of the generated items and those of the Mathematics scores in college entrance examination was 0.51.Experts and student reviewers could not distinguish the automatically generated items from the hand-made items,which indicated that the automatically generated items had good criterion-related validity and ecological validity.The automatic generation of probability word problems is basically realized.Conclusion: the cognitive attributes based on verbal reports can explain the difficulty of probability word problems to a certain extent,up to 76%.The automatic generation algorithm of probability word problem based on template method can generate the test of probability application questions which can reach a certain quality standard.