Research On PISA Problem Solving Competency Based On Log-file Data Analyses

The rapid changes in technology in the 21st century have led to an increasing complexity of work and real-life problems,which significantly increase the importance of lifelong learning and problem-solving competency.International organizations and governments have developed the frameworks of 21st Century Skills or Key Competencies.Moreover,PISA and other large-scale assessment have change focus to the assessment of problem solving competency and other 21st century competencies.However,the results of PISA 2012 show that problem-solving skills have not been paid attention to in school education yet.The average performance in problem solving of students in Shanghai China is lower than expected,and the reason behind it is worth analyzing,since up till now,the analysis of PISA data mainly focus on the results instead of the problem solving process of the students.The study will compare the thinking process of students from different countries?regions? during PISA problem solving,and try to answer the question of how the indicators derived from problem solving process influence the results,including three sub-questions:1.How to evaluate and analyze the process of problem solving?2.In terms of the process and strategy of problem solving,what is the difference among the students from different countries?regions??3.To what extent the process indicators affect the performance of problem solving?On the basis of answering these questions,the dissertation puts forward some suggestions on how to improve the problem-solving ability of students in Shanghai-China.The study combined the research methods of comparative education,psychology and educational technology,and uses the computer-generated log-file data to track the trace of thinking of students during the process of PISA problem solving assessment.It's the first time of process data-mining of PISA problem solving in mainland ofChina.It is also the first international comparative study of PISA problem-solving process indicators that covered all the sample items and sample students.The process indicators involve problem solving strategies,metacognition,motivation and persistence,as well as basic indicators such as time,steps and order,which are difficult to obtain reliable data through pen and paper test and questionnaire.Hence,the study has made a deep development and utilization of large international comparative databases to provide a new way of thinking and methods for research on problem solving.The main findings of the research are as follows:1.Interactive problem solving can be an effective carrier for evaluating and cultivating creativity.2.It is possible to discriminate the students who are lack of motivation,to evaluate the perseverance of students reliably,to decrease the bias caused by the subjective reports in student questionnaire,and to provide a viable example of evaluation of non-cognitive factors by analyzing the computer-generated log-file data.3.The process indicators derived from log-file data can help diagnose the cause of the error and distinguish the low proficiency students more effectively than analyzing the results only,especially in the case of very low correct rate.4.The analysis of process indicator showed that the under-achievement of students in Shanghai China is mainly caused by lack of planning and the incompleteness of exploring rather than deficiency of the ICT familiarity.The performance of some test items also affected by the limited real-life experiences of the students.5.Girls in Shanghai had less interaction with the computer than boys,which explains the gender difference in static problem solving.In terms of the interactive problem solving items,the girls are significantly lower than boys on the index of VOTAT strategy and planning,which are the two main factors explain the performance on MicroDYN items.The international comparative study on the problem solving process of vocational students found that the students from Shanghai-China rank the top in the VOTAT strategy and perseverance,which laid a good foundation to cultivate the spirit of the artisans.6.The key to further improve the problem solving competency of students in Shanghai is to improve students'metacognitive ability and promote knowledge transfer and application.These abilities can be effectively cultivated early in school age and need to be designed coherently and systematically in curriculum,teaching and evaluation throughout K-12 education.