Research On Online Judge System Test Data Optimization Method Oriented To Fault Localization

With the development of computer programming education,nowadays,more and more colleges have developed their own online judge systems for programming skills training,competition qualifications and automatic assignment evaluation.The traditional online judge system can automatically evaluate the results of programs,but it cannot provide feedback of which statements in the program may contain faults.For most introductory course students,who lack programming experience,it is difficult for them to associate such simple platform interaction information with the root cause of the fault in the program.The location of faulty statements in faulty programs is a kind of effective personalized guidance information,which helps students better understand and fix the bugs of programs.Program automatic fault localization technology aims to automatically locate faults in programs,and it is a popular research issue in the field of software engineering and software testing.Existing researches on automated fault localization technology are mainly designed for industrial programs,not on student programs.Unlike industrial programs developed by skilled programmers,student programs have various fault types and high fault density.As a result,when existing automated fault localization technologies are used in student programs,the accuracy of them needs to be improved.In this paper,we first combine the automated fault localization technology with the online judge system,and evaluate the effectiveness of several widely-studied automated fault localization technologies on student programs through empirical researches.It is found that the original test data used to evaluate programs in online judge system is mainly designed to detect faults in the program.When using these original test data to locate bug in student programs,the accuracy of fault localization is not high.Subsequently,by studying the relationship between test data granularity and fault localization,we propose a fine-grained method to improve the accuracy of fault localization by refining the test data.To evaluate the effectiveness of our method,we conduct empirical researches in 1136 student programs.The experimental results show that compared with the original test data,using the refined test data under the same automatic fault localization technology,the fault localization accuracy evaluation is averagely improved by 23.14%.Both the original test data in the online judge system and the refined test data of the previous study need to be carefully constructed manually,which requires a lot of labor cost.In order to automatically generate highquality test data for fault localization,this paper proposes a genetic algorithm-based fault localization-oriented test data generation method,which takes the faulty programs in online judge system as training data,designs fitness function for evaluating the fault location ability of test data,and automatically evolves to generate test data with continuously improved fault location accuracy.The experimental results of 1136 faulty program with 7 real problems show that compared with the original test data,the test data generated based on the genetic algorithm can effectively improve the fault localization accuracy;compared with the refined test data,the fault localization accuracy has no significant difference under the same fault localization technology.Finally,based on the above-mentioned research results,this paper designs and develops a programming teaching assistant system based on B/S architecture to fit the actual teaching needs.The system guides students to quickly locate program faults and fix programs by generating diversified interactive information.On the one hand,it can help teachers save a lot of time and labor cost;on the other hand,it helps students improve programming experience and enhance programming confidence.