Research On The Static Detection Technology Of Flaky Tests

Flaky test is a type of tests which pass or fail in multiple runs.Its complex causes,uni-versality in software systems,and difficulty to reproduce lead to the reduction of software testing quality and software development efficiency.In recent years,growing researches have focused on the impact of flaky tests and how to detect them more efficiently.How-ever,these studies fall short in terms of coverage or detection performance.Targeting at the deficiencies in current researches,this paper investigates the impact of flaky tests on the field of automated program repair for the first time,and proposes a new static method to efficiently detect flaky tests in software systems for mitigating the adverse im-pact caused by flaky tests.In order to understand the hazard of flashy tests,this paper investigate the distribution and life cycle of flaky tests in the program defect dataset,analyzes the influence of flaky tests in different environments on fault localization and the performance of automated program repair tools,and concludes discoveries that worthy for attention:(1)flaky tests reduce the positioning accuracy of the fault localization tools?(2)flaky tests will weaken the performance of the program repair tools,which uncover the negative effects that the flaky tests exert on areas highly depend on test cases,such as automatic program repair.To alleviate the influence of flaky tests,this paper proposes a new static method for flaky test detection,this paper defines a graph structure based on the program dependency graph and extracts paths from it to depict the dependency relationships within the test code and between the test code and the code under test,designs a neural network model to generate a vector representation for each test case,constructs a neural network classifier to detect flaky tests,and analyzes the performance difference between our detection method and the most advanced methods through experimental evaluation.The evaluation result shows that it performs significantly better than the state-of-the-art methods in terms of accuracy and recall metrics.It is able to reduce the negative impact of flaky tests in real-world software systems,as well as improve the work efficiency of developers.