A Research On The Correlation Among Test Suite Size,Test Coverage And Test Effectiveness

Software testing technology is one of the important ways to guarantee software quality.Previously,researchers mainly used test coverage to evaluate testing quality.Test coverage is mainly used to measure the extent of the source code is covered.The more source code is executed during the test,the higher test coverage can be achieved.Therefore,it is generally accepted that the test coverage is positively correlated with the defect detection ability.However,the latest research shows that test coverage can not objectively evaluate the defect detection ability of a test suite,and there is an inseparable relationship between the size of the test suite and test coverage.This leads to the following research question,that is,how to explain the relationship between the size of the test suite,the test coverage and the test validity? In order to study the relationship among them,this paper presents a new algorithm to select test cases,which makes each test case has the same probability to be selected.Taking the six source projects in Defects4 j dataset as the experimental subject,which all have high utilization rate,generating two kinds of test suites with test method and assertion as different granularities.These test suites are used to conduct an empirical study of the relationship between the three.The main research work of this article includes:(1)Whether the test coverage is effected by the number of test cases.The experimental result shows that the size of the test suite is larger,the test coverage is higher,which means there is a positive correlation between them.(2)Whether the test validity is effected by the number of test cases.The experimental result shows that the size of the test suite is larger,the test validity is higher,which means there is a positive correlation between them.(3)The relationship between the test coverage and the test validity.The hybrid effect model based on linear regression is used to characterize the relationship between the three.This paper finds that there is highly correlation between the test coverage and the test validity when the test suite is not considered;there is no significant correlation between the test coverage and the test validity when the test suite is considered.