Cost Reduction Techniques For Boolean-specification Testing Based On Higher-order Mutation

Mutation testing is a general method for evaluating the effectiveness of test suites.In the process of mutation testing,test suites are executed on plenty of automatically generated mutants,and the kill rate of these test suites on the mutants is calculated.Mutation testing is a costly technique due to several mutants and the high cost of test suite execution.To reduce the cost of mutation testing,attempt to reduce the number of mutants by combining multiple first-order mutants into higher-order mutants and replace traditional mutation testing with weak mutation testing that focuses only on the program’s internal state.As a common weak mutation testing technique,fault-based Boolean-specification testing sometimes suffers from an excessive number of mutants.Therefore,this paper tries to reduce the cost of Boolean specification testing by using higher-order mutation.Higher-order mutants are by executing multiple mutation operators on the source program.This paper takes second-order mutants as the representative of higher-order mutants.Since only independent second-order mutants can be the merger of two first-order mutants,this paper divides the second-order mutants into independent second-order mutants and non-independent second-order mutants.This paper takes independent second-order mutants as the experimental objects.This paper takes 20 Boolean expressions extracted from the TCAS system as the experimental objects and generates 19131 first-order mutants and 39011764 independent second-order mutants for empirical study.The main points of this paper are as follows.(1)The fault detection difficulty of first-order mutants and independent second-order mutants are experimentally compared.All test suites that can kill the mutant are constructed for each mutant.The fault detection probability,the ratio of the number of test suites that can kill the mutant to the number of all test suites,is used to measure the difficulty of fault detection for that mutant.The results show that independent second-order mutants are to detect than first-order mutants.(2)The fault detection capability of the adequate test suite for higher-order mutants is experimentally investigated.The failed test suites for independent second-order mutants are reduced based on the greedy algorithm to obtain the second-order adequate test suite,and the percentage of first-order mutants killed by the adequate test suite for second-order mutants is a measure of the fault detection ability of this suite.The results show that the adequate test suite for independent second-order mutants can detect a high percentage of first-order mutants.(3)Based on the above findings,this paper proposes a Boolean-specification testing cost reduction method based on higher-order mutation,focusing on the cases in which second-order mutation can reduce the mutation testing cost and which first-order mutation should be combined when constructing independent second-order mutation.The results show that higher-order mutation can reduce the Boolean-specification testing cost when the fault classes include ASF or when the fault class is CCF+ENF,CCF+LNF,CDF+ENF,CDF+ORF,ENF+LRF,ENF+MLF,or LRF+ORF.