Metamorphic Testing Of Image Classification Algorithm

In the past few years,Image Classification Algorithms,especially Deep Neural Networks(Deep Neural Networks,DNN)have been widely used in many fields,and have achieved remarkable progress.However,recent studies have shown that adversarial samples are often crafted through adversarial perturbation,i.e.,manipulating the original sample with minor modifications so that the DNN model labels the sample incorrectly.At the same time,frequent reports of self-driving accidents have heightened concerns about the DNN systems.In practical applications,DNN systems,especially in some safety-critical areas,need to be tested more thoroughly,rather than simply relying on a simple evaluation of the accuracy of the new datasetSoftware testing technology can help us evaluate the robustness of the DNN and thus detect vulnerabilities in the system.However,traditional software testing techniques,such as unit testing,cannot be directly applied to DNN system.At the same time,the DNN system defines a new data-driven program paradigm.Its internal logic is derived from the unknown inherent quantity regularity of the training data.It is extremely difficult to define its input-output relationship.Therefore,the research content of this article is to construct a metamorphic relationship(MR)that conforms to the concept of metamorphic testing for the DNN image classification system.Finally,the Variant are executed on the set of follow-up test set generated by MR and the ratio of 'killed'Variants evaluating the completeness of the MR.In this paper,we construct a Metamorphic Testing(MT)approach for DNN Image Classification Systems.This test method is based on the use of image processing knowledge to simulate common image distortion problems in real life applications,and proposes four types of MR suitable for DNN image classification systems.In order to prove the completeness of these four types of MR,experiments were performed to construct Variants and perform them.The ratio of the identified mutants was used as the evaluation criterion of completeness,and compared with the recently proposed transformation test method.Experiments show that the method proposd is better than other methods The main contributions of this article:(1)Based on the common image distortion phenomenon in reality,simulate the generation of changing use case sets,and explore and analyze the recognition mechanism of neural network image classification system.(2)Based on the concept of transformation test,and according to the influence of common image distortion changes on neural network image classification recognition mechanism,four transformation relationships suitable for neural network image classification system are constructed.(3)Through the method of constructing mutants,,the transformation method proposed in this paper can catch 64.28%of the constructed mutants,it is confirmed that the constructed metamorphic relationship has a good ability to identify mutants.