Research On Induction Input Generation Of Multi-objective DNN Models And The Implemention Of System

With the progress of society,computer science has stepped into people’s life,therefore,a variety of intelligent systems have been applied to people’s real life,such as automatic driving car system and so on.However,the correctness and predictability of such a system is crucial,especially in the case of corner cases.Therefore,how to automatically generate test input for Deep Learning(DL)system has become the focus of software testing research.In order to apply traditional software testing methods to DL system,Deep Xplore test method was proposed.However,this method has the following two defects: First,when defining the differential behavior of different DNN(Deep Neuron Network)models for the same test input,the method does not consider the differential behavior of different DNN models.Second,in the process of joint optimization,only improving the neuron coverage can not fully guide the generation of test input,and there is no need to consider the neuron coverage of each layer in DNNS.In order to solve these two defects,this thesis redefines the differential behavior between different DNN models,and introduces the neuron coverage coefficient to consider the coverage of hidden layer neurons.Finally,a prototype system of test input generation and analysis is presented and verified experimentally.Experimental results show that the differential behavior and neuron coverage coefficient defined in this thesis can obtain more diverse test inputs.The main work of this thesis is as follows:(1)In this thesis,the differential behavior between different DNNS is redefined,and then the neuron coverage coefficient is defined,and a new DNN model testing framework Deep Gradient is proposed.First of all,when defining differential behavior,this thesis first inputted a set of test inputs into the tested DNN,and then added constraints in specific fields into the tested DNN again.The absolute value of the difference between neuron outputs obtained twice was used to measure the differential behavior between different DNN models.Then,in order to consider the coverage of hidden layer neurons in DNN model,the concept of neuron coverage coefficient is defined,which can make the obtained test input more diverse.Finally,the problem of generating test input is expressed as a three-objective optimization problem,which is solved by constructing a loss function and using the gradient ascent algorithm.In this thesis,a series of experiments were conducted to evaluate the method and it was found that by redefining the differential behavior between different DNN models and introducing the neuron coverage coefficient,more diverse test inputs could indeed be generated.(2)In order to solve the optimization problems of the three objectives under the Deep Gradient framework,this study associates with more direct multi-objective optimization algorithms in the field of computer.Therefore,the DEEP-NGSA method is proposed under the framework of the Deep Gradient method.The introduction of this method is mainly to avoid the tuning problem of multiple hyperparameters.Because it is difficult to find the appropriate parameter to find the maximum value of the loss function in the process of tuning.Therefore,it is more direct to use multi-objective optimization algorithm to directly optimize differential behavior,neuron coverage,and neuron coverage coefficient.In this study,a series of experiments were conducted to evaluate this method,which can generate diverse test inputs while maintaining the efficiency of Deep Xplore and Deep Gradient,meeting our expected goals.(3)A prototype system(DNNTESTING＿SY)was designed and implemented based on the above two algorithms and the test input of RT,Adversarial Testing and Deep Xplore DNN model.This thesis introduces the overall architecture design and testing process of DNNTESTING＿SY,shows the interface design of the testing prototype system,and explains the three main functional modules of DNNTESTING＿SY in detail.After debugging,DNNTESTING＿SY can be automated.