Research On Testing Technology For Deep Learning Object Detection Model

Deep learning model has been widely applied in computer vision and raised more concern due to its security issues.This reveals the fact that the quality of deep learning model is not guaranteed,and implies the necessity and urgency of adequately testing and evaluating deep learning model.Testing methods for traditional software no longer works for deep learning model owing to black-box property of the deep learning model,so there is an urgency need for new testing methods and tools for deep learning models.How to systematically,comprehensively,effectively,and adequately test deep learning models has become an urgent need.Currently,as representative deep learning testing techniques,neuron coverage and adversarial attack attracted greatly concern.Neuron coverage get limited testing results as it can analogous to code coverage in traditional software.Even so,it’s toughly to improve the neuron coverage rate while in complicated model.Adversarial attack is a powerful tool for attacking deep learning model,but the generated adversarial examples have limited in improving neuron coverage.Therefore,there is an urgency need for looking for efficient test cases.Based on the theory of deep learning testing,using the deep learning model for object detection as the tested object and with the research goal of improving the adequacy and effectiveness of deep learning testing,our paper put forward a testing framework for deep learning-based object detection model carry out research around key technologies such as the adequacy of deep learning model testing and test case generation technology,and finally,we designed and developed a prototype tool of the deep learning model testing for object detection.Our paper verifies the correctness and effectiveness of the proposed technology and algorithm through experiments.The main innovation points of the paper are as follows:(1)Aiming to how to systematically,comprehensively,effectively,and adequately test deep learning models,we propose a testing framework for deep learning-based object detection model.The framework analyses and defines the connotation of test cases,tested models and test outputs in deep learning-based object detection model testing.The framework points out that test cases can be generated by metamorphosis testing and adversarial attack,while neuron coverage theory can be used to measure test adequacy,and the test output can be used to calculate and analyze various indexes of the evaluation model.The contribution of this framework is that it provides a scheme of how to systematically,comprehensively,effectively and adequately test the deep learning-based object detection model.(2)Aiming to the limited neuron coverage testing results,we propose a neuronpath-based neuron coverage criterion.Based on the neuron activation that make up the path under different test inputs,our paper proposes k-length path segment coverage and k-length path segment activation coverage.Compared to neuron coverage and kmultisection neuron coverage,they are more sensitive to images with significant perturbations.Therefore,k-length path segment coverage and k-length path segment activation coverage can be combined with neuron coverage and k-multisection neuron coverage to improve testing adequacy.(3)Aiming to how to generate efficient test cases,we propose a test case generation algorithm DWP and a test case set image sorting algorithm TSNC based on neuron coverage.Among them,DWP algorithm combines and optimizes Deep Xplore and PGD algorithms.DWP generate test cases with error-caused and neuron-coveragerate-improved.For example,the images DWP algorithm genarated can improve Faster RCNN’s neuron coverage rate from 0.51 to 0.59 and m AP value is reduced from 0.59 to 0.32.TSNC algorithm sorts the test case set images according to the number of neuron activation times and replaces big test case set with small test case set while remaining the neuron coverage rate,so as to simplify the test case set.For example,TSNC algorithm changes the number of input images needed for Faster RCNN to reach the peak of the neuron coverage rate from 6000 to 4000.(4)Aiming to improving the neuron coverage rate,we show several suggestions to do it.Our paper studies the neuron coverage criteria based on metamorphic testing,and draws the following conclusions with experiments: high resolution and bright color images can improve the neuron coverage and k-multisection neuron coverage,and images with significant perturbations can improve the k-length path segment coverage and k-length path segment activation coverage.The above conclusions can provide reference for testers to conduct neuron coverage testing.