Deep Learning Library Testing Via Differential Testing And Mutation Testing

Deep learning technique has developed rapidly in recent years,and deep learning systems have been widely used in various fields and shown good capabilities.However,deep learning system may have defects.These defects may cause the deep learning system to make erroneous decisions,leading to serious consequences.Therefore,testing deep learning system has attract more and more attention.The deep learning system is consist of deep learning model and deep learning library.Most researches target at testing deep learning model and few research targets at testing deep learning libraries.Since deep learning libraries are called by many deep learning models,the defects in deep learning libraries may lead to serious consequences.Therefore,it is of great significance to study testing deep learning libraries.This paper proposes a method for testing deep learning libraries based on differential testing and mutation testing.We use differential testing to judge whether the behaviour of the model meets the expectation by comparing the behaviour of the model under different libraries,and find defects by detecting inconsistencies between deep learning libraries.We use mutation testing and generate a large number of deep learning models to test deep learning libraries based on the existing deep learning models,which improves the test adequacy of deep learning libraries.We also put forward a heuristic logic to guide the generation of models during mutation testing,so as to increase the efficiency of finding defects in deep learning libraries.In the experiment,our method detected 11 defects in several latest version of deep learning libraries.The experimental results show that our method has good ability to detect defects in deep learning libraries,and can find many inconsistencies and defects between deep learning libraries.