Research And Implementation Of Robust Evaluation Methods For Deep Learning Models

Deep learning models have been increasingly applied to deal with complex tasks in the real world.However,recent studies have shown that the adversarial samples formed by adding disturbances that are not easily detected by human eyes to data can effectively reduce the performance of the models and cause them to make wrong decisions.As a result,deep learning models have great security risks in some sensitive fields.Because the current methods used to evaluate the robustness of deep learning models have complex evaluation indexes,unintuitive results,and only focus on the field of image classification,this thesis respectively screens the evaluation indexes in the robustness evaluation index system of the existing deep learning models and establishes a more simplified evaluation framework.A robustness evaluation method with certain cross-task capability is designed,and a prototype system is developed based on this method,which aims to provide reliable security guarantee for deep learning model in adversarial environment.The main research content of this thesis includes the following three aspects:(1)Aiming at the problems of complex types and high redundancy among the evaluation indexes of the current deep learning model,a representative evaluation index framework of the deep learning model is constructed.The index system divides indicators according to hierarchical structure to ensure low redundancy among indicators,high correlation between indicators and model robustness and easy combination,and can complete index migration under different task scenarios.(2)Aiming at the problems that the current deep learning model robustness evaluation methods rely on specific countermeasures,high computing cost and limited application scenarios,a model robustness evaluation method based on fuzzy analytic hierarchy process(FAHP)is proposed.By using fuzzy analytic hierarchy process(FAHP)to determine index weights and calculate robustness scores of models,the robustness of different models can be compared and distinguished.At the same time,the method is not limited to specific task scenarios and has certain universality.(3)In order to improve the robustness of the deep learning model in the adversarial environment and ensure the security of the deep learning model,a prototype system was designed and implemented,which includes three functions: counter attack,counter defense and model robustness evaluation,to improve the robustness of the deep learning model.The main purpose of the system is to evaluate the robustness of different models by applying the proposed robustness evaluation method.By comparing the performance differences between the original model and the defense model,the system can provide reference for users to understand the safety of the models and further improve the robustness of the models.At the same time,the system has good scalability and supports users to upload custom models and data sets.