A Ground-truth-based Framework For Evaluating Explanation Methods And Its Application

In recent years,research on the explanation of neural network models has developed rapidly.However,current research in the evaluation is still limited.The evaluation of explanation methods falls into two aspects,model and user.In the model aspect,a widely used approach is to perturb the input based on the explanation.This method is essentially a validation tool and does not know the actual feature relationships.As a result,it performs poorly when facing multiple related features.In the user aspect,commonly used techniques include questionnaires and physiological monitoring.They all require users to participate,which is time-consuming and costly.To address these problems,we propose an a priori evaluation framework that does not require users and apply it to multiple explanation methods.The evaluation framework is made up of the metric,the ground truth and the white-box models.The design of the evaluation framework is based on the ground truth.It is clear from the definition and the evaluation criteria that the explanation focuses on the mechanisms of the model and the needs of users.We have designed three perspectives for the evaluation,including perceptive,cognitive and intuitive.The perceptive perspective is concerned with the ability to capture the perceptual area of the model.The cognitive perspective is concerned with the ability to know the attributed area of the model’s decisions.The intuitive perspective is concerned with the performance on user intuition.The framework defines three types of ground truth corresponding to the evaluation perspective.The evaluation is achieved by comparing the results of the explanation methods and the ground truth,both of which are obtained by the white-box models.Considering the function of neural networks,we use traditional image operators and classification rules to construct the counting and classification white-box models.We design a metric based on the Hamming distance.Experiments are also designed to apply the framework to three classical explanatory methods,including Occlusion,Saliency and LRP,providing both qualitative and quantitative assessment results.We provide both model-and user-related evaluations in our framework.By quantifying quantitative ground truth,we avoid some of the previous problems and allow the evaluation is no longer be influenced by the results of the explanation methods.The experiments show that the framework can be applied to different tasks and extend to natural datasets.Thus,it can be considered as an effective assessment tool.In practical applications,this evaluation framework can be used as a low-cost preliminary filter tool for explanation methods.Building an evaluation platform based on user requirements allows users to select the explanation methods that work better in the given situation.