Global Inte Rp Retation Of Gbdt Model Based On P Robabilistic Finite-state Automata

With the rapid development and wide application of artificial intelligence,machine learning interpretability becomes critical for users to understand and trust the decision results of machine learning models.However,most machine learning models are black box models,whose internal complex structure and huge parameter scale make it difficult for users to understand the decision of the model.Therefore,when deploying machine learning models with excellent performance in key application scenarios,the research on the interpretability of the model itself becomes particularly important.This thesis studies the interpretability of machine learning models,summarizes the classification and research status of interpretability,analyzes and compares mainstream interpretable tools,investigates how to introduce probabilistic models into interpretable work,and proposes a global interpretation method of ladder boosting decision trees based on probabilistic finite state automata.Namely,a probability model is extracted from the trained model to be explained to explain the original model,and a modelspecific global interpretation method is implemented.The object of this thesis is the gradient boosting decision tree.This model is widely used in many industrial scenarios.Its interpretability is slightly higher than that of the complex neural network,and it has the same accuracy as the neural network in some application scenarios.The main research contribution of this thesis include:(1)A global interpretation method of gradient boosting decision tree based on probabilistic finite state automata is proposed.This method extracts important cross features from the trained gradient boosting tree,constructs an abstract sequence set through filtering,clustering and other operations,and then uses the merging algorithm to transform the abstract sequence set into a probabilistic finite state automaton.By adjusting the parameters of each link and training constantly,the probability model can simulate the decision of the original model as much as possible,that is,to achieve high fidelity.At this point,the extracted important cross feature is the global interpretation of the original model gradient boosting decision tree on the dataset.With the growth of the scale of the given gradient boosting decision tree model,the proposed method in this thesis maintains a high fidelity of the probability model.(2)Design experiments and case analysis for the interpretation method are proposed in this thesis.We build and train models on several commonly used tabular datasets,optimize the construction algorithm process of the probability model to improve the fidelity of the probability model to the original ones,compare experimental results with mainstream interpretable tools,and analyze the coverage and accuracy of interpretation methods.The use method of the obtained global interpretation is briefly described through case analysis,that is,the global interpretation is used to directly predict the test samples.(3)Apply the global interpretation method proposed in this thesis to the crime prediction task,and conduct fairness analysis.For the crime prediction task dataset,the dataset is preprocessed through data cleaning,label sorting and feature engineering,and the gradient boosting decision tree is used for training and prediction.On this basis,the probability model proposed in this thesis is used to extract important cross features,and the important cross features are used for fairness analysis of the model.