A Study On Feature Impact In Complex Prediction Model

In recent years,with the abundant data resources,machine learning models have been widely used to explore hidden patterns and trends behind data,and have been initially applied in many fields,such as medical care,finance,marketing and so on.Although machine learning models has a certain wide range of applications and advantages,there are still some people hold a negative attitude towards the adoption of machine learning models due to lack of understanding and experience.And there are many barriers to companies adopting machine learning models in practice.It shows that there are many barriers to using machine learning models.In the pursuit of high prediction accuracy,many machine learning models are extremely complex and show the characteristics of black boxes.The black box model that has not been clearly interpreted has become a major obstacle to the wide use of machine learning models.Complex prediction model refers to the model with complex nonlinear relationship between input features and prediction.Generally speaking,most machine learning models have this characteristic,such as neural network,random forest and so on,which internal mechanism is difficult to understand.It is difficult to make decisions by analyzing the input features and prediction when people cannot explain results of the model clearly.In addition,low transparency of the model is difficult to win the trust of the users.The theoretical and method research on the interpretability of complex prediction models will help improving users’ trust and confidence in machine learning models,and promote the wider use of machine learning models.Based on this,the content of this paper mainly includes two parts:(1)The main influencing factors of user adoption of machine learning model and how to promote the adoption of machine learning models from these factors.In this study,we use literature research,questionnaire investigation and empirical study.We summarize the related literature of the technology acceptance model,and combine the characteristics of the machine learning model,integrating with other external variables,to explore the key factors that encourage enterprises to adopt machine learning models.We try to identify the factors that have significant influence and make recommendations to promote the adoption of machine learning models so that machine learning can provide a better service to the enterprise.Among the factors influencing the adoption of machine learning models,the poor interpretability of models is always the key factor influencing the adoption of users.Therefore,the research on the influence of the interpretability of machine learning black box models on user adoption is the focus of this paper.(2)It is necessary to study the interpretability of complex prediction models from the perspective of user adoption.For the interpretability of complex prediction models,we mainly study how to identify the different impact of model input features on prediction,and how to make the model results support decision making effectively.In this study,we use literature research and experimental methods.We summarize the related literature on the interpretability of black box models,and focuses on the impact of input features on prediction in supervised learning models.For input features,A method to explain the black box model by visualizing the influence of input features on the prediction results is proposed from two different perspectives of independent feature and correlated feature respectively.Moreover,we apply this method to artificial data sets and real data sets,combine with real scenes,to analyze the practical significance of feature impact for the interpretability of black box models and decision support.The existing researches of adoption to machine learning models mainly focused of machine learning implementation platform,execution frameworks or algorithms themselves or attempts to assess the acceptance of machine learning models from organizational factors.However,they fail to analyze the main factors that influence the adoption of machine learning models from the perspective of users of machine learning models and there is a lack of empirical research.The interpretability of complex prediction models is an important factor affecting user adoption.Most current methods of interpretability research can be divided into three categories:Measuring the contributions or importance of input features to prediction,detecting feature interaction and discovering the impact of input features.This study focuses on measuring the feature impact patterns.According to the current researches,some methods can only observe part of the impact and ignore some important patterns.There are also some methods that cannot simply and intuitively analyze the interaction impact between features.These limitations are the direction to be improved in this study.In addition,the current research is only focus on the study of the impact of independent features,and we also study the impact of correlated features in this research.Compared with previous research results,the main research conclusions and implications of this paper are as follows:(1)In terms of influencing factors of users’ adoption,the interpretability of machine learning model has a very important influence on the trust factor,and trust plays a key role in influencing perceived usefulness and attitude.Perceived usefulness and perceived ease of use have positive effects on users’ attitudes towards using machine learning models,but perceived ease of use has no significant effect on perceived usefulness.This shows that even though machine learning models are difficult to use,people still think the models are useful.Organizational factors(demands from managers and pressure from competitors)also have a strong positive influence on attitudes and behavioral intentions using machine learning models.The research on theories and methods based on the interpretability of machine learning models will help improve users’trust and confidence in machine learning models,and promote the wider use of machine learning models.(2)In terms of identifying the impact pattern of features,we propose a new method to analyze the impact of independent features on the prediction.The proposed method distinguishes some typical impact that correspond to different groups of observations.The method maps the detected impact into feature space using tree rules that help locate the impact in the feature space.More importantly,the feature relationships embedded in the prediction models can be revealed through this tree rulebased feature relationship network.We apply the proposed method to various simulated and real data,and the results demonstrate how it can help us understand how features affect model prediction results and the relationships among features.(3)In addition to the study of the impact of independent features,this paper also proposes a method to analyze the impact of correlated features on the prediction.Comparing with the impact of independent features,the research of feature impact has important significance for supporting decision making.Prediction models can be compared and diagnosed based on the impact curve plots and feature relationship network.The relationships among the features embedded in the prediction models can be identified from the feature relationship network.Some meaningful impact embedded in the models can be distinguished that can guide the design of strategies to change the predicted results of the model.(4)The application of feature impact identification in reality is mainly embodied in four aspects:comparing prediction models based on impact curves,evaluating the quality of prediction models,identifying the most important feature through feature relation network,and improving the design of strategies to upgrade the predicted results of the model.The main contributions and innovations of this paper are as follows:First,a new feature impact identification method based on the assumption under the condition of independent features is proposed.This method can show the typical impact of input features on the prediction results succinctly,intuitively and completely,and can reveal the interaction between features through tree rules,which improves the identification effect of feature impact.Second,a new feature impact identification method based on the assumption under the condition of correlated features is proposed.This method solves the deficiency of the existing research which is mainly based on the assumption of independent features and improves the applicability of feature impact identification method to various application scenarios.Thirdly,based on the extended technology acceptance model,a theory model of machine learning technology adoption is proposed,which verifies the necessity of model interpretation research.