Prediction Methods Based On DEA And RBF/SVM For Big Data

In the era of big data,how to use big data for effective analysis has become the focus of attention from all walks of life.Due to the diversified characteristics of big data,such as the source,quantity,structure,and real-time nature,the value it covers is very high,but its value density is very low.The existence of data noise and data redundancy in large data sets will have an inestimable negative impact on data analysis.In addition,the large data set also covers the functional relationships between multiple variables,which may cause certain deviations in the data analysis results.Therefore,before using big data for analysis and research,it is necessary to perform data preprocessing on the big data to eliminate redundant and invalid data.However,the traditional big data preprocessing method does not consider the functional relationship between variables.Data envelopment analysis can effectively deal with the deviation problem caused by the functional relationship between variables.In the process of data preprocessing using DEA,there is no need to predict the functional relationship between input and output variables,and it is not necessary to set weights in advance.The most effective data is obtained by filtering the obtained efficiency values,and outliers and redundant values are eliminated.Reducing the amount of data without changing the quality of the data is an effective way to apply data preprocessing to machine learning.In addition,commonly used big data modeling tools cannot effectively model big data with complex non-linear relationships.At present,the better methods for big data modeling include radial basis functions and support vector machine functions.RBF can approximate arbitrary non-linear variable relationships with arbitrary precision,better handle complex rules between variables,and provide a new idea and method for the development of prediction models.The prediction accuracy is good,and it has achieved satisfactory results.SVM can effectively overcome the adverse effects of sample distribution,redundant features,and overfitting,and has great advantages in small samples and non-linear prediction,which better solves practical problems such as high dimensions and local minimum points,Has a strong generalization ability.Based on the effectiveness of DEA data preprocessing and the advantages of higher prediction accuracy of RBF and SVM,this paper proposes two prediction methods(DEA-RBF and DEA-SVM)that combine DEA with RBF and SVM.DEA was used to preprocess the data,and the most effective data set was screened to reduce the training time of RBF and SVM.In addition,under the premise of maintaining the universality of big data,outliers are eliminated to prevent negatively affected data from being applied to RBF and SVM,which in turn makes the model's prediction accuracy higher.In this paper,two improved models,DEA-RBF and DEA-SVM,are compared with pure RBF and SVM models.From the two aspects of time cost and prediction accuracy,compared with the simple RBF and SVM models,both the DEA-RBF and DEA-SVM models have improved prediction accuracy and reduced prediction time with reduced training time.The validity of the model is verified.Both the RBF model and the SVM model have their advantages.At present,there is no mature theory that can guide which model to choose under what circumstances.It is more dependent on the experience of managers or engineers and the characteristics of the data set.This paper proposes two improved modeling methods designed to provide managers or engineers with a wider choice.Finally,this paper applies the DEA-SVM model to wine quality assessment,which provides decision support for the development of the wine industry and the management of enterprises.