Boosting Algorithm And Its Application

During the last ten years, there appears an effective method, which can boost the precision of the algorithm by many rounds learning and obtain much more accurate prediction rule by majority. It is always called Boosting. This method can effectively transform the weak learning algorithm into strong learning algorithm. As a new ensemble learning method, Boosting shows its good property in many fields.This paper introduces the main idea of Boosting and some classic Boosting algorithms (i.e.AdaBoost.M1, Real AdaBoost), then explains the underlying theory of Boosting, which includes the analysis of training error, generalization error, optimization theory, bias & variance, game theory. Moreover, we are interested in the consistency of Boosting, and obtain the stopping strategy to be used in AdaBoost to achieve universal consistency. Then we give an effectiveness explanation of the Boosting theoretically.We apply the Boosting algorithm to bank individual credit evaluation. With the development of market economy, individual credit evaluation becomes more and more important in commercial banks. Even if the accuracy of individual credit evaluation improves a little, that means a huge profit for the bank. We choose the German Credit Database and the Australian Credit Database from UCI , and apply both AdaBoost.M1 and Bagging to the two databases. The result shows AdaBoost.M1 indeed boost the accuracy of the weak sort algorithm by 5.8% and 1.02%. It gives an effectiveness explanation of the Boosting practically.