Research On Margin Distribution Regularization Machine Learning Theory And Its Application

Regularization can improve the generalization performance of classifier by joining in prior information and avoiding over-fitting.Support Vector Machine(SVM)tries to max-imize the minimum margin to regularize the model,obtains strong generalization perfor-mance,and has great value in theory and application.Recent theory results revealed that margin distribution is really critical to the generalization performance rather than simply considering the minimum margin or a single margin.Based on these theory results,Large margin Distribution Machine(LDM)is proposed to maximizes the margin distribution and gets strong generalization performance.Both theoretical results and many experimental results verify that LDM is superior to SVM and several state-of-the-art methods.There-fore,margin distribution learning becomes a new research hotspot.This thesis focuses on relevant theory and application research and designs classification algorithms with strong generalization performance and robustness.Furthermore,it explores these algorithms for pedestrian detection and scene classification.The tasks include the following aspects:(1)In view of the unbalanced classification on unbalanced size data,this thesis pro-poses Large Cost-Sensitive margin Distribution Machine(LCSDM)which includes cost-sensitive margin distribution and error penalty.By analyzing the margin mean and margin variance of LDM,cost-sensitive margin mean and cost-sensitive margin variance are de-fined and the cost-sensitive margin distribution regularization model is established.The property of cost-sensitive parameter in the cost-sensitive model is deeply studied and the constraints of cost-sensitive parameter are derived through theoretical analysis.Based on these constrained conditions,LCSDM with some cost-sensitive parameters conformed to constrained conditions is proposed.It can adjust the margin distribution on unbalanced ex-amples through the adjustment of 55cost-sensitive parameter,implement the adjustment of the separator,and increase the minority class detection rate.Because LCSDM has cost-sensitive margin distribution which is critical to generalization performance,it can adjust the detection rate and get strong generalization performance.(2)To improve the robustness about margin size and noise,this thesis proposes a novel machine called Double Distribution margin distribution Machine(DDM)based on margin distribution regularization.In effective feature classification,the example statistical prop-erty generally has important marked effect.Base on the study on the relation between the example statistical property and margin distribution,the margin distribution is described by the average and mean square error of the margin of two example means to realize balanced margin size and make the model more intuitive.Therefore,DDM is designed to obtain strong generalization performance by maximizing the margin distribution of two example means,and it can get better robust performance by the noise suppression of example mean.(3)As for the imbalanced margin cost between two classes,this thesis proposes Cost-Sensitive Double margin Distribution Machine(CS-DDM),which has more intuitive geo-metric meaning.Based on the analysis of the structure and geometric meaning of DDM,cost-sensitive double margin distribution is defined and cost-sensitive double margin dis-tribution optimization model is designed.This model can adjust the margin distribution according to cost-sensitive parameter and then readjust the separator and detection rate of different cost examples.By deeply researching the mathematical sense of the cost-sensitive parameter in the model,it reveals that the nature of cost-sensitive parameter adjusting sep-arator and the dual variable,and the constraint conditions of the cost-sensitive parameter are obtained.Based on these conclusions,CS-DDM with eligible parameters is designed to address unbalanced cost binary classification.(4)In order to quickly and cost-sensitively detect pedestrian,this thesis designs pedes-trian detection model based on BING and DDM.DDM is trained by HOG features of pedes-trian and non-pedestrian images.Based on objectness proposals from BING frame,heuristic region location algorithm is proposed.The region including pedestrian can be obtained by objectness select and region location algorithm.Region location effectively reduces the search space of DDM,therefore DDM only scan the location region to detect pedestrian to obtain detection result and position.Region location improves the efficiency and speed of pedestrian detection.Furthermore,we can use CS-DDM to detect pedestrian according to the need of detection rate and obtain cost-sensitive detection rate.(5)Binary Double margin Distribution Machine is promoted to recognize a variety of scene category.This thesis introduces neighborhood coding and promotes DDM to ad-dress the problem of multiclass scene classification.The feature represent of neighborhood coding can enhance the expressiveness compared with original image,and the subsequent linear classifier can obtain high prediction accuracy.Base on the neighborhood coding fea-ture,we promote DDM to recognize multiclass scene categories by One Vs Rest(OVR)scheme.The experimental results suggest that the approach of neighborhood coding feature and multiclass DDM can markedly improve the speed and stability of scene recognition.