Regularized Logistic Regression And Its Application

Highly correlated categorical data are common in economic society,survey research,and medicine.However,just doing plain logistic regression may not get the desired result.In order to solve the above problems,a coefficient regularization penalty is often added to the loss function,and the regularization problem is essentially an optimization problem.Therefore,this paper studies the regularization of binary and multi-class logistic regression models based on ADMM algorithm,mainly studying ridge regression and adaptive elastic net regularization penalty.This paper studies the ADMM algorithm of ridge regression.In the numerical simulation,the influence of the sample size and the correlation degree of the explanatory variables on the model was considered.The simulation results show that the ADMM algorithm used in this paper will have better classification effect in the process of enhancing the correlation between explanatory variables,and will be significantly better than other iterative optimization algorithms.However,ridge regression cannot produce sparse solutions and cannot implement variable selection.When there are high-dimensional variables,only some important variables may play a role in the model,and too many variables will increase the redundancy of model fitting.Adaptive elastic net is an algorithm with Oracle properties,which can realize variable selection and solve the influence of variable correlation at the same time.From the simulation results,the classification accuracy of this algorithm is significantly higher than that of elastic net,lasso and other sparse models.Compared with the ridge regression results,it is found that both perform well when the variables are highly correlated.Adaptive elastic net classification performs better than ridge regression when the variable coefficients are sparse.This paper applies the regularized logistic regression model to the financial early warning of listed companies,the selection of rural labor employment mode,and the study of Beijing’s air quality level.The example results show that ridge regression based on ADMM algorithm is suitable for models with high correlation between variables and weak coefficient sparsity of variables;adaptive elastic net performs well in solving multicollinearity between variables,and it can perform variable selection,and it can be used in large-scale and large-scale models.It performs better in data with sparse coefficients of variables.