Research On Inexact Gradient Mirror Descent Algorithm For Non-smooth Convex Optimization Problem And Their Application

The nonsmooth convex optimization problem is to minimize the sum of the smooth convex function and the nonsmooth convex function.It is a mathematical model commonly used in machine learning and artificial intelligence.Its main applications include sparse matrix decomposition,Lasso problem,sparse regression,linear support vector machine,feature selection,image restoration,face recognition,etc.Because the first-order methods for solving nonsmooth convex optimization problems usually have better iterative complexity and improved convergence speed,they have received widespread attention.This paper proposes three first-order methods to solve the nonsmooth convex optimization problem.This article mainly studies the following:First,we propose an inexact gradient mirror descent algorithm for nonsmooth convex optimization problem.It is a generalization of gradient mirror descent algorithm for smooth convex optimization problem which is proposed by Allen-Zhu in 2016.At each iteration,it allows the errors of gradient calculations of smooth terms and proximity operator calculations of non-smooth terms in the objective function.Moreover,the convergence rate ofO(1/k~2)of the function value sequence of the algorithm is analyzed under appropriate conditions.The number of iterations is denoted by k.Finally,the algorithm is used to solve the Lasso and Logistic problems,numerical results illustrate the efficiency of the proposed algorithm.Secondly,based on the monotonic idea,a monotonic inexact gradient mirror descent algorithm for solving the nonsmooth convex optimization problem is proposed,and the application of this algorithm to the problem of L1 regularization and L2 loss support vector machine is discussed.Numerical results illustrate that the algorithm is effective.Finally,based on the ideas of multilevel optimization,a multilevel random coordinate descent algorithm for solving nonsmooth convex optimization problems is proposed,and the application of the algorithm to the L1 regularization and L2 loss support vector machine is discussed.The numerical results illustrate that the algorithm is effective.