Research On Iterative Algorithms For Optimization Problems With Certain Separable Structures

In the era of big data,with the advance in science and technology and the development of information technology,the study of optimization problems and the design of algorithms for solving these problems have become hot research issues in science and engineering.In many fields,such as machine learning,image denoising,signal processing and traffic,etc.,solving nonsmooth(nonconvex)optimization problems with certain separable structures are involved.In this paper,according to the(partially)separable structure of these problems and the proper properties of each component functions,we design executable iterative algorithms and analyze the convergence properties of these algorithms theoretically.Based on the classical alternating direction method of multipliers,the proximal gradient method and the Douglas-Rachford splitting algorithm,combining with the Bregman regularization strategy,the method of majorization,the inertial acceleration technique and the prediction-correction idea,we propose several efficient and convergent iterative algorithms for solving(partially)separable multi-block optimization problems with linear constraints and unconstrained nonconvex and nonsmooth optimization problems.The main contributions are as follows:1.Aiming at convex optimization problems with linear constraints and three separable blocks in the objective function,we propose a linearized alternating direction method of multipliers.In the solving process of the algorithm,it is ensured that the solution of the subproblems only need to solve the proximal mapping of the component functions,so as to obtain the explicit solutions.In the theoretical analysis,first,the global convergence and sublinear convergence rate of the algorithm are proved under the condition that any two coefficient matrices are orthogonal,then,it is proved that the algorithm does not converge in general by giving a concrete example.2.Considering the convex optimization problems with linear constraints,in which the objective function consists of two parts,one is three block separable function,the other is smooth function without separable structure,based on the properties of the smooth function,we propose a majorized proximal alternating direction method of multipliers.Relying on the generalized Mean-Value theorem,we prove the convergence property of the algorithm under the condition that the coefficient matrix and the smooth term satisfy a certain relation.3.For a class of monotone variational inequality problems with separable structure,we propose a prediction-correction type method,that is,inexact parallel alternating direction method of multipliers.The subproblems of the algorithm can be solved and the convergence of the algorithm is obtained.And,it is applied to traffic problems to verify the effectiveness of the algorithm.4.For the optimization problems where the objective function is the sum of a smooth nonconvex function and a nonsmooth nonconvex function,we propose an inertial Douglas-Rachford splitting algorithm by combining the DouglasRachford splitting algorithm with the inertial acceleration technique,which takes into account the separable structure of the objective function.By using KurdykaLojasiewicz property,the convergence of the algorithm is proved.The validity of the algorithm is verified by numerical experiments.5.For the nonconvex optimization problem where the objective function is the sum of a smooth function and two separable functions,regularizing the smooth function by Bregman distance,we propose a new Bregman inertial alternating linearized minimization algorithm,which can explore the structure of the separable parts in the objective function.Based on the Kurdyka-Lojasiewicz property,the convergence of the algorithm is proved.The effectiveness of the algorithm is verified by applying on sparse signal recovery problem.