Studies On Active Set Methods For Bound Constrained Optimization

Bound constrained nonlinear optimization problem is a special kind of nonlinear optimization problems only with upper and lower bounds on the variables. It appears in a wide range of applications such as engineering and scientific computing. In this paper, the active set methods for bound constrained optimization problems and their applications for support vector machine classification are studied. The paper is divided into four parts and the details are as follows:First, in chapter 1, the general model and related concepts of optimization problems are introduced briefly. The practical applications and research status of bound constrained nonlinear optimization problems are presented which lead to the main results of the paper.In chapter 2, the basic idea of active set methods is introduced. And the active set methods for bound constrained optimization problems proposed in recent years are summarized, including the projected conjugate gradient methods and the active set identification function methods. The advantages and disadvantages of these methods are analyzed.In chapter 3, a new method for bound constrained problems with linear equality constraints is proposed. It transforms the original problem into an equivalent linear inequality constrained problem utilizing the null-space method, and then, uses the partial spectral projected gradient method proposed by Marina Andretta to solve the equivalent problem. The solution of the linear inequality constrained problem is also the original problem's optimal solution. This method removes the equality constraints of the original problem, which simplifies the construction and reduces the dimension of the problem. Under appropriate assumptions, the global convergence results are established, and the algorithm terminates in a finite number of iterations. Moreover, the method is successfully applied to standard support vector machine and the corresponding algorithm is given.Finally, in the last chapter, the summary of this paper is presented and the future research direction is pointed out.