Augmented Lagrangian Function Methods For Solving Constrained Optimization Problems

Constrained optimization problems abound in many important fields such as en-gineering, national defence, economy, finance, social sciences and so on. One of themain approaches for solving constrained optimization problems is to transform theminto unconstrained optimization problems. Penalty function methods are a class ofprevailing techniques to implement the transformation. Penalty function methodsseek to obtain the solutions of a constrained optimization problem by solving asingle penalty problem or a sequence of penalty problems. The objective functionof penalty problems is called a penalty function. In general, a penalty parameterappears in a penalty function. We say that a penalty function is an exact penaltyfunction if, for some values of the penalty parameter, it is possible to establish somecorrespondence between the minimizers of the corresponding penalty problems andthe solutions of the original constrained problem. Otherwise, the penalty functionis called a sequential penalty function. The sequential penalty function methodsrequire, in principle, an infinite sequence of penalty problems to be solved. Themain drawback of the sequential penalty function methods is that when the penaltyparameter approaches its limiting value, the corresponding penalty problems areill-conditioned which will result in some shortcomings such as numerical instabil-ities and slow convergence. On the contrary, the exact penalty function methodsrequire only a single penalty problem to be solved which avoids the ill-conditioningassociated with the sequential penalty function methods. Exact penalty functionscan be divided into two classes: nondi?erentiable exact penalty functions and con-tinuously di?erentiable exact penalty functions. In practical computation aspects,the"Maratos'e?ect"emerges in nondi?erentiable exact penalty function methodsdue to the nondi?erentiability of penalty functions which gives rise to locally slowconvergence. Two classes of continuously di?erentiable exact penalty functions havebeen proposed so far: the functions which are defined on the same space of the vari-ables of the original constrained problem and the functions which are defined on theproduct space of the problem variables and of the KKT multipliers. The first classpenalize the violation of the KKT conditions by making use of multiplier functions,namely, of functions which yield estimates of the KKT multipliers as functions ofthe original constrained problem variables. In general a multiplier function is quiteexpensive from the computational point of view when the number of constraints islarge, which may limit somewhat the applicability of continuously di?erentiable ex-act penalty functions of this kind. The second class can be in turn divided into twosubclasses: the penalty functions in which the terms that account for the KKT con-ditions are added to the objective function of the original constrained problem andthe augmented Lagrangian functions in which the terms that account for the KKTconditions are added to the usual Lagrangian function. Both theoretical resultsand computational experience have indicated that augmented Lagrangian functionmethods o?er significant advantages over the usual penalty methods, i.e., sequentialpenalty function methods and nondi?erentiable exact penalty function methods, bycircumventing some of the traditional shortcomings of these techniques such as nu-...