Merging trust-region and limited memory technologies for large-scale optimization

This research is motivated by large-scale applications having difficult to evaluate objectives (as measured by cpu time). A trust-region strategy is employed to exploit accumulated local information at each step with the goal of reducing the overall number of function and gradient evaluations. It is shown that limited memory updates can be implemented in a trust-region framework with only a modest increase in the linear algebra costs.;In our research, we focus on two types of problems: the unconstrained optimization problem and the optimization problem with simple bounds. To solve the large-scale unconstrained problem, we propose a trust-region algorithm which initiates the trust-region radius as infinity at each iterate, and shrinks the radius if the quasi-Newton solution does not sufficiently reduce the function value. Numerical results show that with the incorporation of the proposed algorithm with limited memory BFGS update, we achieved goal of reducing the number of function and gradient evaluation on a selected subset of the MINPACK-2 test problems.;We propose the algorithm active-set trust-region algorithm (ASTRAL) to solve large-scale optimization problem with simple bounds. The algorithm ASTRAL is an ℓinfinity-norm trust-region method that uses both active set identification techniques as well as limited memory BEGS updating for the Hessian approximation. The trust-region subproblems are solved using primal-dual interior point techniques that exploit the structure of the limited memory Hessian approximation. A restart strategy ensures that the algorithm identifies the optimal constraints in finite number iterations under a standard nondegeneracy hypothesis. The numerical results based on the subset of CUTEr problems shows that the algorithm ASTRAL outperforms the classic L-BFGS-B in the number of function and gradient evaluations.;In addition, we proposed a xi penalty algorithm to solve the optimization problem with both bounded and linear constraints and the preliminary results are shown at the end of the dissertation.