Sampling selection and space-narrowing methods for stochastic optimization

In this thesis we develop a new methodology for stochastic optimization. Our methodology includes the Sampling-Selection and Space-Narrowing methods. Our methods are especially effective to handle those problems which have large and complicated search spaces, little analytical information and large observation noises. We also demonstrate the applicability and effectiveness of our methods by applying them to two traditional optimal control problems.; The traditional stochastic optimization methodology requires the accurate estimations of performances. Many stochastic optimization problems do not have analytical formulas of the performance functions. This leaves simulation the only tool for performance estimation. However, simulation may impose considerable computational cost. When the search space is large and complicated, the traditional stochastic optimization methodology may find the optimization problem practically unsolvable.; In this thesis, we provide a fresh look at stochastic optimization problems. Instead of only focusing on the optimal solution, we broaden our view into the global statistical picture of the performances. We are not only interested in the optimal solution, but also other "good" solutions. The Sampling-Selection method first randomly generates a number of sample designs and then selects the solution or solutions based on their observed performances. The quality of the selected set is characterized by alignment probability which is the probability that there are at least some good designs in the selected set.; We also develop the Space-Narrowing method. The key to this method lies in the statistical comparison of two specifications of the search space to determine which one contains more good designs. Space-Narrowing method is very effective to deal with those problems with large and complicated search spaces. Finally, we apply the Space-Narrowing method to two classic optimal control problems.