Large-scale network parameter configuration using on-line simulation framework

Today's Internet relies on a variety of important network protocols. The current parameter configuration process of these protocols is mainly manual and widely considered a black art. This thesis tackles this parameter setting problem by formulating it as a “black-box” optimization problem. In this approach, we use an on-line simulation system to monitor and simulate the network, and then use a black-box optimization algorithm to optimize the parameters of the concerned network protocol. This black-box approach allows flexibility in terms of objectives and metrics of the desired optimization and can be applied to a wide range of network protocols.; We first investigate the properties of the underlying optimization problems. The desired optimization algorithm is expected to be highly efficient, scalable to high dimensions and robust to noisy objective functions. Based on these requirements, we propose a Recursive Random Search (RRS) algorithm whose major feature is its basis on random sampling. We empirically validate the advantages of RRS with extensive tests on a suite of benchmark functions and application in some real network optimization problems. To provide a more general solution for practical optimization problems, we also propose a Unified Search Framework (USF), which includes a variety of search techniques as building blocks and can be used as the platform to build tailored optimization strategies by combining a selection of building blocks. Furthermore, USF includes the mechanisms to parallelize search techniques and allocate available computing resources among them such that the resources are optimally utilized.; Finally, we investigate the configuration problems of several network protocols, such as, Random Early Drop (RED), Open Shortest Path First (OSPF) and Border Gateway Protocol (BGP). We formulate these problems into black-box optimization problems, some of which have thousands of parameters. We then apply the optimization techniques developed in this thesis to them. Simulations and experiments have demonstrated the effectiveness and efficiency of the on-line simulation system and the proposed optimization techniques.