Synchronous And Asynchronous Parallel Optimization

In this paper, we analyses the advantages and disadvantages of synchronization and asynchronization in parallel optimization based on the parallel gradient distribution (PGD) algorithm proposed by Mangasarian in 1995. And we discussed the parallel variable distribution (PVD) algorithm for constrained nonlinear optimization, which was proposed by Ferris in 1998.Firstly, we introduced two kinds of parallel optimization and the current situation about them.In the second chapter, we analyses the advantages and disadvantages of synchronization and asynchronization in parallel optimization. We seek to remove synchronization points from a parallel implementation in PGD. Computational results showed that asynchronous implementation outperforms its synchronous counterpart.In the third chapter, we point out that synchronization and communication delays are the major sources of performance degradation of parallel algorithms, while asynchronous implementations have the potential to reduce the over-head to minimum and to accelerate the convergence. A modification of the PVT algorithm, due to Fukushima (1998), is presented here. We propose a totally asynchronous PVT algorithm at last.In the last chapter, we analyses the PVD algorithm, proposed by Ferris in 1998, remove its synchronization points like the front chapters and employ a feasible direction instead of the residual projected gradient in the algorithm.