Overlapping of communication and computation and early binding: Fundamental mechanisms for improving parallel performance on clusters of workstations

This study considers software techniques for improving performance on clusters of workstations and approaches for designing message-passing middleware that facilitate scalable, parallel processing. Early binding and overlapping of communication and computation are identified as fundamental approaches for improving parallel performance on clusters. Cluster computers using the Message-Passing Interface for interprocess communication are the predominant choice for building high-performance computing facilities, which makes the findings of this work relevant to a wide audience from the area parallel processing.; The performance-enhancing techniques studied in this work are presently underutilized in practice because of the lack of adequate support by existing message-passing libraries and are also rarely considered by parallel algorithm designers. Furthermore, commonly accepted methods for performance analysis and evaluation of parallel systems omit these techniques and focus primarily on more obvious communication characteristics such as latency and bandwidth.; This study provides a theoretical framework for describing early binding and overlapping of communication and computation in models for parallel programming. This framework defines four new metrics that facilitate new approaches for performance analysis of parallel systems. This dissertation provides experimental data that validate the accuracy of the new performance analysis. The theoretical results of this analysis can be used by parallel software designers for estimating the effective application performance benefits from early binding and overlapping.; This work presents a new MPI implementation, MIP/Pro, that is specifically optimized for clusters of workstations. This implementation emphasizes features such as persistent communication, asynchronous processing, low processor overhead, and independent message progress. These features are identified as critical for delivering maximum performance to applications. The experimental section of this dissertation demonstrates the capability of MPI/Pro to facilitate software techniques that result in significant application performance improvement. Specific demonstrations with the Virtual Interface Architecture and TCP/IP over Ethernet are offered.