OpenMP on Networks of Workstations

Abstract

The OpenMP Application Programming Interface (API) is an emerging standard for parallel programming on shared memory architectures. Networks of workstations (NOWs) are attractive parallel programming platforms because of their good price/performance ratio, as well as their and their potential to scale. This work is the first to extend the support for OpenMP to networks of workstations. The design is based on integrating the compiler and the run-time system. In the combined system, the run-time library remains the basic vehicle for implementing shared memory, while the compiler performs optimization rather than implementation. The integrated approach can effectively optimize irregular applications, for which an exact compile-time analysis is not possible. One optimization aggregates messages for irregular applications based on the data access information provided by the compiler. Another optimization improves the scalability of the system by computation replication and multicast. The integrated system also simplifies the run-time implementation. In the implementation of OpenMP on networks of shared-memory multiprocessors, the compile-time information greatly reduces the number of changes required to the run-time system in order to exploit the intra-node hardware shared memory. In another implementation that allows OpenMP programs to change the number of computing nodes during the execution, the OpenMP semantics provides convenient points for efficient adaptation.