Modern multi-node systems necessitate parallel programming models (PPMs) like mpi to facilitate execution and communication among multiple processing elements. These single program, multiple data (SPMD) PPMs offer features such as RMA or accelerator support. However, SPMD program tools, such as those for correctness checks or performance optimization, are typically developed for specific PPMs or rely on tool-internal abstractions. To overcome this limitation, the spmd ir was introduced as an intermediate representation (IR) within a multi-layer program representation and realized as a dialect in mlir (llvm). This work extends the spmd ir by incorporating, among others, capabilities for RMA and related completion mechanisms. These enhancements increase compatibility with mpi and shmem, while newly integrating support for nvshmem. By leveraging traits in mlir, spmd ir provides an extensible approach for implementing SPMD program analysis. The applicability of the spmd ir is demonstrated through the use case of static local data race detection. It is implemented in a generalized fashion, covering not only RMA but also non-blocking communication in general, and is independent of specific API calls, offering increased extensibility. Using a comprehensive set of micro-benchmark suites and proxy apps, the spmd ir is evaluated against both static and dynamic tools. Overall, the spmd ir verification distinguishes itself with extensive PPM support and high detection accuracy. Notably, it is the first tool capable of detecting data races across shmem, nvshmem, and their hybrid combinations (with mpi).

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Extending the SPMD IR for RMA Models and Static Data Race Detection

  • Semih Burak,
  • Simon Schwitanski,
  • Felix Tomski,
  • Jens Domke,
  • Matthias Müller

摘要

Modern multi-node systems necessitate parallel programming models (PPMs) like mpi to facilitate execution and communication among multiple processing elements. These single program, multiple data (SPMD) PPMs offer features such as RMA or accelerator support. However, SPMD program tools, such as those for correctness checks or performance optimization, are typically developed for specific PPMs or rely on tool-internal abstractions. To overcome this limitation, the spmd ir was introduced as an intermediate representation (IR) within a multi-layer program representation and realized as a dialect in mlir (llvm). This work extends the spmd ir by incorporating, among others, capabilities for RMA and related completion mechanisms. These enhancements increase compatibility with mpi and shmem, while newly integrating support for nvshmem. By leveraging traits in mlir, spmd ir provides an extensible approach for implementing SPMD program analysis. The applicability of the spmd ir is demonstrated through the use case of static local data race detection. It is implemented in a generalized fashion, covering not only RMA but also non-blocking communication in general, and is independent of specific API calls, offering increased extensibility. Using a comprehensive set of micro-benchmark suites and proxy apps, the spmd ir is evaluated against both static and dynamic tools. Overall, the spmd ir verification distinguishes itself with extensive PPM support and high detection accuracy. Notably, it is the first tool capable of detecting data races across shmem, nvshmem, and their hybrid combinations (with mpi).