Predicting MPI Communication for Solving Partial Differential Equations Using Machine Learning
摘要
High-performance computing (HPC) is crucial for obtaining solutions to complex scientific and engineering problems described by partial differential equations (PDEs). However, most of these simulations are very sensitive to one major cause of performance bottlenecks: the communication overhead between nodes. While the Message Passing Interface (MPI) is a standard library for communication on HPC systems, the best-suiting MPI communication function is very hard to decide due to numerous variations of algorithms and their dependency on multiple factors. In this paper, we present a machine learning-enabled approach to predict optimal MPI communication functions for large-scale PDE simulations. Our model encompasses many diversified features, like mesh size, number of variables, compiler type, number of cores, and variable types, in its construction to take a more holistic approach toward the optimization of communications. In fact, the model predicts the most suitable MPI function for a given simulation environment by taking into account the aforementioned parameters. Experiment results show clearly that incorporating these features into the prediction model yields the best communication strategy which is highly accurate, reliable, ensuring better scalability and performance. To the best of our knowledge, this is the first thorough study addressing the use of machine learning techniques to optimize MPI communication for PDE simulations; therefore, it is an important contribution for leveraging the predictive capability of machine learning models for the co-design of large-scale numerical simulations.