Combining Malleability and Distributed Control Mechanisms to Reduce I/O Contention
摘要
Some data-intensive applications perform periodic read and write operations, which can lead to slowdowns and contention when multiple applications are running simultaneously on a shared filesystem. In high-performance computing (HPC), malleability refers to the ability to dynamically adjust job resources at runtime to optimize performance, reduce execution times, and improve efficiency. This paper presents a decentralized probabilistic framework designed to reduce I/O contention in data-intensive malleable applications. A performance model is used to obtain the application maximum I/O performance, and an algorithm is used to detect when there is contention when running multiple applications. In such cases, malleability is applied to a subset of running applications to mitigate performance issues. We tested the model on a parallel, data-intensive, agent-based epidemiological simulator that performs periodic write operations. The results show that our model efficiently reduces contention, particularly during the most data-intensive critical stages of the execution, improving I/O throughput up to a 15%.