An Empirical Method for Processing I/O Traces to Analyze the Performance of DL Applications
摘要
The exponential growth of data handled by Deep Learning (DL) applications has led to an unprecedented demand for computational resources, necessitating their execution on High Performance Computing (HPC) systems. However, understanding and optimizing Input/Output (I/O) of the DL applications can be challenging due to the complexity and scale of DL workloads and the heterogeneous nature of I/O operations. This paper addresses this issue by proposing an I/O traces processing method that simplifies the generation of reports on global I/O patterns and performance to aid in I/O performance analysis. Our approach focuses on understanding the temporal and spatial distributions of I/O operations and related with the behavior at I/O system level. The proposed method enables us to synthesize and extract key information from the reports generated by tools such as Darshan tool and the seff command. These reports offer a detailed view of I/O performance, providing a set of metrics that deepen our understanding of the I/O behavior of DL applications.