During the training of large models, traditional checkpointing introduces significant overhead, as it requires pausing training while model states are copied from GPU memory to storage. At the same time, frequent checkpoint is needed to ensure an efficient use of resources. Efficient checkpointing is crucial for large-scale training of Artificial Intelligence (AI) models, especially on high-performance computing (HPC) systems. In this work, we evaluate distributed asynchronous checkpointing (DACP) applied on various LLM models on the Leonardo supercomputer, hosted by CINECA. By integrating asynchronous checkpointing, we enable overlapping data transfer operations with training iterations, significantly reducing training time. Our experiments span multiple LLM configurations, leveraging PyTorch DACP to optimize checkpointing frequency and minimize graphics processing unit (GPU) idle time. We evaluate this approach at scales of up to 256 GPUs using different model sizes. Results demonstrate a substantial reduction in checkpoint overhead, achieving up to a 6x improvement compared to synchronous methods. Our evaluations highlight the benefits of asynchronous checkpointing for large-scale training and provide insights into its practical deployment on HPC infrastructures.