Data processing units (DPUs) represent a significant advancement in networking technology. With their extensive capabilities, DPUs are well-positioned to accelerate AI and HPC workloads by offloading tasks that would otherwise rely on the host for additional resources. However, fully realizing the potential of DPUs is currently hindered by the lack of robust frameworks that facilitate the development of services on these devices. While DPU SDKs like NVIDIA’s DOCA are rapidly evolving, they still fall short of providing seamless offloading to DPUs. To address this gap, we introduce DOCA Unified Resource and Offload Manager (UROM): a framework designed to enable seamless offloading of parallel computing tasks from the host to NVIDIA DPUs. This paper details the architecture of DOCA UROM, highlighting its key components and their functionalities, including resource discovery, host-DPU coordination, and task management on the DPU. We describe how developers can leverage DOCA UROM’s flexible architecture to develop new DPU-based services. To demonstrate its versatility, we showcase three diverse DPU-based services developed using DOCA UROM: RDMOs, progress for non-blocking collectives, and utilizing DPU memory to enhance AI training. Our results illustrate the significant benefits these services offer, underscoring the potential of DOCA UROM in accelerating workloads.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

DOCA UROM: A Vehicle for Offloading HPC and AI to DPUs

  • Ferrol Aderholdt,
  • Zach Tiffany,
  • Rohit Zambre,
  • Manjunath Gorentla Venkata,
  • Yuri Shatsman,
  • Muhammad Abu Saleh,
  • Gil Bloch

摘要

Data processing units (DPUs) represent a significant advancement in networking technology. With their extensive capabilities, DPUs are well-positioned to accelerate AI and HPC workloads by offloading tasks that would otherwise rely on the host for additional resources. However, fully realizing the potential of DPUs is currently hindered by the lack of robust frameworks that facilitate the development of services on these devices. While DPU SDKs like NVIDIA’s DOCA are rapidly evolving, they still fall short of providing seamless offloading to DPUs. To address this gap, we introduce DOCA Unified Resource and Offload Manager (UROM): a framework designed to enable seamless offloading of parallel computing tasks from the host to NVIDIA DPUs. This paper details the architecture of DOCA UROM, highlighting its key components and their functionalities, including resource discovery, host-DPU coordination, and task management on the DPU. We describe how developers can leverage DOCA UROM’s flexible architecture to develop new DPU-based services. To demonstrate its versatility, we showcase three diverse DPU-based services developed using DOCA UROM: RDMOs, progress for non-blocking collectives, and utilizing DPU memory to enhance AI training. Our results illustrate the significant benefits these services offer, underscoring the potential of DOCA UROM in accelerating workloads.