Emerging multi-hundred gigabit Ethernet speeds are outpacing improvements in CPU performance and memory bandwidth, challenging host-based I/O processing capacity. SmartNICs, such as NVIDIA BlueField, promise to overcome this challenge by offloading network-intensive computation, freeing up host resources. We show, however, that SmartNICs fall short of this goal, offloading only a small portion of the host CPU and wire bandwidth for common tasks—BlueField-2 NVMe-over-TCP storage disaggregation offloads up to 4 host cores while BlueField-3 achieves up to 14 cores. Prior work attributes this limitation to weaker SmartNIC cores; in contrast, in this work we identify SmartNIC memory bandwidth as the key bottleneck to line-rate performance. We then leverage SmartNIC support for direct cache access to overcome this bottleneck by constraining I/O buffers to the last-level cache (LLC). Our evaluation shows the benefits of this approach by improving BlueField-2 and BlueField-3 throughput on the previous benchmark by up to 56% and 20%, respectively.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Characterizing Bluefields’ Memory Bandwidth Bottlenecks

  • Michał Piotr Podleś,
  • Idelfonso Tafur Monroy,
  • Juan Jose Vegas Olmos,
  • Boris Pismenny

摘要

Emerging multi-hundred gigabit Ethernet speeds are outpacing improvements in CPU performance and memory bandwidth, challenging host-based I/O processing capacity. SmartNICs, such as NVIDIA BlueField, promise to overcome this challenge by offloading network-intensive computation, freeing up host resources. We show, however, that SmartNICs fall short of this goal, offloading only a small portion of the host CPU and wire bandwidth for common tasks—BlueField-2 NVMe-over-TCP storage disaggregation offloads up to 4 host cores while BlueField-3 achieves up to 14 cores. Prior work attributes this limitation to weaker SmartNIC cores; in contrast, in this work we identify SmartNIC memory bandwidth as the key bottleneck to line-rate performance. We then leverage SmartNIC support for direct cache access to overcome this bottleneck by constraining I/O buffers to the last-level cache (LLC). Our evaluation shows the benefits of this approach by improving BlueField-2 and BlueField-3 throughput on the previous benchmark by up to 56% and 20%, respectively.