Scholarlink

Testbeds for computing technologies play a crucial role in evaluating novel architectures and optimizing the user experience is critical to enabling meaningful and efficient use and testing of the platform. In this paper, we share our experience with Ookami, a computing technology testbed that has been operational for over four years, serving more than 500 users. Due to the novel nature of its technology and architecture, typical usage patterns differ significantly from those observed in conventional high-performance computing (HPC) systems. To monitor system performance and user interactions, we implemented a tracking approach that helps us assess project progress, identify beneficial and non-beneficial decisions, and continuously refine both system usage and user experience. Based on our findings, we provide recommendations for other testbed centers on key metrics to monitor and how to leverage collected data for system and user optimization. Additionally, we highlight aspects that in hindsight, would have been helpful, offering insights for future testbed development. This work serves as a practical guide for institutions managing similar testbeds, helping them make informed decisions on data collection, analysis, and operational improvements.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

What Time Taught Us: Monitoring a Computing Technology Testbed Across Multiple Years

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Eva Siegmann,
  • David Carlson,
  • Nikolay A. Simakov,
  • Anthony Curtis,
  • Alan Calder,
  • Robert J. Harrison

摘要

Testbeds for computing technologies play a crucial role in evaluating novel architectures and optimizing the user experience is critical to enabling meaningful and efficient use and testing of the platform. In this paper, we share our experience with Ookami, a computing technology testbed that has been operational for over four years, serving more than 500 users. Due to the novel nature of its technology and architecture, typical usage patterns differ significantly from those observed in conventional high-performance computing (HPC) systems. To monitor system performance and user interactions, we implemented a tracking approach that helps us assess project progress, identify beneficial and non-beneficial decisions, and continuously refine both system usage and user experience. Based on our findings, we provide recommendations for other testbed centers on key metrics to monitor and how to leverage collected data for system and user optimization. Additionally, we highlight aspects that in hindsight, would have been helpful, offering insights for future testbed development. This work serves as a practical guide for institutions managing similar testbeds, helping them make informed decisions on data collection, analysis, and operational improvements.