In this short paper, we present an innovative approach to limit the required bandwidth when transferring data during in transit analysis. This approach is called hybrid because it combines existing in situ and in transit solutions. It leverages the stable ABI of Catalyst version 2 and the Catalyst-ADIOS2 implementation to seamlessly switch from in situ, in transit and hybrid analysis without modifying the numerical simulation code. The typical use case is to perform data reduction in situ then generate a visualization in transit on the reduced data. This approach makes the numerical simulation workflows very flexible depending on the size of the data, the available computing resources or the analysis type. Our experiment with this hybrid approach, reducing data before sending it, demonstrated large cost reductions for some visualization pipelines compared to in situ and in transit solutions. The implementation is available under an open source permissive license to be usable broadly in any scientific community.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

In Situ in Transit Hybrid Analysis with Catalyst-ADIOS2

  • François Mazen,
  • Louis Gombert,
  • Lucas Givord,
  • Charles Gueunet

摘要

In this short paper, we present an innovative approach to limit the required bandwidth when transferring data during in transit analysis. This approach is called hybrid because it combines existing in situ and in transit solutions. It leverages the stable ABI of Catalyst version 2 and the Catalyst-ADIOS2 implementation to seamlessly switch from in situ, in transit and hybrid analysis without modifying the numerical simulation code. The typical use case is to perform data reduction in situ then generate a visualization in transit on the reduced data. This approach makes the numerical simulation workflows very flexible depending on the size of the data, the available computing resources or the analysis type. Our experiment with this hybrid approach, reducing data before sending it, demonstrated large cost reductions for some visualization pipelines compared to in situ and in transit solutions. The implementation is available under an open source permissive license to be usable broadly in any scientific community.