Scholarlink

<p>Scientific Workflow Systems (SWSs) are advanced software frameworks that drive modern research by orchestrating complex computational tasks and managing extensive data pipelines. These systems offer a range of essential features, including modularity, abstraction, customization, interoperability, workflow composition tools, resource management, error handling, and comprehensive documentation. Utilizing these frameworks accelerates the development of scientific computing, resulting in more efficient and reproducible research outcomes. Despite their significance, developing a user-friendly, efficient, and adaptable SWS poses several challenges that are not always well-documented or understood. This study explores these challenges through an in-depth analysis of interactions on Stack Overflow (SO) and GitHub, key platforms where developers and researchers discuss and resolve issues. In particular, we leveraged topic modeling (BERTopic) to understand the topics SWSs developers discuss on these platforms. Then, we examined the popularity and difficulty of those topics. We identified 10 topics developers discuss on SO (e.g., Workflow Creation and Scheduling, Data Structures and Operations, Workflow Execution) and found that workflow execution is the most challenging among them. By analyzing GitHub issues, we identified 13 topics (e.g., Errors and Bug Fixing, Documentation, Dependencies) and discovered that errors and bug fixing is the most dominant topics in this context. We found system redesign and API migration to be the most challenging topics utilizing GitHub data. A cross-platform comparison revealed overlapping concerns such as task management and data operations. Additionally, we categorized each topic by type (How, Why, What, and Others) and observed that the How type consistently dominates across all topics, indicating a need for procedural guidance among developers. This dominance of the How type is also prevalent in other domains, such as Chatbots and Mobile development. We believe that our study will guide future research in proposing tools and techniques to help the community overcome the challenges developers face when developing SWSs.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

An Empirical Investigation on the Challenges in Scientific Workflow Systems Development

  • 期刊论文

  • 发表日期：

    2025年8月1日

查看全文
  • Khairul Alam,
  • Banani Roy,
  • Chanchal K. Roy,
  • Kartik Mittal

摘要

Scientific Workflow Systems (SWSs) are advanced software frameworks that drive modern research by orchestrating complex computational tasks and managing extensive data pipelines. These systems offer a range of essential features, including modularity, abstraction, customization, interoperability, workflow composition tools, resource management, error handling, and comprehensive documentation. Utilizing these frameworks accelerates the development of scientific computing, resulting in more efficient and reproducible research outcomes. Despite their significance, developing a user-friendly, efficient, and adaptable SWS poses several challenges that are not always well-documented or understood. This study explores these challenges through an in-depth analysis of interactions on Stack Overflow (SO) and GitHub, key platforms where developers and researchers discuss and resolve issues. In particular, we leveraged topic modeling (BERTopic) to understand the topics SWSs developers discuss on these platforms. Then, we examined the popularity and difficulty of those topics. We identified 10 topics developers discuss on SO (e.g., Workflow Creation and Scheduling, Data Structures and Operations, Workflow Execution) and found that workflow execution is the most challenging among them. By analyzing GitHub issues, we identified 13 topics (e.g., Errors and Bug Fixing, Documentation, Dependencies) and discovered that errors and bug fixing is the most dominant topics in this context. We found system redesign and API migration to be the most challenging topics utilizing GitHub data. A cross-platform comparison revealed overlapping concerns such as task management and data operations. Additionally, we categorized each topic by type (How, Why, What, and Others) and observed that the How type consistently dominates across all topics, indicating a need for procedural guidance among developers. This dominance of the How type is also prevalent in other domains, such as Chatbots and Mobile development. We believe that our study will guide future research in proposing tools and techniques to help the community overcome the challenges developers face when developing SWSs.