Modern software systems increasingly consist of complex, multi-stage data processing flows that integrate artifacts such as code, database queries, and parameters. Maintaining accurate and up-to-date documentation of such systems is challenging due to the dynamic nature and system evolution. This article presents an LLM-based framework for automating documentation of complex data processing flows. The prototype system leverages modular agents and multi-level caching to generate both task-level and process-level documentation. Evaluation using the LLM-as-a-judge approach demonstrates accurate and coherent results on real-world data processing specifications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automating Documentation of Complex Data Processing Flows with Large Language Models

  • Parisa Mahya,
  • Jorge Martinez-Gil,
  • Mario Winterer,
  • Cornelia Neumüller,
  • Matthias Krump

摘要

Modern software systems increasingly consist of complex, multi-stage data processing flows that integrate artifacts such as code, database queries, and parameters. Maintaining accurate and up-to-date documentation of such systems is challenging due to the dynamic nature and system evolution. This article presents an LLM-based framework for automating documentation of complex data processing flows. The prototype system leverages modular agents and multi-level caching to generate both task-level and process-level documentation. Evaluation using the LLM-as-a-judge approach demonstrates accurate and coherent results on real-world data processing specifications.