Scholarlink

<b>Purpose</b> <p>The Jiangmen Underground Neutrino Observatory (JUNO) produces large volumes of detector data that require timely, reliable, and high-quality offline processing to support calibration, reconstruction, and physics analysis. To meet these demands, we developed an automated offline raw data processing system capable of sustaining long-term, high-throughput operations with minimal manual intervention.</p> <b>Methods</b> <p>The system integrates data ingestion, Kafka-based workflow orchestration, computing and storage resources, unified metadata and data management, and monitoring and alerting services. Automated job submission and error recovery mechanisms ensure stable and reliable data processing. The framework manages the entire chain—from raw data ingestion to the production of reconstructed and analysis-ready datasets—with continuous quality control checks embedded throughout the workflow.</p> <b>Results</b> <p>During JUNO’s commissioning and early data-taking periods, the system achieved efficient and reliable processing of large-scale datasets. Performance tests showed stable throughput and rapid recovery from transient failures. The automated monitoring tools reduced operational overhead and improved output consistency.</p> <b>Conclusion</b> <p>The automated offline raw data processing system provides a scalable, flexible, and robust solution tailored to the JUNO experiment. Its architecture supports current operational needs and offers a solid foundation for future upgrades, expanded computing resources, and potential adaptation to other high energy physics experiments.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

An automated offline raw data processing system for the JUNO experiment

  • 期刊论文

  • 发表日期：

    2026年6月16日

查看全文
  • Weiqing Yin,
  • Tao Lin,
  • Yizhou Zhang,
  • Weidong Li,
  • Jiaheng Zou,
  • Shuzhu Jin,
  • Xiaomei Zhang,
  • Jingyan Shi,
  • Shan Zeng,
  • Yujiang Bi,
  • Zhengyun You,
  • Minghua Liao,
  • Yuning Su

摘要

Purpose

The Jiangmen Underground Neutrino Observatory (JUNO) produces large volumes of detector data that require timely, reliable, and high-quality offline processing to support calibration, reconstruction, and physics analysis. To meet these demands, we developed an automated offline raw data processing system capable of sustaining long-term, high-throughput operations with minimal manual intervention.

Methods

The system integrates data ingestion, Kafka-based workflow orchestration, computing and storage resources, unified metadata and data management, and monitoring and alerting services. Automated job submission and error recovery mechanisms ensure stable and reliable data processing. The framework manages the entire chain—from raw data ingestion to the production of reconstructed and analysis-ready datasets—with continuous quality control checks embedded throughout the workflow.

Results

During JUNO’s commissioning and early data-taking periods, the system achieved efficient and reliable processing of large-scale datasets. Performance tests showed stable throughput and rapid recovery from transient failures. The automated monitoring tools reduced operational overhead and improved output consistency.

Conclusion

The automated offline raw data processing system provides a scalable, flexible, and robust solution tailored to the JUNO experiment. Its architecture supports current operational needs and offers a solid foundation for future upgrades, expanded computing resources, and potential adaptation to other high energy physics experiments.