Scholarlink

This paper addresses the challenge of efficiently training Large Language Models (LLMs) on large-scale, sparse omics datasets in high-performance computing (HPC) environments. Using over 1000 BED tracks as a representative data source, we propose a method combining interval-based chunked storage, sparse matrix transformation, and parallel data loading, integrated within a PyTorch Lightning training framework. Our approach minimizes CPU overhead and disk I/O while preserving data sparsity throughout training. Evaluated on multiple GPU configurations (V100, A100, H100), the method achieves over 85% GPU utilization with less than 20% CPU overhead, demonstrating scalability, adaptability, and potential to significantly accelerate LLM applications in bioinformatics.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Optimizing Computational Infrastructure for Large Language Models in Bioinformatics: A Case Study

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Nazar Beknazarov

摘要

This paper addresses the challenge of efficiently training Large Language Models (LLMs) on large-scale, sparse omics datasets in high-performance computing (HPC) environments. Using over 1000 BED tracks as a representative data source, we propose a method combining interval-based chunked storage, sparse matrix transformation, and parallel data loading, integrated within a PyTorch Lightning training framework. Our approach minimizes CPU overhead and disk I/O while preserving data sparsity throughout training. Evaluated on multiple GPU configurations (V100, A100, H100), the method achieves over 85% GPU utilization with less than 20% CPU overhead, demonstrating scalability, adaptability, and potential to significantly accelerate LLM applications in bioinformatics.