Scholarlink

<p>Recent advances in genomic language models have improved the accuracy of <i>in silico</i> analyses, yet many rely on resource-intensive architectures. In this study, we focus on the impact of <i>k</i>-mer tokenization strategies–specifically varying window sizes (three to eight) and overlap schemes–on the performance of transformer-based genomic language models. Through extensive evaluation across multiple plant genomic tasks, including splice site and alternative polyadenylation site prediction, we show that thoughtful design of the <i>k</i>-mer tokenizer plays a critical role in model performance, often outweighing model scale. In particular, overlap-based tokenization generally enhances performance by preserving local sequence context, while certain non-overlap configurations achieve competitive accuracy with improved computational efficiency in some tasks. Despite using a smaller model, our approach performs on par with the state-of-the-art AgroNT model in many cases. These results emphasize that <i>k</i>-mer tokenization, not merely model size, is a key determinant of success in genomic sequence modeling. Our findings provide practical guidance for designing efficient genomic language models tailored to plant biology.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Genomic language models with k-mer tokenization strategies for plant genome annotation and regulatory element strength prediction

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年7月31日

查看全文
  • Shosuke Suzuki,
  • Kazumasa Horie,
  • Toshiyuki Amagasa,
  • Naoya Fukuda

摘要

Recent advances in genomic language models have improved the accuracy of in silico analyses, yet many rely on resource-intensive architectures. In this study, we focus on the impact of k-mer tokenization strategies–specifically varying window sizes (three to eight) and overlap schemes–on the performance of transformer-based genomic language models. Through extensive evaluation across multiple plant genomic tasks, including splice site and alternative polyadenylation site prediction, we show that thoughtful design of the k-mer tokenizer plays a critical role in model performance, often outweighing model scale. In particular, overlap-based tokenization generally enhances performance by preserving local sequence context, while certain non-overlap configurations achieve competitive accuracy with improved computational efficiency in some tasks. Despite using a smaller model, our approach performs on par with the state-of-the-art AgroNT model in many cases. These results emphasize that k-mer tokenization, not merely model size, is a key determinant of success in genomic sequence modeling. Our findings provide practical guidance for designing efficient genomic language models tailored to plant biology.