Scholarlink

Protein structure prediction remains a central challenge in computational biology, with significant implications for drug discovery, disease understanding, and biotechnology. Conventional Transformer architectures suffer from scalability limitations due to the quadratic complexity of self-attention. To address this, we propose a method based on the Performer architecture, an efficient Transformer variant that reduces attention complexity to linear time while preserving expressive capacity. Our model incorporates Position-Specific Scoring Matrices (PSSMs) as evolutionary features within a multi-task framework to jointly predict inter-residue distances and dihedral angles. Experiments on the ProteinNet CASP12 dataset demonstrate an RMSE of 4.07 \(^\circ \) for angle prediction and 197.03 Å for distance prediction, with stable convergence achieved within 20 epochs. By combining PSSM features with linear attention, the proposed model effectively processes longer sequences without memory bottlenecks, making it suitable for large-scale protein folding tasks. These results highlight the potential of efficient Transformer architectures in structural biology, offering improved scalability at substantially reduced computational cost.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Efficient Protein Folding with Transformer Models Using the Performer Architecture

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Nikolaos Roufas,
  • Ioannis Karamitsos,
  • Khalil Al-Hussaeni,
  • Vassilis C. Gerogiannis,
  • Andreas Kanavos

摘要

Protein structure prediction remains a central challenge in computational biology, with significant implications for drug discovery, disease understanding, and biotechnology. Conventional Transformer architectures suffer from scalability limitations due to the quadratic complexity of self-attention. To address this, we propose a method based on the Performer architecture, an efficient Transformer variant that reduces attention complexity to linear time while preserving expressive capacity. Our model incorporates Position-Specific Scoring Matrices (PSSMs) as evolutionary features within a multi-task framework to jointly predict inter-residue distances and dihedral angles. Experiments on the ProteinNet CASP12 dataset demonstrate an RMSE of 4.07 \(^\circ \) for angle prediction and 197.03 Å for distance prediction, with stable convergence achieved within 20 epochs. By combining PSSM features with linear attention, the proposed model effectively processes longer sequences without memory bottlenecks, making it suitable for large-scale protein folding tasks. These results highlight the potential of efficient Transformer architectures in structural biology, offering improved scalability at substantially reduced computational cost.