Scholarlink

<p>Immune repertoire classification (IRC) is a typical multi-instance learning (MIL) problem that is widely applied in fields such as vaccine development and drug discovery in computational biology. In existing MIL-IRC methods, the input modalities consist of amino acid (AA) sequences and V(D)J genes. However, common practices typically involve directly connecting the two modalities or treating them equally by merging them in the hidden space. This leads to the problem of modality laziness, where some modalities appear more dominant than others during multi-modal learning. To address this problem, this paper proposes a two-stage IRC method, called Overcoming Modality Laziness in Multi-modal Multi-instance Learning (OML-M3IL). In stage 1, we use alternating unimodal adaptation to learn from the two modalities separately, ensuring that the shared classifier works reliably with each modality alone. In stage 2, we also use a similarity loss to reduce the difference between modalities and fuse them. In addition, gradient orthogonalization, label disambiguation, and dynamic fusion prediction techniques are used to perform the IRC task more efficiently. Experiments were conducted on CMV and Cancer datasets, and the results verify that OML-M3IL yields consistently competitive or better performance than existing MIL-IRC methods while explicitly mitigating modality laziness.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Overcoming modality laziness in multi-modal multi-instance learning for immune repertoire classification

  • 期刊论文

  • 发表日期：

    2026年6月25日

查看全文
  • Yu-Xuan Zhang,
  • Zhengchun Zhou,
  • Hanjie Luo,
  • Weisha Liu,
  • Ming Li

摘要

Immune repertoire classification (IRC) is a typical multi-instance learning (MIL) problem that is widely applied in fields such as vaccine development and drug discovery in computational biology. In existing MIL-IRC methods, the input modalities consist of amino acid (AA) sequences and V(D)J genes. However, common practices typically involve directly connecting the two modalities or treating them equally by merging them in the hidden space. This leads to the problem of modality laziness, where some modalities appear more dominant than others during multi-modal learning. To address this problem, this paper proposes a two-stage IRC method, called Overcoming Modality Laziness in Multi-modal Multi-instance Learning (OML-M3IL). In stage 1, we use alternating unimodal adaptation to learn from the two modalities separately, ensuring that the shared classifier works reliably with each modality alone. In stage 2, we also use a similarity loss to reduce the difference between modalities and fuse them. In addition, gradient orthogonalization, label disambiguation, and dynamic fusion prediction techniques are used to perform the IRC task more efficiently. Experiments were conducted on CMV and Cancer datasets, and the results verify that OML-M3IL yields consistently competitive or better performance than existing MIL-IRC methods while explicitly mitigating modality laziness.