Scholarlink

Background <p>Neoadjuvant therapy (NAC) is a standard treatment for breast cancer, yet only some patients gain significant benefit. Identifying those most likely to benefit from NAC is crucial. Single-modality data often overlook patient heterogeneity, so we developed an interpretable, attention-based multimodal full information feature fusion transformer, MuFi, to predict NAC responses by integrating whole slide images (WSI) and magnetic resonance imaging (MRI).</p> Methods <p>Data from 567 biopsy-confirmed breast cancer patients from two institutions were retrospectively analyzed, with a training cohort (<i>n</i> = 290), validation cohort (<i>n</i> = 73), and external test cohort (<i>n</i> = 204). Multimodal data included pre-treatment pathology slides, MRI scans, and clinical information. A memory-efficient multimodal model was used to fuse WSIs and MRI, with a transformer capturing interactions between histological patches and MRI features.</p> Results <p>MuFi achieved AUCs of 81.9% and 78.5% in discovery and validation cohorts and 79.3% in external testing, outperforming clinical, single-modality and late-fusion-based models. Integrating clinical data (cT and molecular subtype) with MuFi and Feature Re-calibration based Multiple Instance Learning (FRMIL) models further increased AUCs to 90.2%, 81.8%, and 81.6% across the cohorts, indicating enhanced predictive accuracy and generalizability, especially in external testing.</p> Conclusion <p>By fusing pathology and radiology features, MuFi improves decision reliability and identifies critical multimodal predictors. This integration framework better captures patient heterogeneity, supporting personalized NAC decision-making through improved accuracy and generalizability.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Attention-based multimodal fusion transformer for predicting the efficacy of neoadjuvant therapy in breast cancer: a cross-institutional retrospective study

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年12月6日

查看全文
  • Wenchuan Zhang,
  • Shuwan Zhang,
  • Jiadi You,
  • Fengling Li,
  • Xiaoyan Wu,
  • Xunxi Lu,
  • Qingjie Lv,
  • Juan Huang,
  • Yuhao Yi,
  • Hong Bu

摘要

Background

Neoadjuvant therapy (NAC) is a standard treatment for breast cancer, yet only some patients gain significant benefit. Identifying those most likely to benefit from NAC is crucial. Single-modality data often overlook patient heterogeneity, so we developed an interpretable, attention-based multimodal full information feature fusion transformer, MuFi, to predict NAC responses by integrating whole slide images (WSI) and magnetic resonance imaging (MRI).

Methods

Data from 567 biopsy-confirmed breast cancer patients from two institutions were retrospectively analyzed, with a training cohort (n = 290), validation cohort (n = 73), and external test cohort (n = 204). Multimodal data included pre-treatment pathology slides, MRI scans, and clinical information. A memory-efficient multimodal model was used to fuse WSIs and MRI, with a transformer capturing interactions between histological patches and MRI features.

Results

MuFi achieved AUCs of 81.9% and 78.5% in discovery and validation cohorts and 79.3% in external testing, outperforming clinical, single-modality and late-fusion-based models. Integrating clinical data (cT and molecular subtype) with MuFi and Feature Re-calibration based Multiple Instance Learning (FRMIL) models further increased AUCs to 90.2%, 81.8%, and 81.6% across the cohorts, indicating enhanced predictive accuracy and generalizability, especially in external testing.

Conclusion

By fusing pathology and radiology features, MuFi improves decision reliability and identifies critical multimodal predictors. This integration framework better captures patient heterogeneity, supporting personalized NAC decision-making through improved accuracy and generalizability.