MM + CD Fusion: Deep Learning-based 3D Multi-Modal Fusion for Early Pathological Complete Response Prediction in Breast Cancer
摘要
This study aims to evaluate the predictive potential of baseline \(^{18}\) F-Fluorodeoxyglucose Positron Emission Tomography/Computed Tomography ( \(^{18}\) F-FDG PET/CT), clinical data (CD), dynamic contrast-enhanced MRI (DCE-MRI), and T1-weighted MRI (T1w-MRI) parameters, including advanced texture features, for assessing pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) in patients with advanced breast cancer (BC), using a cohort of 81 BC patients. Accurate identification of patients who achieve pCR to NAC is crucial for guiding personalized treatment strategies and assessing prognosis in breast cancer. However, most existing pCR prediction methods rely on single-modality data, which limits their ability to capture tumor dynamics and fully represent tumor heterogeneity at both macro and micro levels. Moreover, these approaches often fail to effectively integrate the complementary information available across imaging modalities. In this study, we propose MM (multi-modal) + CD Fusion, a framework that pioneers the integration of MM imaging and CD for enhanced treatment response prediction. Our approach consists of two core components: (1) MM imaging, which includes \(^{18}\) F-FDG PET/CT and multiparametric MRI (DCE and T1w), offering comprehensive metabolic and anatomical representations of breast cancer lesions and their NAC-induced changes; and (2) CD, which incorporates clinical variables characterizing tumor biology. We introduce label propagation techniques to enable automatic lesion segmentation across all MM images using a 3D U-Net deep learning model, followed by pCR prediction through machine learning algorithms. The proposed MM + CD fusion approach significantly outperforms unimodal methods in predicting pCR (AUC: 0.87 vs. 0.63–0.77, \(p<0.012\) ) using 141 optimally selected radiomic features. This improvement supports the potential of MM + CD fusion for more personalized therapy planning in breast cancer patients.