Multi-modal Brain MRI Synthesis with nnU-Net: Exploring Segmentation Performance and Cross-Modality Relationships
摘要
Missing modalities pose a persistent challenge in multi-modal brain MRI analysis, particularly for brain tumor segmentation. While recent methods address cross-modality synthesis, they often emphasize reconstruction fidelity over segmentation effectiveness. We propose a unified framework to assess the segmentation utility of synthetic modalities and their alignment with real counterparts in deep feature space. Using 3D nnU-Net-based translation and segmentation networks, we perform exhaustive pairwise synthesis across four standard MRI sequences (T1n, T1c, T2w and T2f) on a dataset of 1251 glioma patients with full modality coverage and expert-labeled segmentation masks. Each synthetic modality is evaluated using reconstruction metrics (NMSE, SSIM and PSNR), segmentation performance (Dice and HD95), and feature-space similarity (cosine distance, Fréchet distance and L2 norm). This design enables a holistic analysis of whether synthetic modalities retain task-relevant information and how closely they align with real-image embeddings. Our findings offer practical insights into MRI modality imputation, identifying which sequences are more suitable for synthesis and accurate segmentation.