While Positron Emission Tomography (PET) is considered the gold standard for early AD diagnosis, its availability is limited due to high cost and radiation exposure risk, making it less accessible compared to more widely available modalities like tabular data and MRI. Therefore, effectively synthesizing PET features from available modalities presents a promising alternative and is of significant interest. In this work, we propose a novel multi-modal framework for synthesizing PET features based on tabular data-enhanced alignment. Our model requires only tabular and MRI data during the inference stage, yet achieves substantial improvement using synthesized PET features. Specifically, our framework consists of two stages. In the first stage, the model is pre-trained using multimodal data, with a tabular data-guided contrastive learning scheme designed to align features across different modalities. In the second stage, tabular data-guided Transformer blocks are used to synthesize PET features from MRI and tabular data based on the aligned encoders trained in the first stage. The synthesized PET features with tabular data and MRI, are then integrated for early AD diagnosis. Experimental results show that our model outperforms related state-of-the-art methods. This approach holds great promise for enhancing diagnostic accuracy and efficiency in AD diagnosis.Our code is available at https://github.com/internbob/Tab-s-AD .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Tabular Data-Enhanced Multi-modal Alignment and Synthesis for Alzheimer’s Disease Diagnosis

  • Weilin Zhou,
  • Yuxiao Liu,
  • Yuanwang Zhang,
  • Kaicong Sun,
  • Fan Li,
  • Shilun Zhao,
  • Yuanbo Wang,
  • Dinggang Shen

摘要

While Positron Emission Tomography (PET) is considered the gold standard for early AD diagnosis, its availability is limited due to high cost and radiation exposure risk, making it less accessible compared to more widely available modalities like tabular data and MRI. Therefore, effectively synthesizing PET features from available modalities presents a promising alternative and is of significant interest. In this work, we propose a novel multi-modal framework for synthesizing PET features based on tabular data-enhanced alignment. Our model requires only tabular and MRI data during the inference stage, yet achieves substantial improvement using synthesized PET features. Specifically, our framework consists of two stages. In the first stage, the model is pre-trained using multimodal data, with a tabular data-guided contrastive learning scheme designed to align features across different modalities. In the second stage, tabular data-guided Transformer blocks are used to synthesize PET features from MRI and tabular data based on the aligned encoders trained in the first stage. The synthesized PET features with tabular data and MRI, are then integrated for early AD diagnosis. Experimental results show that our model outperforms related state-of-the-art methods. This approach holds great promise for enhancing diagnostic accuracy and efficiency in AD diagnosis.Our code is available at https://github.com/internbob/Tab-s-AD .