A Novel Multimodal Deep Learning Fusion Framework for Predicting Neovascular Activity Evolution in Exudative Age-Related Macular Degeneration
摘要
Age-related macular degeneration (AMD) is a leading cause of vision loss among older adults, characterized by the progressive deterioration of the macula. Accurate monitoring and prediction of AMD progression is crucial for effective treatment planning and improved patient outcomes. This work presents a novel multimodal fusion deep learning framework designed to automate the analysis of optical coherence tomography (OCT) images for two primary tasks defined in the MICCAI 2024 Monitoring AMD Progression in OCT (MARIO) challenge: (1) classifying changes between 2D OCT B-scans captured at consecutive time points, and (2) predicting the three-month evolution of retinal structure based on OCT data from a single time point. The developed framework leverages finetuned RETFound and EfficientNetV2 models to extract feature representations from OCT B-scans and infrared fundus localizer images, while also incorporating additional clinical variables to enhance prediction accuracy. Extensive experiments conducted on the MARIO challenge dataset demonstrate the effectiveness of the proposed framework, with the best models achieving F1 scores of 0.851 for Task 1 and 0.703 for Task 2. These findings underscore the potential of integrating multimodal data to facilitate an automated analysis and prediction of AMD progression, paving the way for more effective and personalized treatment strategies. Software for the developed framework is available at https://github.com/chrisnielsen/miccai-2024-mario-challenge .