The early diagnosis and effective treatment of age-related macular degeneration (AMD), a leading cause of vision impairment, is contingent upon accurate grading. This paper introduces a novel framework, named Mask-UnMask Regions (MUMR), designed to distinguish between normal retina, intermediate AMD, geographic atrophy (GA), and wet AMD using retinal fundus images, with the input resolution standardized to \(1024 \times 1024\) pixels. The framework begins by downscaling images to a quarter of their size using a Preserving High-Frequency Information (PHFI) module, which maintains critical details essential for furthure analysis. Furthermore, we developed a simple, lightweight, yet effective ResNet-like network for efficient feature extraction and introduced a Region Interaction (RI) module, which consists of Adaptive Mask and UnMask Sub-Modules. This module identifies significant regions while reconstructing the insignificant ones using a direction-constrained self-attention mechanism to ensure the learning of global structural cues of AMD grades. The proposed method was evaluated on a dataset of 864 retinal fundus images. Our model consistently achieves superior results compared to other state-of-the-art models, with mean accuracy, mean F1-score, and mean Cohen’s Kappa of 92.55%, 92.59%, and 89.97%, respectively. Additionally, we demonstrate that these results are statistically significant compared to other models based on F1-score, indicating that our proposed framework achieves robust and improved AMD grading performance.