Functional magnetic resonance imaging (fMRI), a noninvasive neuroimaging technique for mapping neural activity, has demonstrated substantial potential in identifying brain disease. However, clinical applications face a critical challenge: patient data are typically scarce compared to abundant healthy control samples. This severe class imbalance significantly limits the performance of classification-based diagnostic models. To address this issue, we propose the Region-Specific Anomaly Detection (RSAD) framework, which formulates the brain disease identification as an anomaly detection task. We first employ pre-training to capture normal patterns of healthy data through a reconstruction task, and then develop the discrepancy score to enhance the model’s ability to perceive potential abnormalities, thereby improving the AD performance. Specifically, we design an affinity matrix learning module and an adaptive region of interest (ROI) masking strategy to improve the performance of signal representation learning. Additionally, we propose a region-based discrepancy score weighting strategy to amplify the distinction between potential abnormalities and healthy controls by assigning higher weights to key brain regions, thereby improving the model’s ability to detect anomalies. We conduct experiments across six different brain diseases, and the superior results demonstrate that RSAD effectively enables disease diagnosis, even with extreme class imbalance. Our code is available at https://github.com/kylin1112/RSAD .