<p>Medical datasets often lack high-quality annotated images, making it difficult to apply traditional deep learning techniques to create an unbiased model with strong generalization capabilities. To tackle this issue, we redefine the problem as a few-shot learning challenge and propose a meta-learning-based solution called “DerMeL.” This research presents a lightweight meta-learning algorithm designed to classify rare skin diseases in long-tailed distributions and resource-constrained environments. DerMeL employs first-order gradient-based FOMAML networks for disease identification in skin lesion datasets to ensure the approach remains lightweight. The method involves model pruning during the meta training phase to retain only common essential part of the model. The deleted connections are utilized during the meta testing phase to learn specific features of a rare disease in a limited dataset regime. DerMeL works best for rare disease prediction because pruning removes redundant or less crucial connections and parameters. This results in a concise and compact model. This size reduction can notably reduce memory demands, enhancing the model’s efficiency for deployment on devices or platforms with limited resources. Moreover, reconnecting these deleted connections allows quick learning of rare disease conditions in the meta test phase. The results show an increase in few-shot classification scores by approximately 2% for 5-shot task settings compared to the benchmark first-order meta-learning algorithms on publicly available datasets (Derm7pt and ISIC 2018) that demonstrate the efficacy of DerMeL in tackling healthcare challenges posed by limited data availability for rare diseases.</p>