Artificial intelligence (AI) and deep learning (DL) are widespread in healthcare systems and medical decision-making. These recent AI applications offer advanced diagnostics, treatment planning, and patient monitoring, enhancing patient treatment and care. The fact that DL models are “black boxes,” with their inner functioning unknown, is a significant barrier. This nature raises concerns about reliability, transparency, and trust, especially for life-saving decisions in Healthcare. The explainable AI (XAI) paradigm has been proposed and employed recently to tackle these issues. These techniques deliver explainability of the decision-making processes of AI systems, resulting in better understanding among clinicians and professionals. This chapter explores the evolution, techniques, applications, and state-of-the-art studies of XAI in medical decision-making. The XAI-enabled medical practices, challenges, and avenues for future research in this domain are discussed.