In just a few years after they first appeared, large language models (LLMs) have brought a broad range of potential and promise, making ambitious and creative ideas in any field feasible and reachable far more easily and quickly than previously could have been imagined. The healthcare domain is no exception, as the integration of LLMs into healthcare can potentially revolutionize clinical workflows, from diagnostics to patient education. However, their "black box" nature, i.e., not knowing exactly what path they take to reach the output they generate, makes their adoption impossible without achieving explainability. Explainability and the ability to interpret LLM models’ outcomes is the main principle for transparency, building trust, ensuring patient safety, and following standards and regulations. This chapter provides a comprehensive overview of core techniques for explaining LLMs in clinical applications, broken down into three general categories: (i) post-hoc techniques, (ii) prompting-based techniques, and (iii) augmentation methods and model architectures. We review all key aspects of each technique and encompass notable implementations and applications of each in the healthcare domain. Finally, we also review the procedures and key insights regarding the evaluation of the LLMs and their explanations.